public inbox for [email protected]help / color / mirror / Atom feed
[PATCH v7 7/9] Update comment obsolete since 69c3936a 35+ messages / 2 participants [nested] [flat]
* [PATCH v7 7/9] Update comment obsolete since 69c3936a @ 2020-02-15 21:53 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Justin Pryzby @ 2020-02-15 21:53 UTC (permalink / raw) --- src/backend/executor/nodeAgg.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index 867da6eebf..9aa2941457 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -1538,8 +1538,7 @@ lookup_hash_entry(AggState *aggstate, uint32 hash) } /* - * Look up hash entries for the current tuple in all hashed grouping sets, - * returning an array of pergroup pointers suitable for advance_aggregates. + * Look up hash entries for the current tuple in all hashed grouping sets. * * Be aware that lookup_hash_entry can reset the tmpcontext. */ -- 2.17.0 --UfEAyuTBtIjiZzX6 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v7-0008-Add-explain-MACHINE.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH v3 7/7] Update comment obsolete since 69c3936a @ 2020-02-15 21:53 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Justin Pryzby @ 2020-02-15 21:53 UTC (permalink / raw) --- src/backend/executor/nodeAgg.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index 396446b..472b2b1 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -1499,8 +1499,7 @@ lookup_hash_entry(AggState *aggstate) } /* - * Look up hash entries for the current tuple in all hashed grouping sets, - * returning an array of pergroup pointers suitable for advance_aggregates. + * Look up hash entries for the current tuple in all hashed grouping sets. * * Be aware that lookup_hash_entry can reset the tmpcontext. */ -- 2.7.4 --SNIs70sCzqvszXB4-- ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH v4 7/7] Update comment obsolete since 69c3936a @ 2020-02-15 21:53 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Justin Pryzby @ 2020-02-15 21:53 UTC (permalink / raw) --- src/backend/executor/nodeAgg.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index f432825..1faea5b 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -1530,8 +1530,7 @@ lookup_hash_entry(AggState *aggstate, uint32 hash) } /* - * Look up hash entries for the current tuple in all hashed grouping sets, - * returning an array of pergroup pointers suitable for advance_aggregates. + * Look up hash entries for the current tuple in all hashed grouping sets. * * Be aware that lookup_hash_entry can reset the tmpcontext. */ -- 2.7.4 --jI8keyz6grp/JLjh-- ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH v5 7/7] Update comment obsolete since 69c3936a @ 2020-02-15 21:53 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Justin Pryzby @ 2020-02-15 21:53 UTC (permalink / raw) --- src/backend/executor/nodeAgg.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index c338ace..e5cd87d 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -1535,8 +1535,7 @@ lookup_hash_entry(AggState *aggstate, uint32 hash) } /* - * Look up hash entries for the current tuple in all hashed grouping sets, - * returning an array of pergroup pointers suitable for advance_aggregates. + * Look up hash entries for the current tuple in all hashed grouping sets. * * Be aware that lookup_hash_entry can reset the tmpcontext. */ -- 2.7.4 --M0YLxmUXciMpOLPE-- ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH v9 8/8] Update comment obsolete since 69c3936a @ 2020-02-15 21:53 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Justin Pryzby @ 2020-02-15 21:53 UTC (permalink / raw) --- src/backend/executor/nodeAgg.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index c6d03521e4..5cdd92acaf 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -2058,8 +2058,7 @@ lookup_hash_entry(AggState *aggstate, uint32 hash, bool *in_hash_table) } /* - * Look up hash entries for the current tuple in all hashed grouping sets, - * returning an array of pergroup pointers suitable for advance_aggregates. + * Look up hash entries for the current tuple in all hashed grouping sets. * * Be aware that lookup_hash_entry can reset the tmpcontext. * -- 2.17.0 --wwtQuX191/I956S7-- ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH v6 7/7] Update comment obsolete since 69c3936a @ 2020-02-15 21:53 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Justin Pryzby @ 2020-02-15 21:53 UTC (permalink / raw) --- src/backend/executor/nodeAgg.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index 3d60f7a314..567d65bb6c 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -1538,8 +1538,7 @@ lookup_hash_entry(AggState *aggstate, uint32 hash) } /* - * Look up hash entries for the current tuple in all hashed grouping sets, - * returning an array of pergroup pointers suitable for advance_aggregates. + * Look up hash entries for the current tuple in all hashed grouping sets. * * Be aware that lookup_hash_entry can reset the tmpcontext. */ -- 2.17.0 --xHbokkKX1kTiQeDC-- ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH 02/18] Update comment obsolete since 69c3936a @ 2020-02-15 21:53 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Justin Pryzby @ 2020-02-15 21:53 UTC (permalink / raw) --- src/backend/executor/nodeAgg.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index 601b6dab03..394b4e667b 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -2064,8 +2064,7 @@ initialize_hash_entry(AggState *aggstate, TupleHashTable hashtable, } /* - * Look up hash entries for the current tuple in all hashed grouping sets, - * returning an array of pergroup pointers suitable for advance_aggregates. + * Look up hash entries for the current tuple in all hashed grouping sets. * * Be aware that lookup_hash_entry can reset the tmpcontext. * -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0003-pg_restore-must-be-specified-and-list.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH 02/18] Update comment obsolete since 69c3936a @ 2020-02-15 21:53 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Justin Pryzby @ 2020-02-15 21:53 UTC (permalink / raw) --- src/backend/executor/nodeAgg.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index 601b6dab03..394b4e667b 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -2064,8 +2064,7 @@ initialize_hash_entry(AggState *aggstate, TupleHashTable hashtable, } /* - * Look up hash entries for the current tuple in all hashed grouping sets, - * returning an array of pergroup pointers suitable for advance_aggregates. + * Look up hash entries for the current tuple in all hashed grouping sets. * * Be aware that lookup_hash_entry can reset the tmpcontext. * -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0003-pg_restore-must-be-specified-and-list.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH 02/18] Update comment obsolete since 69c3936a @ 2020-02-15 21:53 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Justin Pryzby @ 2020-02-15 21:53 UTC (permalink / raw) --- src/backend/executor/nodeAgg.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index 601b6dab03..394b4e667b 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -2064,8 +2064,7 @@ initialize_hash_entry(AggState *aggstate, TupleHashTable hashtable, } /* - * Look up hash entries for the current tuple in all hashed grouping sets, - * returning an array of pergroup pointers suitable for advance_aggregates. + * Look up hash entries for the current tuple in all hashed grouping sets. * * Be aware that lookup_hash_entry can reset the tmpcontext. * -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0003-pg_restore-must-be-specified-and-list.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH 02/18] Update comment obsolete since 69c3936a @ 2020-02-15 21:53 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Justin Pryzby @ 2020-02-15 21:53 UTC (permalink / raw) --- src/backend/executor/nodeAgg.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index 601b6dab03..394b4e667b 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -2064,8 +2064,7 @@ initialize_hash_entry(AggState *aggstate, TupleHashTable hashtable, } /* - * Look up hash entries for the current tuple in all hashed grouping sets, - * returning an array of pergroup pointers suitable for advance_aggregates. + * Look up hash entries for the current tuple in all hashed grouping sets. * * Be aware that lookup_hash_entry can reset the tmpcontext. * -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0003-pg_restore-must-be-specified-and-list.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH 02/18] Update comment obsolete since 69c3936a @ 2020-02-15 21:53 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Justin Pryzby @ 2020-02-15 21:53 UTC (permalink / raw) --- src/backend/executor/nodeAgg.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index 601b6dab03..394b4e667b 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -2064,8 +2064,7 @@ initialize_hash_entry(AggState *aggstate, TupleHashTable hashtable, } /* - * Look up hash entries for the current tuple in all hashed grouping sets, - * returning an array of pergroup pointers suitable for advance_aggregates. + * Look up hash entries for the current tuple in all hashed grouping sets. * * Be aware that lookup_hash_entry can reset the tmpcontext. * -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0003-pg_restore-must-be-specified-and-list.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH 02/18] Update comment obsolete since 69c3936a @ 2020-02-15 21:53 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Justin Pryzby @ 2020-02-15 21:53 UTC (permalink / raw) --- src/backend/executor/nodeAgg.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index 601b6dab03..394b4e667b 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -2064,8 +2064,7 @@ initialize_hash_entry(AggState *aggstate, TupleHashTable hashtable, } /* - * Look up hash entries for the current tuple in all hashed grouping sets, - * returning an array of pergroup pointers suitable for advance_aggregates. + * Look up hash entries for the current tuple in all hashed grouping sets. * * Be aware that lookup_hash_entry can reset the tmpcontext. * -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0003-pg_restore-must-be-specified-and-list.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH 02/18] Update comment obsolete since 69c3936a @ 2020-02-15 21:53 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Justin Pryzby @ 2020-02-15 21:53 UTC (permalink / raw) --- src/backend/executor/nodeAgg.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index 601b6dab03..394b4e667b 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -2064,8 +2064,7 @@ initialize_hash_entry(AggState *aggstate, TupleHashTable hashtable, } /* - * Look up hash entries for the current tuple in all hashed grouping sets, - * returning an array of pergroup pointers suitable for advance_aggregates. + * Look up hash entries for the current tuple in all hashed grouping sets. * * Be aware that lookup_hash_entry can reset the tmpcontext. * -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0003-pg_restore-must-be-specified-and-list.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH 02/18] Update comment obsolete since 69c3936a @ 2020-02-15 21:53 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Justin Pryzby @ 2020-02-15 21:53 UTC (permalink / raw) --- src/backend/executor/nodeAgg.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index 601b6dab03..394b4e667b 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -2064,8 +2064,7 @@ initialize_hash_entry(AggState *aggstate, TupleHashTable hashtable, } /* - * Look up hash entries for the current tuple in all hashed grouping sets, - * returning an array of pergroup pointers suitable for advance_aggregates. + * Look up hash entries for the current tuple in all hashed grouping sets. * * Be aware that lookup_hash_entry can reset the tmpcontext. * -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0003-pg_restore-must-be-specified-and-list.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH 02/18] Update comment obsolete since 69c3936a @ 2020-02-15 21:53 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Justin Pryzby @ 2020-02-15 21:53 UTC (permalink / raw) --- src/backend/executor/nodeAgg.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index 601b6dab03..394b4e667b 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -2064,8 +2064,7 @@ initialize_hash_entry(AggState *aggstate, TupleHashTable hashtable, } /* - * Look up hash entries for the current tuple in all hashed grouping sets, - * returning an array of pergroup pointers suitable for advance_aggregates. + * Look up hash entries for the current tuple in all hashed grouping sets. * * Be aware that lookup_hash_entry can reset the tmpcontext. * -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0003-pg_restore-must-be-specified-and-list.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH 02/18] Update comment obsolete since 69c3936a @ 2020-02-15 21:53 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Justin Pryzby @ 2020-02-15 21:53 UTC (permalink / raw) --- src/backend/executor/nodeAgg.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index 601b6dab03..394b4e667b 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -2064,8 +2064,7 @@ initialize_hash_entry(AggState *aggstate, TupleHashTable hashtable, } /* - * Look up hash entries for the current tuple in all hashed grouping sets, - * returning an array of pergroup pointers suitable for advance_aggregates. + * Look up hash entries for the current tuple in all hashed grouping sets. * * Be aware that lookup_hash_entry can reset the tmpcontext. * -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0003-pg_restore-must-be-specified-and-list.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH 02/18] Update comment obsolete since 69c3936a @ 2020-02-15 21:53 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Justin Pryzby @ 2020-02-15 21:53 UTC (permalink / raw) --- src/backend/executor/nodeAgg.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index 601b6dab03..394b4e667b 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -2064,8 +2064,7 @@ initialize_hash_entry(AggState *aggstate, TupleHashTable hashtable, } /* - * Look up hash entries for the current tuple in all hashed grouping sets, - * returning an array of pergroup pointers suitable for advance_aggregates. + * Look up hash entries for the current tuple in all hashed grouping sets. * * Be aware that lookup_hash_entry can reset the tmpcontext. * -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0003-pg_restore-must-be-specified-and-list.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH 02/18] Update comment obsolete since 69c3936a @ 2020-02-15 21:53 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Justin Pryzby @ 2020-02-15 21:53 UTC (permalink / raw) --- src/backend/executor/nodeAgg.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index 601b6dab03..394b4e667b 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -2064,8 +2064,7 @@ initialize_hash_entry(AggState *aggstate, TupleHashTable hashtable, } /* - * Look up hash entries for the current tuple in all hashed grouping sets, - * returning an array of pergroup pointers suitable for advance_aggregates. + * Look up hash entries for the current tuple in all hashed grouping sets. * * Be aware that lookup_hash_entry can reset the tmpcontext. * -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0003-pg_restore-must-be-specified-and-list.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH 02/18] Update comment obsolete since 69c3936a @ 2020-02-15 21:53 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Justin Pryzby @ 2020-02-15 21:53 UTC (permalink / raw) --- src/backend/executor/nodeAgg.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index 601b6dab03..394b4e667b 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -2064,8 +2064,7 @@ initialize_hash_entry(AggState *aggstate, TupleHashTable hashtable, } /* - * Look up hash entries for the current tuple in all hashed grouping sets, - * returning an array of pergroup pointers suitable for advance_aggregates. + * Look up hash entries for the current tuple in all hashed grouping sets. * * Be aware that lookup_hash_entry can reset the tmpcontext. * -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0003-pg_restore-must-be-specified-and-list.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH 02/18] Update comment obsolete since 69c3936a @ 2020-02-15 21:53 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Justin Pryzby @ 2020-02-15 21:53 UTC (permalink / raw) --- src/backend/executor/nodeAgg.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index 601b6dab03..394b4e667b 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -2064,8 +2064,7 @@ initialize_hash_entry(AggState *aggstate, TupleHashTable hashtable, } /* - * Look up hash entries for the current tuple in all hashed grouping sets, - * returning an array of pergroup pointers suitable for advance_aggregates. + * Look up hash entries for the current tuple in all hashed grouping sets. * * Be aware that lookup_hash_entry can reset the tmpcontext. * -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0003-pg_restore-must-be-specified-and-list.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH 02/18] Update comment obsolete since 69c3936a @ 2020-02-15 21:53 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Justin Pryzby @ 2020-02-15 21:53 UTC (permalink / raw) --- src/backend/executor/nodeAgg.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index 601b6dab03..394b4e667b 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -2064,8 +2064,7 @@ initialize_hash_entry(AggState *aggstate, TupleHashTable hashtable, } /* - * Look up hash entries for the current tuple in all hashed grouping sets, - * returning an array of pergroup pointers suitable for advance_aggregates. + * Look up hash entries for the current tuple in all hashed grouping sets. * * Be aware that lookup_hash_entry can reset the tmpcontext. * -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0003-pg_restore-must-be-specified-and-list.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH 02/18] Update comment obsolete since 69c3936a @ 2020-02-15 21:53 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Justin Pryzby @ 2020-02-15 21:53 UTC (permalink / raw) --- src/backend/executor/nodeAgg.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index 601b6dab03..394b4e667b 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -2064,8 +2064,7 @@ initialize_hash_entry(AggState *aggstate, TupleHashTable hashtable, } /* - * Look up hash entries for the current tuple in all hashed grouping sets, - * returning an array of pergroup pointers suitable for advance_aggregates. + * Look up hash entries for the current tuple in all hashed grouping sets. * * Be aware that lookup_hash_entry can reset the tmpcontext. * -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0003-pg_restore-must-be-specified-and-list.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH 02/18] Update comment obsolete since 69c3936a @ 2020-02-15 21:53 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Justin Pryzby @ 2020-02-15 21:53 UTC (permalink / raw) --- src/backend/executor/nodeAgg.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index 601b6dab03..394b4e667b 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -2064,8 +2064,7 @@ initialize_hash_entry(AggState *aggstate, TupleHashTable hashtable, } /* - * Look up hash entries for the current tuple in all hashed grouping sets, - * returning an array of pergroup pointers suitable for advance_aggregates. + * Look up hash entries for the current tuple in all hashed grouping sets. * * Be aware that lookup_hash_entry can reset the tmpcontext. * -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0003-pg_restore-must-be-specified-and-list.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH 02/18] Update comment obsolete since 69c3936a @ 2020-02-15 21:53 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Justin Pryzby @ 2020-02-15 21:53 UTC (permalink / raw) --- src/backend/executor/nodeAgg.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index 601b6dab03..394b4e667b 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -2064,8 +2064,7 @@ initialize_hash_entry(AggState *aggstate, TupleHashTable hashtable, } /* - * Look up hash entries for the current tuple in all hashed grouping sets, - * returning an array of pergroup pointers suitable for advance_aggregates. + * Look up hash entries for the current tuple in all hashed grouping sets. * * Be aware that lookup_hash_entry can reset the tmpcontext. * -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0003-pg_restore-must-be-specified-and-list.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH 02/18] Update comment obsolete since 69c3936a @ 2020-02-15 21:53 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Justin Pryzby @ 2020-02-15 21:53 UTC (permalink / raw) --- src/backend/executor/nodeAgg.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index 601b6dab03..394b4e667b 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -2064,8 +2064,7 @@ initialize_hash_entry(AggState *aggstate, TupleHashTable hashtable, } /* - * Look up hash entries for the current tuple in all hashed grouping sets, - * returning an array of pergroup pointers suitable for advance_aggregates. + * Look up hash entries for the current tuple in all hashed grouping sets. * * Be aware that lookup_hash_entry can reset the tmpcontext. * -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0003-pg_restore-must-be-specified-and-list.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH 02/18] Update comment obsolete since 69c3936a @ 2020-02-15 21:53 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Justin Pryzby @ 2020-02-15 21:53 UTC (permalink / raw) --- src/backend/executor/nodeAgg.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index 601b6dab03..394b4e667b 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -2064,8 +2064,7 @@ initialize_hash_entry(AggState *aggstate, TupleHashTable hashtable, } /* - * Look up hash entries for the current tuple in all hashed grouping sets, - * returning an array of pergroup pointers suitable for advance_aggregates. + * Look up hash entries for the current tuple in all hashed grouping sets. * * Be aware that lookup_hash_entry can reset the tmpcontext. * -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0003-pg_restore-must-be-specified-and-list.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH 02/18] Update comment obsolete since 69c3936a @ 2020-02-15 21:53 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Justin Pryzby @ 2020-02-15 21:53 UTC (permalink / raw) --- src/backend/executor/nodeAgg.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index 601b6dab03..394b4e667b 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -2064,8 +2064,7 @@ initialize_hash_entry(AggState *aggstate, TupleHashTable hashtable, } /* - * Look up hash entries for the current tuple in all hashed grouping sets, - * returning an array of pergroup pointers suitable for advance_aggregates. + * Look up hash entries for the current tuple in all hashed grouping sets. * * Be aware that lookup_hash_entry can reset the tmpcontext. * -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0003-pg_restore-must-be-specified-and-list.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH 02/18] Update comment obsolete since 69c3936a @ 2020-02-15 21:53 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Justin Pryzby @ 2020-02-15 21:53 UTC (permalink / raw) --- src/backend/executor/nodeAgg.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index 601b6dab03..394b4e667b 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -2064,8 +2064,7 @@ initialize_hash_entry(AggState *aggstate, TupleHashTable hashtable, } /* - * Look up hash entries for the current tuple in all hashed grouping sets, - * returning an array of pergroup pointers suitable for advance_aggregates. + * Look up hash entries for the current tuple in all hashed grouping sets. * * Be aware that lookup_hash_entry can reset the tmpcontext. * -- 2.17.0 --lc9FT7cWel8HagAv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0003-pg_restore-must-be-specified-and-list.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH 02/21] Update comment obsolete since 69c3936a @ 2020-02-15 21:53 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Justin Pryzby @ 2020-02-15 21:53 UTC (permalink / raw) --- src/backend/executor/nodeAgg.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index 1d1bf958b6..d80adc519d 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -2070,8 +2070,7 @@ initialize_hash_entry(AggState *aggstate, TupleHashTable hashtable, } /* - * Look up hash entries for the current tuple in all hashed grouping sets, - * returning an array of pergroup pointers suitable for advance_aggregates. + * Look up hash entries for the current tuple in all hashed grouping sets. * * Be aware that lookup_hash_entry can reset the tmpcontext. * -- 2.17.0 --jI8keyz6grp/JLjh Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0003-pg_restore-must-be-specified-and-list.patch" ^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH v2 7/7] Update comment obsolete since 69c3936a @ 2020-02-15 21:53 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Justin Pryzby @ 2020-02-15 21:53 UTC (permalink / raw) --- src/backend/executor/nodeAgg.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c index 4008e27..ab97a35 100644 --- a/src/backend/executor/nodeAgg.c +++ b/src/backend/executor/nodeAgg.c @@ -1499,8 +1499,7 @@ lookup_hash_entry(AggState *aggstate) } /* - * Look up hash entries for the current tuple in all hashed grouping sets, - * returning an array of pergroup pointers suitable for advance_aggregates. + * Look up hash entries for the current tuple in all hashed grouping sets. * * Be aware that lookup_hash_entry can reset the tmpcontext. */ -- 2.7.4 --DiL7RhKs8rK9YGuF-- ^ permalink raw reply [nested|flat] 35+ messages in thread
* XLog size reductions: Reduced XLog record header size for PG17 @ 2023-06-20 20:01 Matthias van de Meent <[email protected]> 0 siblings, 1 reply; 35+ messages in thread From: Matthias van de Meent @ 2023-06-20 20:01 UTC (permalink / raw) To: pgsql-hackers; Heikki Linnakangas <[email protected]>; Robert Haas <[email protected]>; Dilip Kumar <[email protected]>; Andres Freund <[email protected]> Hi, As I've mentioned earlier on-list [0][1] and off-list, I've been working on reducing the volume of WAL that we write. This is one intermediate step towards that effort. Attached is a patchset that reduces the storage overhead of each WAL record, with theoretical total savings in pgbench transactions somewhere between 4-5%, depending on platform and the locality of updates. These savings are achieved by reducing the amount of data stored, and by not emitting bytes that don't carry data relevant to each record's redo- and decode routines. Patches contained: 0001 is copied essentially verbatim from [1] and reduces overhead in the registered block's length field where possible. It is included to improve code commonality between varcoded integer fields. See [1] for more details. 0002 updates accesses to the rmgr-managed bits of xl_info with its own macro returning only rmgr-managed bits, and updates XLogRecGetInfo() to return only the xlog-managed bits. 0003 renames the rm_identify argument from 'info' to 'rmgrinfo'; and stops providing the xlog-managed bits to the function - rmgrs have no need to know the xlog internal info bits. 0004 continues on 0003 and moves the rmgr info bits into their own xl_rmgrinfo of type uint8, stored in the alignment hole in the XLogRecord struct. 0005 updates the code to only include a valid XID in the record when the rmgr actually needs to use that XID. 0006 implements a new, variable length, WAL record header format, previously discussed at [0] and [2]. This new WAL record header is a minimum of 14 bytes large, but generally will be 15 to 21 bytes in size, depending on the data contained, the type of record, and whether the record needs an XID. Notes: - The patchset includes [1] for its variable-length encoding of uint32, and this is included in the savings calculation. - Not all records include the backend's XID anymore. XLog API users must explicitly request the inclusion of XID in the record with the XLOG_INCLUDE_XID record flag. - XLog records are now always aligned to 8 bytes. This was needed to reduce the complexity of var-coding the record length on 32-bit systems. Savings on 32-bit systems still exist, but can be expected to be less impactful. - XLog length is now varlength encoded. No more records with <255 bytes of data storing 3 0-bytes - the length is now stored in 0, 1, 2 or 4 bytes. - RMGRs now get their own uint8 info/flags field. No more sharing bits with WAL infrastructure in xl_info. The byte is only stored if it is non-0, and otherwise omitted (indicated by flag bits in xl_info). Todo: - Check for any needed documentation / code comments updates - benchmark this Future work: - we could omit XLR_BLOCK_ID_DATA_[LONG,SHORT] if it is the only "block ID" in the record (such as checkpoint records, commit/rollback records, etc.). This would be indicated by a xl_info bit, and this would save 2-5 bytes per applicable record. - This patch inherits [1]'s property in which we can release the BKPBLOCK_HAS_DATA flag bit (its value is already implied by XLR_BLOCKID_SZCLASS), allowing us to use it for something else, like indicating varcoded RelFileLocator/BlockId. Kind regards, Matthias van de Meent Neon, Inc. [0] https://www.postgresql.org/message-id/flat/CAEze2Whf%3DfwAj7rosf6aDM9t%2B7MU1w-bJn28HFWYGkz%2Bics-hg... [1] https://www.postgresql.org/message-id/flat/CAEze2WjuJqVeB6EUZ1z75_ittk54H6Lk7WtwRskEeGtZubr4bQ%40mai... [2] https://www.postgresql.org/message-id/flat/CA+Tgmoaa9Yc9O-FP4vS_xTKf8Wgy8TzHpjnjN56_ShKE=jrP-Q@mail.... Attachments: [application/octet-stream] v1-0001-Reduce-overhead-of-small-block-data-in-xlog-recor.patch (10.9K, ../../CAEze2WgcW5SkPO_TGTqirRcCS_FN0xEZqUsXDE6pPz2TYzXQBA@mail.gmail.com/2-v1-0001-Reduce-overhead-of-small-block-data-in-xlog-recor.patch) download | inline diff: From 068d2aba4d2fdff18627133653c784913146554b Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Tue, 24 Jan 2023 12:37:09 +0100 Subject: [PATCH v1 1/6] Reduce overhead of small block data in xlog records We reserve 2 bits in the block ID header for how large the block's registered data is: 0, 1 or 2 bytes, for 0, 1-255 and 256-UINT16_MAX bytes to the block, respectively. This reduces the minimum size of any block-referencing xlog record from 48 bytes to 46 bytes, and reduces the size of any non-hot heap update record by at least 2 bytes. It includes more infrastructure for future value-dependent encoding of length fields, even though it is currently only used in one place. --- src/backend/access/transam/xloginsert.c | 13 ++- src/backend/access/transam/xlogreader.c | 34 +++++-- src/include/access/xlog_internal.h | 114 ++++++++++++++++++++++++ src/include/access/xlogrecord.h | 43 +++++++-- 4 files changed, 188 insertions(+), 16 deletions(-) diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c index ea7e2f67af..3ed7cadd58 100644 --- a/src/backend/access/transam/xloginsert.c +++ b/src/backend/access/transam/xloginsert.c @@ -581,6 +581,8 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, XLogRecordBlockHeader bkpb; XLogRecordBlockImageHeader bimg; XLogRecordBlockCompressHeader cbimg = {0}; + int data_length = 0; + XLogSizeClass data_sizeclass = XLS_EMPTY; bool samerel; bool is_compressed = false; bool include_image; @@ -622,7 +624,7 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, bkpb.id = block_id; bkpb.fork_flags = regbuf->forkno; - bkpb.data_length = 0; + data_length = 0; if ((regbuf->flags & REGBUF_WILL_INIT) == REGBUF_WILL_INIT) bkpb.fork_flags |= BKPBLOCK_WILL_INIT; @@ -785,13 +787,16 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, * overall list. */ bkpb.fork_flags |= BKPBLOCK_HAS_DATA; - bkpb.data_length = (uint16) regbuf->rdata_len; + data_length = (uint16) regbuf->rdata_len; + data_sizeclass = XLogLengthToSizeClass(data_length, XLS_UINT16); total_len += regbuf->rdata_len; rdt_datas_last->next = regbuf->rdata_head; rdt_datas_last = regbuf->rdata_tail; } + bkpb.id |= data_sizeclass << XLR_BLOCKID_SZCLASS_SHIFT; + if (prev_regbuf && RelFileLocatorEquals(regbuf->rlocator, prev_regbuf->rlocator)) { samerel = true; @@ -804,6 +809,10 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, /* Ok, copy the header to the scratch buffer */ memcpy(scratch, &bkpb, SizeOfXLogRecordBlockHeader); scratch += SizeOfXLogRecordBlockHeader; + + scratch += XLogWriteLength(data_length, data_sizeclass, + XLS_UINT16, scratch); + if (include_image) { memcpy(scratch, &bimg, SizeOfXLogRecordBlockImageHeader); diff --git a/src/backend/access/transam/xlogreader.c b/src/backend/access/transam/xlogreader.c index 631f260f79..58e96faaa9 100644 --- a/src/backend/access/transam/xlogreader.c +++ b/src/backend/access/transam/xlogreader.c @@ -1660,11 +1660,11 @@ DecodeXLogRecord(XLogReaderState *state, */ #define COPY_HEADER_FIELD(_dst, _size) \ do { \ - if (remaining < _size) \ + if (remaining < (_size)) \ goto shortdata_err; \ - memcpy(_dst, ptr, _size); \ - ptr += _size; \ - remaining -= _size; \ + memcpy((_dst), ptr, (_size)); \ + ptr += (_size); \ + remaining -= (_size); \ } while(0) char *ptr; @@ -1690,8 +1690,13 @@ DecodeXLogRecord(XLogReaderState *state, datatotal = 0; while (remaining > datatotal) { + XLogSizeClass sizeClass; COPY_HEADER_FIELD(&block_id, sizeof(uint8)); + sizeClass = (block_id & XLR_BLOCKID_SZCLASS_MASK) >> + XLR_BLOCKID_SZCLASS_SHIFT; + block_id &= XLR_BLOCK_ID_MASK; + if (block_id == XLR_BLOCK_ID_DATA_SHORT) { /* XLogRecordDataHeaderShort */ @@ -1755,7 +1760,26 @@ DecodeXLogRecord(XLogReaderState *state, blk->prefetch_buffer = InvalidBuffer; - COPY_HEADER_FIELD(&blk->data_len, sizeof(uint16)); + { + Size read; + uint32 length = 0; + read = XLogReadLength(&length, sizeClass, + XLS_UINT16, + ptr, remaining); + + if (read < 0) + { + report_invalid_record(state, + "Could not read length from record at %X/%X", + LSN_FORMAT_ARGS(state->ReadRecPtr)); + goto err; + } + + ptr += read; + remaining -= read; + blk->data_len = length; + } + /* cross-check that the HAS_DATA flag is set iff data_length > 0 */ if (blk->has_data && blk->data_len == 0) { diff --git a/src/include/access/xlog_internal.h b/src/include/access/xlog_internal.h index b0fd338a00..42b3c66547 100644 --- a/src/include/access/xlog_internal.h +++ b/src/include/access/xlog_internal.h @@ -328,6 +328,120 @@ typedef enum struct LogicalDecodingContext; struct XLogRecordBuffer; +/* + * XLogSizeClass interactions + */ + +/* + * XLogLengthToSizeClass + * Turns a length into a size class + * MaxSizeClass is used to allow the compiler to eliminate branches that + * can never be hit in the caller's code path; e.g. when processing uint16. + */ +static inline XLogSizeClass XLogLengthToSizeClass(uint32 length, + const XLogSizeClass maxSizeClass) +{ + XLogSizeClass sizeClass; + + if (length == 0) + sizeClass = XLS_EMPTY; + else if (length <= UINT8_MAX && maxSizeClass >= XLS_UINT8) + sizeClass = XLS_UINT8; + else if (length <= UINT16_MAX && maxSizeClass >= XLS_UINT16) + sizeClass = XLS_UINT16; + else + { + Assert(maxSizeClass == XLS_UINT32); + sizeClass = XLS_UINT32; + } + + Assert(sizeClass <= maxSizeClass); + + return sizeClass; +} + +/* + * XLogWriteLength + * + * Write a length with size determined by sizeClass into the output. + * Returns the Size of bytes written. The user is responsible for making + * sure that out-of-bounds write is impossible. + * Writes at most 4 bytes. + */ +static inline Size XLogWriteLength(uint32 length, XLogSizeClass sizeClass, + const XLogSizeClass maxSizeClass, + char *output) +{ + Size written = -1; + + Assert(sizeClass <= maxSizeClass && + XLogLengthToSizeClass(length, maxSizeClass) == sizeClass); + +#define WRITE_OP(caseSizeClass, field_type) \ + case caseSizeClass: \ + if ((caseSizeClass) <= maxSizeClass) \ + { \ + field_type typedLength = length; \ + memcpy(output, &typedLength, sizeof(field_type)); \ + written = sizeof(field_type); \ + } \ + break + + switch (sizeClass) { + case XLS_EMPTY: + written = 0; + break; + WRITE_OP(XLS_UINT8, uint8); + WRITE_OP(XLS_UINT16, uint16); + WRITE_OP(XLS_UINT32, uint32); + } + +#undef WRITE_OP + return written; +} + +/* + * XLogReadLength + * + * Read a length with size determined by sizeClass from the input into + * *length. Returns the Size of bytes read, or -1 if the input size was + * too small to read the relevant length field. + */ +static inline Size XLogReadLength(uint32 *length, XLogSizeClass sizeClass, + const XLogSizeClass maxSizeClass, + char *input, Size inSize) +{ + Size readSize = -1; + + Assert(sizeClass <= maxSizeClass); + +#define READ_OP(caseSizeClass, field_type) \ + case caseSizeClass: \ + if ((caseSizeClass) <= maxSizeClass) \ + { \ + field_type typedLength; \ + if (inSize < sizeof(field_type)) \ + return -1; \ + memcpy(&typedLength, input, sizeof(field_type)); \ + readSize = sizeof(field_type); \ + *length = typedLength; \ + } \ + break + + switch (sizeClass) { + case XLS_EMPTY: + readSize = 0; + *length = 0; + break; + READ_OP(XLS_UINT8, uint8); + READ_OP(XLS_UINT16, uint16); + READ_OP(XLS_UINT32, uint32); + } + +#undef READ_OP + return readSize; +} + /* * Method table for resource managers. * diff --git a/src/include/access/xlogrecord.h b/src/include/access/xlogrecord.h index f355e08e1d..b0aada1031 100644 --- a/src/include/access/xlogrecord.h +++ b/src/include/access/xlogrecord.h @@ -17,6 +17,27 @@ #include "storage/block.h" #include "storage/relfilelocator.h" +/* + * XLogSizeClass + * + * XLog data segments registered to e.g. blocks are often quite small. + * This XLogSizeClass infrastructure allows us to do variable length encoding + * on these fields, so that we may save some bytes in each record on length + * fields. + * + * Note that it is possible to encode smaller sizes in the larger size classes, + * but the reverse is not true. For maximum efficiency, it is important to only + * use + * + * Values > UINT32_MAX are not supported in this scheme. + */ +typedef enum XLogSizeClass { + XLS_EMPTY = 0, /* length == 0 */ + XLS_UINT8 = 1, /* length <= UINT8_MAX; stored in uint8 (1B) */ + XLS_UINT16 = 2, /* length <= UINT16_MAX; stored in uint16 (2B) */ + XLS_UINT32 = 3 /* length <= UINT32_MAX; stored in uint32 (4B) */ +} XLogSizeClass; + /* * The overall layout of an XLOG record is: * Fixed-size header (XLogRecord struct) @@ -104,15 +125,14 @@ typedef struct XLogRecordBlockHeader { uint8 id; /* block reference ID */ uint8 fork_flags; /* fork within the relation, and flags */ - uint16 data_length; /* number of payload bytes (not including page - * image) */ + /* Depending on XLR_BLOCKID_SZCLASS; 0, 1 or 2 bytes of data_length follow */ /* If BKPBLOCK_HAS_IMAGE, an XLogRecordBlockImageHeader struct follows */ /* If BKPBLOCK_SAME_REL is not set, a RelFileLocator follows */ /* BlockNumber follows */ } XLogRecordBlockHeader; -#define SizeOfXLogRecordBlockHeader (offsetof(XLogRecordBlockHeader, data_length) + sizeof(uint16)) +#define SizeOfXLogRecordBlockHeader (offsetof(XLogRecordBlockHeader, fork_flags) + sizeof(uint8)) /* * Additional header information when a full-page image is included @@ -226,6 +246,11 @@ typedef struct XLogRecordDataHeaderLong #define SizeOfXLogRecordDataHeaderLong (sizeof(uint8) + sizeof(uint32)) +#define XLR_BLOCK_ID_MASK 0x3F + +#define XLR_BLOCKID_SZCLASS_SHIFT 6 +#define XLR_BLOCKID_SZCLASS_MASK (3 << XLR_BLOCKID_SZCLASS_SHIFT) + /* * Block IDs used to distinguish different kinds of record fragments. Block * references are numbered from 0 to XLR_MAX_BLOCK_ID. A rmgr is free to use @@ -234,15 +259,15 @@ typedef struct XLogRecordDataHeaderLong * numbers are reserved to denote the "main" data portion of the record, * as well as replication-supporting transaction metadata. * - * The maximum is currently set at 32, quite arbitrarily. Most records only + * The maximum is currently set at 0x20 (32), quite arbitrarily. Most records only * need a handful of block references, but there are a few exceptions that * need more. */ -#define XLR_MAX_BLOCK_ID 32 +#define XLR_MAX_BLOCK_ID 0x20 -#define XLR_BLOCK_ID_DATA_SHORT 255 -#define XLR_BLOCK_ID_DATA_LONG 254 -#define XLR_BLOCK_ID_ORIGIN 253 -#define XLR_BLOCK_ID_TOPLEVEL_XID 252 +#define XLR_BLOCK_ID_DATA_SHORT 0x3F +#define XLR_BLOCK_ID_DATA_LONG 0x3E +#define XLR_BLOCK_ID_ORIGIN 0x3D +#define XLR_BLOCK_ID_TOPLEVEL_XID 0x3C #endif /* XLOGRECORD_H */ -- 2.39.0 [application/octet-stream] v1-0004-Move-rmgr-info-bits-into-their-own-field-in-the-x.patch (26.1K, ../../CAEze2WgcW5SkPO_TGTqirRcCS_FN0xEZqUsXDE6pPz2TYzXQBA@mail.gmail.com/3-v1-0004-Move-rmgr-info-bits-into-their-own-field-in-the-x.patch) download | inline diff: From 3533e1242ebee7435e58d28be93ce923273e0c3a Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Mon, 23 Jan 2023 17:21:12 +0100 Subject: [PATCH v1 4/6] Move rmgr info bits into their own field in the xlog record header. This takes one byte of the 2 previously lost to alignment, and allows each rmgr to use more info bits (useful for e.g. merging the heap rmgrs back into one). --- contrib/pg_walinspect/pg_walinspect.c | 7 +++--- src/backend/access/transam/xact.c | 22 +++++++++-------- src/backend/access/transam/xlog.c | 7 +++--- src/backend/access/transam/xloginsert.c | 27 ++++++++++++--------- src/backend/access/transam/xlogprefetcher.c | 2 +- src/backend/access/transam/xlogreader.c | 2 +- src/backend/access/transam/xlogrecovery.c | 10 ++++---- src/backend/access/transam/xlogstats.c | 12 ++++----- src/backend/catalog/storage.c | 7 +++--- src/backend/commands/dbcommands.c | 20 +++++++++------ src/backend/utils/probes.d | 2 +- src/bin/pg_resetwal/pg_resetwal.c | 3 ++- src/bin/pg_rewind/parsexlog.c | 4 +-- src/bin/pg_waldump/pg_waldump.c | 7 +++--- src/include/access/brin_xlog.h | 3 +-- src/include/access/heapam_xlog.h | 4 +-- src/include/access/nbtxlog.h | 3 +-- src/include/access/xact.h | 6 ++--- src/include/access/xlog_internal.h | 2 +- src/include/access/xloginsert.h | 3 ++- src/include/access/xlogreader.h | 4 +-- src/include/access/xlogrecord.h | 11 ++------- src/include/access/xlogstats.h | 5 ++-- 23 files changed, 89 insertions(+), 84 deletions(-) diff --git a/contrib/pg_walinspect/pg_walinspect.c b/contrib/pg_walinspect/pg_walinspect.c index de65a84aba..1b6dae9185 100644 --- a/contrib/pg_walinspect/pg_walinspect.c +++ b/contrib/pg_walinspect/pg_walinspect.c @@ -697,7 +697,7 @@ GetXLogSummaryStats(XLogStats *stats, ReturnSetInfo *rsinfo, { int rj; - for (rj = 0; rj < MAX_XLINFO_TYPES; rj++) + for (rj = 0; rj < MAX_XLRMGRINFO_TYPES; rj++) { const char *id; @@ -712,10 +712,9 @@ GetXLogSummaryStats(XLogStats *stats, ReturnSetInfo *rsinfo, old_cxt = MemoryContextSwitchTo(tmp_cxt); - /* the upper four bits in xl_info are the rmgr's */ - id = desc.rm_identify(rj << 4); + id = desc.rm_identify(rj); if (id == NULL) - id = psprintf("UNKNOWN (%x)", rj << 4); + id = psprintf("UNKNOWN (%x)", rj); FillXLogStatsRow(psprintf("%s/%s", desc.rm_name, id), count, total_count, rec_len, total_rec_len, fpi_len, diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index 33dfada659..a70cb3e124 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -5651,7 +5651,8 @@ XactLogCommitRecord(TimestampTz commit_time, xl_xact_invals xl_invals; xl_xact_twophase xl_twophase; xl_xact_origin xl_origin; - uint8 info; + uint8 info = 0; + uint8 rmgr_info; Assert(CritSectionCount > 0); @@ -5659,9 +5660,9 @@ XactLogCommitRecord(TimestampTz commit_time, /* decide between a plain and 2pc commit */ if (!TransactionIdIsValid(twophase_xid)) - info = XLOG_XACT_COMMIT; + rmgr_info = XLOG_XACT_COMMIT; else - info = XLOG_XACT_COMMIT_PREPARED; + rmgr_info = XLOG_XACT_COMMIT_PREPARED; /* First figure out and collect all the information needed */ @@ -5737,7 +5738,7 @@ XactLogCommitRecord(TimestampTz commit_time, } if (xl_xinfo.xinfo != 0) - info |= XLOG_XACT_HAS_INFO; + rmgr_info |= XLOG_XACT_HAS_INFO; /* Then include all the collected data into the commit record. */ @@ -5795,7 +5796,7 @@ XactLogCommitRecord(TimestampTz commit_time, /* we allow filtering by xacts */ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); - return XLogInsert(RM_XACT_ID, info); + return XLogInsertExtended(RM_XACT_ID, info, rmgr_info); } /* @@ -5821,7 +5822,8 @@ XactLogAbortRecord(TimestampTz abort_time, xl_xact_dbinfo xl_dbinfo; xl_xact_origin xl_origin; - uint8 info; + uint8 info = 0; + uint8 rmgr_info; Assert(CritSectionCount > 0); @@ -5829,9 +5831,9 @@ XactLogAbortRecord(TimestampTz abort_time, /* decide between a plain and 2pc abort */ if (!TransactionIdIsValid(twophase_xid)) - info = XLOG_XACT_ABORT; + rmgr_info = XLOG_XACT_ABORT; else - info = XLOG_XACT_ABORT_PREPARED; + rmgr_info = XLOG_XACT_ABORT_PREPARED; /* First figure out and collect all the information needed */ @@ -5890,7 +5892,7 @@ XactLogAbortRecord(TimestampTz abort_time, } if (xl_xinfo.xinfo != 0) - info |= XLOG_XACT_HAS_INFO; + rmgr_info |= XLOG_XACT_HAS_INFO; /* Then include all the collected data into the abort record. */ @@ -5941,7 +5943,7 @@ XactLogAbortRecord(TimestampTz abort_time, /* Include the replication origin */ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); - return XLogInsert(RM_XACT_ID, info); + return XLogInsertExtended(RM_XACT_ID, info, rmgr_info); } /* diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index d6c00b896b..70c193c08f 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -738,9 +738,9 @@ XLogInsertRecord(XLogRecData *rdata, pg_crc32c rdata_crc; bool inserted; XLogRecord *rechdr = (XLogRecord *) rdata->data; - uint8 info = rechdr->xl_info & ~XLR_INFO_MASK; + uint8 rmgrinfo = rechdr->xl_rmgrinfo; bool isLogSwitch = (rechdr->xl_rmid == RM_XLOG_ID && - info == XLOG_SWITCH); + rmgrinfo == XLOG_SWITCH); XLogRecPtr StartPos; XLogRecPtr EndPos; bool prevDoPageWrites = doPageWrites; @@ -4730,7 +4730,8 @@ BootStrapXLOG(void) record->xl_prev = 0; record->xl_xid = InvalidTransactionId; record->xl_tot_len = SizeOfXLogRecord + SizeOfXLogRecordDataHeaderShort + sizeof(checkPoint); - record->xl_info = XLOG_CHECKPOINT_SHUTDOWN; + record->xl_info = 0; + record->xl_rmgrinfo = XLOG_CHECKPOINT_SHUTDOWN; record->xl_rmid = RM_XLOG_ID; recptr += SizeOfXLogRecord; /* fill the XLogRecordDataHeaderShort struct */ diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c index 3ed7cadd58..76b508d535 100644 --- a/src/backend/access/transam/xloginsert.c +++ b/src/backend/access/transam/xloginsert.c @@ -135,7 +135,7 @@ static bool begininsert_called = false; /* Memory context to hold the registered buffer and data references. */ static MemoryContext xloginsert_cxt; -static XLogRecData *XLogRecordAssemble(RmgrId rmid, uint8 info, +static XLogRecData *XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecPtr, bool doPageWrites, XLogRecPtr *fpw_lsn, int *num_fpi, bool *topxid_included); @@ -447,9 +447,9 @@ XLogSetRecordFlags(uint8 flags) } /* - * Insert an XLOG record having the specified RMID and info bytes, with the - * body of the record being the data and buffer references registered earlier - * with XLogRegister* calls. + * Insert an XLOG record having the specified RMID and rmgr_info bytes, with + * the body of the record being the data and buffer references registered + * earlier with XLogRegister* calls. * * Returns XLOG pointer to end of record (beginning of next record). * This can be used as LSN for data pages affected by the logged action. @@ -458,7 +458,13 @@ XLogSetRecordFlags(uint8 flags) * WAL rule "write the log before the data".) */ XLogRecPtr -XLogInsert(RmgrId rmid, uint8 info) +XLogInsert(RmgrId rmid, uint8 rmgr_info) +{ + return XLogInsertExtended(rmid, 0, rmgr_info); +} + +XLogRecPtr +XLogInsertExtended(RmgrId rmid, uint8 info, uint8 rmgr_info) { XLogRecPtr EndPos; @@ -470,12 +476,11 @@ XLogInsert(RmgrId rmid, uint8 info) * The caller can set rmgr bits, XLR_SPECIAL_REL_UPDATE and * XLR_CHECK_CONSISTENCY; the rest are reserved for use by me. */ - if ((info & ~(XLR_RMGR_INFO_MASK | - XLR_SPECIAL_REL_UPDATE | + if ((info & ~(XLR_SPECIAL_REL_UPDATE | XLR_CHECK_CONSISTENCY)) != 0) elog(PANIC, "invalid xlog info mask %02X", info); - TRACE_POSTGRESQL_WAL_INSERT(rmid, info); + TRACE_POSTGRESQL_WAL_INSERT(rmid, info, rmgr_info); /* * In bootstrap mode, we don't actually log anything but XLOG resources; @@ -504,7 +509,7 @@ XLogInsert(RmgrId rmid, uint8 info) */ GetFullPageWriteInfo(&RedoRecPtr, &doPageWrites); - rdt = XLogRecordAssemble(rmid, info, RedoRecPtr, doPageWrites, + rdt = XLogRecordAssemble(rmid, info, rmgr_info, RedoRecPtr, doPageWrites, &fpw_lsn, &num_fpi, &topxid_included); EndPos = XLogInsertRecord(rdt, fpw_lsn, curinsert_flags, num_fpi, @@ -532,8 +537,7 @@ XLogInsert(RmgrId rmid, uint8 info) * current subtransaction. */ static XLogRecData * -XLogRecordAssemble(RmgrId rmid, uint8 info, - XLogRecPtr RedoRecPtr, bool doPageWrites, +XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecPtr, bool doPageWrites, XLogRecPtr *fpw_lsn, int *num_fpi, bool *topxid_included) { XLogRecData *rdt; @@ -924,6 +928,7 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, rechdr->xl_tot_len = (uint32) total_len; rechdr->xl_info = info; rechdr->xl_rmid = rmid; + rechdr->xl_rmgrinfo = rmgr_info; rechdr->xl_prev = InvalidXLogRecPtr; rechdr->xl_crc = rdata_crc; diff --git a/src/backend/access/transam/xlogprefetcher.c b/src/backend/access/transam/xlogprefetcher.c index 906e3d9469..f928c9aa44 100644 --- a/src/backend/access/transam/xlogprefetcher.c +++ b/src/backend/access/transam/xlogprefetcher.c @@ -536,7 +536,7 @@ XLogPrefetcherNextBlock(uintptr_t pgsr_private, XLogRecPtr *lsn) if (replaying_lsn < record->lsn) { uint8 rmid = record->header.xl_rmid; - uint8 record_type = record->header.xl_info & ~XLR_INFO_MASK; + uint8 record_type = record->header.xl_rmgrinfo; if (rmid == RM_XLOG_ID) { diff --git a/src/backend/access/transam/xlogreader.c b/src/backend/access/transam/xlogreader.c index 58e96faaa9..e15b0f9580 100644 --- a/src/backend/access/transam/xlogreader.c +++ b/src/backend/access/transam/xlogreader.c @@ -860,7 +860,7 @@ restart: * Special processing if it's an XLOG SWITCH record */ if (record->xl_rmid == RM_XLOG_ID && - (record->xl_info & ~XLR_INFO_MASK) == XLOG_SWITCH) + record->xl_rmgrinfo == XLOG_SWITCH) { /* Pretend it extends to end of segment */ state->NextRecPtr += state->segcxt.ws_segsize - 1; diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index 6ca898c6e9..aa9f21ca5c 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -632,7 +632,7 @@ InitWalRecovery(ControlFileData *ControlFile, bool *wasShutdown_ptr, if (record != NULL) { memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint)); - wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN); + wasShutdown = (record->xl_rmgrinfo == XLOG_CHECKPOINT_SHUTDOWN); ereport(DEBUG1, (errmsg_internal("checkpoint record is at %X/%X", LSN_FORMAT_ARGS(CheckPointLoc)))); @@ -786,7 +786,7 @@ InitWalRecovery(ControlFileData *ControlFile, bool *wasShutdown_ptr, (errmsg("could not locate a valid checkpoint record"))); } memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint)); - wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN); + wasShutdown = (record->xl_rmgrinfo == XLOG_CHECKPOINT_SHUTDOWN); } /* @@ -1857,7 +1857,7 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl { TimeLineID newReplayTLI = *replayTLI; TimeLineID prevReplayTLI = *replayTLI; - uint8 info = record->xl_info & ~XLR_INFO_MASK; + uint8 info = record->xl_rmgrinfo; if (info == XLOG_CHECKPOINT_SHUTDOWN) { @@ -3992,12 +3992,12 @@ ReadCheckpointRecord(XLogPrefetcher *xlogprefetcher, XLogRecPtr RecPtr, (errmsg("invalid resource manager ID in checkpoint record"))); return NULL; } - info = record->xl_info & ~XLR_INFO_MASK; + info = record->xl_rmgrinfo; if (info != XLOG_CHECKPOINT_SHUTDOWN && info != XLOG_CHECKPOINT_ONLINE) { ereport(LOG, - (errmsg("invalid xl_info in checkpoint record"))); + (errmsg("invalid xl_rmgrinfo in checkpoint record"))); return NULL; } if (record->xl_tot_len != SizeOfXLogRecord + SizeOfXLogRecordDataHeaderShort + sizeof(CheckPoint)) diff --git a/src/backend/access/transam/xlogstats.c b/src/backend/access/transam/xlogstats.c index 267b108943..28d6d971ac 100644 --- a/src/backend/access/transam/xlogstats.c +++ b/src/backend/access/transam/xlogstats.c @@ -74,21 +74,21 @@ XLogRecStoreStats(XLogStats *stats, XLogReaderState *record) /* * Update per-record statistics, where the record is identified by a - * combination of the RmgrId and the four bits of the xl_info field that - * are the rmgr's domain (resulting in sixteen possible entries per + * combination of the RmgrId and the eight bits of the xl_rmgrinfo field + * that are the rmgr's domain (resulting in 256 possible entries per * RmgrId). */ - recid = XLogRecGetRmgrInfo(record) >> 4; + recid = XLogRecGetRmgrInfo(record); /* * XACT records need to be handled differently. Those records use the * first bit of those four bits for an optional flag variable and the - * following three bits for the opcode. We filter opcode out of xl_info - * and use it as the identifier of the record. + * following three bits for the opcode. We filter opcode out of + * xl_rmgrinfo and use it as the identifier of the record. */ if (rmid == RM_XACT_ID) - recid &= 0x07; + recid &= 0x70; stats->record_stats[rmid][recid].count++; stats->record_stats[rmid][recid].rec_len += rec_len; diff --git a/src/backend/catalog/storage.c b/src/backend/catalog/storage.c index 4d73023a65..5b48b24e7b 100644 --- a/src/backend/catalog/storage.c +++ b/src/backend/catalog/storage.c @@ -194,7 +194,7 @@ log_smgrcreate(const RelFileLocator *rlocator, ForkNumber forkNum) XLogBeginInsert(); XLogRegisterData((char *) &xlrec, sizeof(xlrec)); - XLogInsert(RM_SMGR_ID, XLOG_SMGR_CREATE | XLR_SPECIAL_REL_UPDATE); + XLogInsertExtended(RM_SMGR_ID, XLR_SPECIAL_REL_UPDATE, XLOG_SMGR_CREATE); } /* @@ -375,8 +375,9 @@ RelationTruncate(Relation rel, BlockNumber nblocks) XLogBeginInsert(); XLogRegisterData((char *) &xlrec, sizeof(xlrec)); - lsn = XLogInsert(RM_SMGR_ID, - XLOG_SMGR_TRUNCATE | XLR_SPECIAL_REL_UPDATE); + lsn = XLogInsertExtended(RM_SMGR_ID, + XLR_SPECIAL_REL_UPDATE, + XLOG_SMGR_TRUNCATE); /* * Flush, because otherwise the truncation of the main relation might diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c index 528330a0b3..c05a6fb7df 100644 --- a/src/backend/commands/dbcommands.c +++ b/src/backend/commands/dbcommands.c @@ -624,8 +624,9 @@ CreateDatabaseUsingFileCopy(Oid src_dboid, Oid dst_dboid, Oid src_tsid, XLogRegisterData((char *) &xlrec, sizeof(xl_dbase_create_file_copy_rec)); - (void) XLogInsert(RM_DBASE_ID, - XLOG_DBASE_CREATE_FILE_COPY | XLR_SPECIAL_REL_UPDATE); + (void) XLogInsertExtended(RM_DBASE_ID, + XLR_SPECIAL_REL_UPDATE, + XLOG_DBASE_CREATE_FILE_COPY); } pfree(srcpath); pfree(dstpath); @@ -2079,8 +2080,9 @@ movedb(const char *dbname, const char *tblspcname) XLogRegisterData((char *) &xlrec, sizeof(xl_dbase_create_file_copy_rec)); - (void) XLogInsert(RM_DBASE_ID, - XLOG_DBASE_CREATE_FILE_COPY | XLR_SPECIAL_REL_UPDATE); + (void) XLogInsertExtended(RM_DBASE_ID, + XLR_SPECIAL_REL_UPDATE, + XLOG_DBASE_CREATE_FILE_COPY); } /* @@ -2173,8 +2175,9 @@ movedb(const char *dbname, const char *tblspcname) XLogRegisterData((char *) &xlrec, sizeof(xl_dbase_drop_rec)); XLogRegisterData((char *) &src_tblspcoid, sizeof(Oid)); - (void) XLogInsert(RM_DBASE_ID, - XLOG_DBASE_DROP | XLR_SPECIAL_REL_UPDATE); + (void) XLogInsertExtended(RM_DBASE_ID, + XLR_SPECIAL_REL_UPDATE, + XLOG_DBASE_DROP); } /* Now it's safe to release the database lock */ @@ -2896,8 +2899,9 @@ remove_dbtablespaces(Oid db_id) XLogRegisterData((char *) &xlrec, MinSizeOfDbaseDropRec); XLogRegisterData((char *) tablespace_ids, ntblspc * sizeof(Oid)); - (void) XLogInsert(RM_DBASE_ID, - XLOG_DBASE_DROP | XLR_SPECIAL_REL_UPDATE); + (void) XLogInsertExtended(RM_DBASE_ID, + XLR_SPECIAL_REL_UPDATE, + XLOG_DBASE_DROP); } list_free(ltblspc); diff --git a/src/backend/utils/probes.d b/src/backend/utils/probes.d index 0af275587b..131067b5ac 100644 --- a/src/backend/utils/probes.d +++ b/src/backend/utils/probes.d @@ -87,7 +87,7 @@ provider postgresql { probe smgr__md__write__start(ForkNumber, BlockNumber, Oid, Oid, Oid, int); probe smgr__md__write__done(ForkNumber, BlockNumber, Oid, Oid, Oid, int, int, int); - probe wal__insert(unsigned char, unsigned char); + probe wal__insert(unsigned char, unsigned char, unsigned char); probe wal__switch(); probe wal__buffer__write__dirty__start(); probe wal__buffer__write__dirty__done(); diff --git a/src/bin/pg_resetwal/pg_resetwal.c b/src/bin/pg_resetwal/pg_resetwal.c index e7ef2b8bd0..80cb9a5bc4 100644 --- a/src/bin/pg_resetwal/pg_resetwal.c +++ b/src/bin/pg_resetwal/pg_resetwal.c @@ -1066,7 +1066,8 @@ WriteEmptyXLOG(void) record->xl_prev = 0; record->xl_xid = InvalidTransactionId; record->xl_tot_len = SizeOfXLogRecord + SizeOfXLogRecordDataHeaderShort + sizeof(CheckPoint); - record->xl_info = XLOG_CHECKPOINT_SHUTDOWN; + record->xl_info = 0; + record->xl_rmgrinfo = XLOG_CHECKPOINT_SHUTDOWN; record->xl_rmid = RM_XLOG_ID; recptr += SizeOfXLogRecord; diff --git a/src/bin/pg_rewind/parsexlog.c b/src/bin/pg_rewind/parsexlog.c index 2b6b46f0e3..aa03fe8031 100644 --- a/src/bin/pg_rewind/parsexlog.c +++ b/src/bin/pg_rewind/parsexlog.c @@ -438,9 +438,9 @@ extractPageInfo(XLogReaderState *record) * track that change. */ pg_fatal("WAL record modifies a relation, but record type is not recognized: " - "lsn: %X/%X, rmid: %d, rmgr: %s, info: %02X", + "lsn: %X/%X, rmid: %d, rmgr: %s, info: %02X, rmgrinfo: %02X", LSN_FORMAT_ARGS(record->ReadRecPtr), - rmid, RmgrName(rmid), info | rminfo); + rmid, RmgrName(rmid), info, rminfo); } for (block_id = 0; block_id <= XLogRecMaxBlockId(record); block_id++) diff --git a/src/bin/pg_waldump/pg_waldump.c b/src/bin/pg_waldump/pg_waldump.c index e3390f3d39..1fc4e8dc65 100644 --- a/src/bin/pg_waldump/pg_waldump.c +++ b/src/bin/pg_waldump/pg_waldump.c @@ -693,7 +693,7 @@ XLogDumpDisplayStats(XLogDumpConfig *config, XLogStats *stats) } else { - for (rj = 0; rj < MAX_XLINFO_TYPES; rj++) + for (rj = 0; rj < MAX_XLRMGRINFO_TYPES; rj++) { const char *id; @@ -706,10 +706,9 @@ XLogDumpDisplayStats(XLogDumpConfig *config, XLogStats *stats) if (count == 0) continue; - /* the upper four bits in xl_info are the rmgr's */ - id = desc->rm_identify(rj << 4); + id = desc->rm_identify(rj); if (id == NULL) - id = psprintf("UNKNOWN (%x)", rj << 4); + id = psprintf("UNKNOWN (%x)", rj); XLogDumpStatsRow(psprintf("%s/%s", desc->rm_name, id), count, total_count, rec_len, total_rec_len, diff --git a/src/include/access/brin_xlog.h b/src/include/access/brin_xlog.h index 45a688d09b..92190688c9 100644 --- a/src/include/access/brin_xlog.h +++ b/src/include/access/brin_xlog.h @@ -25,8 +25,7 @@ /* * WAL record definitions for BRIN's WAL operations * - * XLOG allows to store some information in high 4 bits of log - * record xl_info field. + * XLOG allows to store some information in the 8-bit xl_rmgrinfo field. */ #define XLOG_BRIN_CREATE_INDEX 0x00 #define XLOG_BRIN_INSERT 0x10 diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h index 70e9acd350..6705c8a316 100644 --- a/src/include/access/heapam_xlog.h +++ b/src/include/access/heapam_xlog.h @@ -26,8 +26,8 @@ /* * WAL record definitions for heapam.c's WAL operations * - * XLOG allows to store some information in high 4 bits of log - * record xl_info field. We use 3 for opcode and one for init bit. + * XLOG allows to store some information in the 8-bit xl_rmgrinfo field. + * We use 3 for opcode and one for init bit. */ #define XLOG_HEAP_INSERT 0x00 #define XLOG_HEAP_DELETE 0x10 diff --git a/src/include/access/nbtxlog.h b/src/include/access/nbtxlog.h index 422ac13d56..6d1231a01e 100644 --- a/src/include/access/nbtxlog.h +++ b/src/include/access/nbtxlog.h @@ -21,8 +21,7 @@ /* * XLOG records for btree operations * - * XLOG allows to store some information in high 4 bits of log - * record xl_info field + * XLOG allows to store some information in the 8-bit xl_rmgrinfo field. */ #define XLOG_BTREE_INSERT_LEAF 0x00 /* add index tuple without split */ #define XLOG_BTREE_INSERT_UPPER 0x10 /* same, on a non-leaf page */ diff --git a/src/include/access/xact.h b/src/include/access/xact.h index b94b264402..2a9a708279 100644 --- a/src/include/access/xact.h +++ b/src/include/access/xact.h @@ -163,8 +163,8 @@ typedef struct SavedTransactionCharacteristics */ /* - * XLOG allows to store some information in high 4 bits of log record xl_info - * field. We use 3 for the opcode, and one about an optional flag variable. + * XLOG allows to store some information in the 8-bit xl_rmgrinfo field. + * We use 3 for the opcode, and one about an optional flag variable. */ #define XLOG_XACT_COMMIT 0x00 #define XLOG_XACT_PREPARE 0x10 @@ -232,7 +232,7 @@ typedef struct xl_xact_assignment * A minimal commit/abort record only consists of a xl_xact_commit/abort * struct. The presence of additional information is indicated by bits set in * 'xl_xact_xinfo->xinfo'. The presence of the xinfo field itself is signaled - * by a set XLOG_XACT_HAS_INFO bit in the xl_info field. + * by a set XLOG_XACT_HAS_INFO bit in the xl_rmgrinfo field. * * NB: All the individual data chunks should be sized to multiples of * sizeof(int) and only require int32 alignment. If they require bigger diff --git a/src/include/access/xlog_internal.h b/src/include/access/xlog_internal.h index 42b3c66547..876e2790f9 100644 --- a/src/include/access/xlog_internal.h +++ b/src/include/access/xlog_internal.h @@ -448,7 +448,7 @@ static inline Size XLogReadLength(uint32 *length, XLogSizeClass sizeClass, * This struct must be kept in sync with the PG_RMGR definition in * rmgr.c. * - * rm_identify must return a name for the record based on xl_info (without + * rm_identify must return a name for the record based on xl_rmgrinfo (without * reference to the rmid). For example, XLOG_BTREE_VACUUM would be named * "VACUUM". rm_desc can then be called to obtain additional detail for the * record, if available (e.g. the last block). diff --git a/src/include/access/xloginsert.h b/src/include/access/xloginsert.h index 31785dc578..c6c46e5409 100644 --- a/src/include/access/xloginsert.h +++ b/src/include/access/xloginsert.h @@ -41,7 +41,8 @@ /* prototypes for public functions in xloginsert.c: */ extern void XLogBeginInsert(void); extern void XLogSetRecordFlags(uint8 flags); -extern XLogRecPtr XLogInsert(RmgrId rmid, uint8 info); +extern XLogRecPtr XLogInsertExtended(RmgrId rmid, uint8 info, uint8 rmgr_info); +extern XLogRecPtr XLogInsert(RmgrId rmid, uint8 rmgr_info); extern void XLogEnsureRecordSpace(int max_block_id, int ndatas); extern void XLogRegisterData(char *data, uint32 len); extern void XLogRegisterBuffer(uint8 block_id, Buffer buffer, uint8 flags); diff --git a/src/include/access/xlogreader.h b/src/include/access/xlogreader.h index d6f49b1a38..cec7930b19 100644 --- a/src/include/access/xlogreader.h +++ b/src/include/access/xlogreader.h @@ -406,8 +406,8 @@ extern bool DecodeXLogRecord(XLogReaderState *state, */ #define XLogRecGetTotalLen(decoder) ((decoder)->record->header.xl_tot_len) #define XLogRecGetPrev(decoder) ((decoder)->record->header.xl_prev) -#define XLogRecGetInfo(decoder) ((decoder)->record->header.xl_info & XLR_INFO_MASK) -#define XLogRecGetRmgrInfo(decoder) (((decoder)->record->header.xl_info) & XLR_RMGR_INFO_MASK) +#define XLogRecGetInfo(decoder) ((decoder)->record->header.xl_info) +#define XLogRecGetRmgrInfo(decoder) ((decoder)->record->header.xl_rmgrinfo) #define XLogRecGetRmid(decoder) ((decoder)->record->header.xl_rmid) #define XLogRecGetXid(decoder) ((decoder)->record->header.xl_xid) #define XLogRecGetOrigin(decoder) ((decoder)->record->record_origin) diff --git a/src/include/access/xlogrecord.h b/src/include/access/xlogrecord.h index b0aada1031..42b06f163e 100644 --- a/src/include/access/xlogrecord.h +++ b/src/include/access/xlogrecord.h @@ -66,7 +66,8 @@ typedef struct XLogRecord XLogRecPtr xl_prev; /* ptr to previous record in log */ uint8 xl_info; /* flag bits, see below */ RmgrId xl_rmid; /* resource manager for this record */ - /* 2 bytes of padding here, initialize to zero */ + uint8 xl_rmgrinfo; /* rmgr flag bits, see below */ + /* 1 byte of padding here, initialize to zero */ pg_crc32c xl_crc; /* CRC for this record */ /* XLogRecordBlockHeaders and XLogRecordDataHeader follow, no padding */ @@ -75,14 +76,6 @@ typedef struct XLogRecord #define SizeOfXLogRecord (offsetof(XLogRecord, xl_crc) + sizeof(pg_crc32c)) -/* - * The high 4 bits in xl_info may be used freely by rmgr. The - * XLR_SPECIAL_REL_UPDATE and XLR_CHECK_CONSISTENCY bits can be passed by - * XLogInsert caller. The rest are set internally by XLogInsert. - */ -#define XLR_INFO_MASK 0x0F -#define XLR_RMGR_INFO_MASK 0xF0 - /* * XLogReader needs to allocate all the data of a WAL record in a single * chunk. This means that a single XLogRecord cannot exceed MaxAllocSize diff --git a/src/include/access/xlogstats.h b/src/include/access/xlogstats.h index 89410ce92b..30974f5ba7 100644 --- a/src/include/access/xlogstats.h +++ b/src/include/access/xlogstats.h @@ -16,7 +16,8 @@ #include "access/rmgr.h" #include "access/xlogreader.h" -#define MAX_XLINFO_TYPES 16 +/* all 8 bits of xl_rmgrinfo are available */ +#define MAX_XLRMGRINFO_TYPES 256 typedef struct XLogRecStats { @@ -33,7 +34,7 @@ typedef struct XLogStats XLogRecPtr endptr; #endif XLogRecStats rmgr_stats[RM_MAX_ID + 1]; - XLogRecStats record_stats[RM_MAX_ID + 1][MAX_XLINFO_TYPES]; + XLogRecStats record_stats[RM_MAX_ID + 1][MAX_XLRMGRINFO_TYPES]; } XLogStats; extern void XLogRecGetLen(XLogReaderState *record, uint32 *rec_len, -- 2.39.0 [application/octet-stream] v1-0005-Stop-the-logging-of-XIDs-in-records-where-the-XID.patch (10.4K, ../../CAEze2WgcW5SkPO_TGTqirRcCS_FN0xEZqUsXDE6pPz2TYzXQBA@mail.gmail.com/4-v1-0005-Stop-the-logging-of-XIDs-in-records-where-the-XID.patch) download | inline diff: From 501ace5f11cb90101b3a770de3ada9f541af698c Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Wed, 25 Jan 2023 15:40:30 +0100 Subject: [PATCH v1 5/6] Stop the logging of XIDs in records where the XID is unused. Many WAL record types do not need to contain an XID. This change prepares for the removal of the xid from the main xlog record header, by requiring the user of the XLog insertion API to explicitly request the inclusion of the current xid in the log record - in all other cases we'll log, assume and track InvalidXid instead. --- src/backend/access/heap/heapam.c | 16 +++++++++------- src/backend/access/heap/pruneheap.c | 1 + src/backend/access/heap/rewriteheap.c | 1 + src/backend/access/transam/generic_xlog.c | 3 +++ src/backend/access/transam/multixact.c | 1 + src/backend/access/transam/xact.c | 4 ++-- src/backend/access/transam/xloginsert.c | 12 ++++++++++-- src/backend/commands/tablecmds.c | 2 +- src/backend/replication/logical/message.c | 2 +- src/backend/utils/cache/inval.c | 1 + src/include/access/xlog.h | 1 + 11 files changed, 31 insertions(+), 13 deletions(-) diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 2a2525cee9..c9f3dbf9ca 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -1969,7 +1969,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid, heaptup->t_len - SizeofHeapTupleHeader); /* filtering by origin on a row level is much more efficient */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP_ID, info); @@ -2365,7 +2365,7 @@ heap_multi_insert(Relation relation, TupleTableSlot **slots, int ntuples, XLogRegisterBufData(0, tupledata, totaldatalen); /* filtering by origin on a row level is much more efficient */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP2_ID, info); @@ -2865,7 +2865,7 @@ l1: } /* filtering by origin on a row level is much more efficient */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP_ID, XLOG_HEAP_DELETE); @@ -5535,6 +5535,7 @@ l4: cleared_all_frozen ? XLH_LOCK_ALL_FROZEN_CLEARED : 0; XLogRegisterData((char *) &xlrec, SizeOfHeapLockUpdated); + XLogSetRecordFlags(XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP2_ID, XLOG_HEAP2_LOCK_UPDATED); @@ -5685,7 +5686,7 @@ heap_finish_speculative(Relation relation, ItemPointer tid) XLogBeginInsert(); /* We want the same filtering on this as on a plain insert */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); XLogRegisterData((char *) &xlrec, SizeOfHeapConfirm); XLogRegisterBuffer(0, buffer, REGBUF_STANDARD); @@ -5831,7 +5832,8 @@ heap_abort_speculative(Relation relation, ItemPointer tid) XLogRegisterData((char *) &xlrec, SizeOfHeapDelete); XLogRegisterBuffer(0, buffer, REGBUF_STANDARD); - /* No replica identity & replication origin logged */ + /* No replica identity & replication origin logged, but XID is required */ + XLogSetRecordFlags(XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP_ID, XLOG_HEAP_DELETE); @@ -8563,7 +8565,7 @@ log_heap_update(Relation reln, Buffer oldbuf, } /* filtering by origin on a row level is much more efficient */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP_ID, info); @@ -8638,7 +8640,7 @@ log_heap_new_cid(Relation relation, HeapTuple tup) XLogRegisterData((char *) &xlrec, SizeOfHeapNewCid); /* will be looked at irrespective of origin */ - + XLogSetRecordFlags(XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP2_ID, XLOG_HEAP2_NEW_CID); return recptr; diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index 20df39c149..36459e75a1 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -445,6 +445,7 @@ heap_page_prune(Relation relation, Buffer buffer, XLogRegisterBufData(0, (char *) prstate.nowunused, prstate.nunused * sizeof(OffsetNumber)); + XLogSetRecordFlags(XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP2_ID, XLOG_HEAP2_PRUNE); PageSetLSN(BufferGetPage(buffer), recptr); diff --git a/src/backend/access/heap/rewriteheap.c b/src/backend/access/heap/rewriteheap.c index 424958912c..9f77537335 100644 --- a/src/backend/access/heap/rewriteheap.c +++ b/src/backend/access/heap/rewriteheap.c @@ -921,6 +921,7 @@ logical_heap_rewrite_flush_mappings(RewriteState state) src->off += len; XLogBeginInsert(); + XLogSetRecordFlags(XLOG_INCLUDE_XID); XLogRegisterData((char *) (&xlrec), sizeof(xlrec)); XLogRegisterData(waldata_start, len); diff --git a/src/backend/access/transam/generic_xlog.c b/src/backend/access/transam/generic_xlog.c index 6c68191ca6..c8983b40ad 100644 --- a/src/backend/access/transam/generic_xlog.c +++ b/src/backend/access/transam/generic_xlog.c @@ -399,6 +399,9 @@ GenericXLogFinish(GenericXLogState *state) } } + /* Assume XID is required */ + XLogSetRecordFlags(XLOG_INCLUDE_XID); + /* Insert xlog record */ lsn = XLogInsert(RM_GENERIC_ID, 0); diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c index 4e1244541d..e2ff98a1d0 100644 --- a/src/backend/access/transam/multixact.c +++ b/src/backend/access/transam/multixact.c @@ -834,6 +834,7 @@ MultiXactIdCreateFromMembers(int nmembers, MultiXactMember *members) * Not clear that it's worth the trouble though. */ XLogBeginInsert(); + XLogSetRecordFlags(XLOG_INCLUDE_XID); XLogRegisterData((char *) (&xlrec), SizeOfMultiXactCreate); XLogRegisterData((char *) members, nmembers * sizeof(MultiXactMember)); diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index a70cb3e124..003f12a97d 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -5794,7 +5794,7 @@ XactLogCommitRecord(TimestampTz commit_time, XLogRegisterData((char *) (&xl_origin), sizeof(xl_xact_origin)); /* we allow filtering by xacts */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); return XLogInsertExtended(RM_XACT_ID, info, rmgr_info); } @@ -5941,7 +5941,7 @@ XactLogAbortRecord(TimestampTz abort_time, XLogRegisterData((char *) (&xl_origin), sizeof(xl_xact_origin)); /* Include the replication origin */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); return XLogInsertExtended(RM_XACT_ID, info, rmgr_info); } diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c index 76b508d535..b105d90e3f 100644 --- a/src/backend/access/transam/xloginsert.c +++ b/src/backend/access/transam/xloginsert.c @@ -456,6 +456,10 @@ XLogSetRecordFlags(uint8 flags) * (LSN is the XLOG point up to which the XLOG must be flushed to disk * before the data page can be written out. This implements the basic * WAL rule "write the log before the data".) + * + * Note: To include the current backend's TransactionID in the record, + * you have to set the XLOG_INCLUDE_XID flag using XLogRecordSetFlags + * before calling XLogInsert. */ XLogRecPtr XLogInsert(RmgrId rmid, uint8 rmgr_info) @@ -847,7 +851,7 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP } /* followed by toplevel XID, if not already included in previous record */ - if (IsSubxactTopXidLogPending()) + if (curinsert_flags & XLOG_INCLUDE_XID && IsSubxactTopXidLogPending()) { TransactionId xid = GetTopTransactionIdIfAny(); @@ -924,7 +928,11 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP * once we know where in the WAL the record will be inserted. The CRC does * not include the record header yet. */ - rechdr->xl_xid = GetCurrentTransactionIdIfAny(); + if (curinsert_flags & XLOG_INCLUDE_XID) + rechdr->xl_xid = GetCurrentTransactionIdIfAny(); + else + rechdr->xl_xid = InvalidTransactionId; + rechdr->xl_tot_len = (uint32) total_len; rechdr->xl_info = info; rechdr->xl_rmid = rmid; diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index c7a8a689b7..deeb905942 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -2132,7 +2132,7 @@ ExecuteTruncateGuts(List *explicit_rels, XLogRegisterData((char *) &xlrec, SizeOfHeapTruncate); XLogRegisterData((char *) logrelids, list_length(relids_logged) * sizeof(Oid)); - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); (void) XLogInsert(RM_HEAP_ID, XLOG_HEAP_TRUNCATE); } diff --git a/src/backend/replication/logical/message.c b/src/backend/replication/logical/message.c index c31dc1203f..af86d18457 100644 --- a/src/backend/replication/logical/message.c +++ b/src/backend/replication/logical/message.c @@ -69,7 +69,7 @@ LogLogicalMessage(const char *prefix, const char *message, size_t size, XLogRegisterData(unconstify(char *, message), size); /* allow origin filtering */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); return XLogInsert(RM_LOGICALMSG_ID, XLOG_LOGICAL_MESSAGE); } diff --git a/src/backend/utils/cache/inval.c b/src/backend/utils/cache/inval.c index 0008826f67..b0afdd6f90 100644 --- a/src/backend/utils/cache/inval.c +++ b/src/backend/utils/cache/inval.c @@ -1632,6 +1632,7 @@ LogLogicalInvalidations(void) ProcessMessageSubGroupMulti(group, RelCacheMsgs, XLogRegisterData((char *) msgs, n * sizeof(SharedInvalidationMessage))); + XLogSetRecordFlags(XLOG_INCLUDE_XID); XLogInsert(RM_XACT_ID, XLOG_XACT_INVALIDATIONS); } } diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h index 9007f559f3..f4dfaeda29 100644 --- a/src/include/access/xlog.h +++ b/src/include/access/xlog.h @@ -150,6 +150,7 @@ extern PGDLLIMPORT bool XLOG_DEBUG; */ #define XLOG_INCLUDE_ORIGIN 0x01 /* include the replication origin */ #define XLOG_MARK_UNIMPORTANT 0x02 /* record not important for durability */ +#define XLOG_INCLUDE_XID 0x04 /* include the transaction ID */ /* Checkpoint statistics */ -- 2.39.0 [application/octet-stream] v1-0003-Rename-rmgr_identify-info-bits-argument-to-rmgrin.patch (21.6K, ../../CAEze2WgcW5SkPO_TGTqirRcCS_FN0xEZqUsXDE6pPz2TYzXQBA@mail.gmail.com/5-v1-0003-Rename-rmgr_identify-info-bits-argument-to-rmgrin.patch) download | inline diff: From 892c9e5957315cccc88a2c747ccca88ed57b31df Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Mon, 23 Jan 2023 17:31:18 +0100 Subject: [PATCH v1 3/6] Rename rmgr_identify 'info'-bits argument to rmgrinfo This indicates the provided bits are fully owned by the rmgr, and none of the xlog record's owned bits are shared with the rmgr anymore. --- src/backend/access/rmgrdesc/brindesc.c | 4 ++-- src/backend/access/rmgrdesc/clogdesc.c | 4 ++-- src/backend/access/rmgrdesc/dbasedesc.c | 4 ++-- src/backend/access/rmgrdesc/gindesc.c | 4 ++-- src/backend/access/rmgrdesc/gistdesc.c | 4 ++-- src/backend/access/rmgrdesc/hashdesc.c | 4 ++-- src/backend/access/rmgrdesc/heapdesc.c | 8 ++++---- src/backend/access/rmgrdesc/logicalmsgdesc.c | 4 ++-- src/backend/access/rmgrdesc/mxactdesc.c | 4 ++-- src/backend/access/rmgrdesc/nbtdesc.c | 4 ++-- src/backend/access/rmgrdesc/relmapdesc.c | 4 ++-- src/backend/access/rmgrdesc/seqdesc.c | 4 ++-- src/backend/access/rmgrdesc/smgrdesc.c | 4 ++-- src/backend/access/rmgrdesc/spgdesc.c | 4 ++-- src/backend/access/rmgrdesc/standbydesc.c | 4 ++-- src/backend/access/rmgrdesc/tblspcdesc.c | 4 ++-- src/backend/access/rmgrdesc/xlogdesc.c | 4 ++-- src/include/access/brin_xlog.h | 2 +- src/include/access/clog.h | 2 +- src/include/access/ginxlog.h | 2 +- src/include/access/gistxlog.h | 2 +- src/include/access/hash_xlog.h | 2 +- src/include/access/heapam_xlog.h | 4 ++-- src/include/access/multixact.h | 2 +- src/include/access/nbtxlog.h | 2 +- src/include/access/spgxlog.h | 2 +- src/include/access/xlog.h | 2 +- src/include/catalog/storage_xlog.h | 2 +- src/include/commands/dbcommands_xlog.h | 2 +- src/include/commands/sequence.h | 2 +- src/include/commands/tablespace.h | 2 +- src/include/replication/message.h | 2 +- src/include/storage/standbydefs.h | 2 +- src/include/utils/relmapper.h | 2 +- src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c | 4 ++-- 35 files changed, 56 insertions(+), 56 deletions(-) diff --git a/src/backend/access/rmgrdesc/brindesc.c b/src/backend/access/rmgrdesc/brindesc.c index 1d3e5ee03e..fa272f0720 100644 --- a/src/backend/access/rmgrdesc/brindesc.c +++ b/src/backend/access/rmgrdesc/brindesc.c @@ -71,11 +71,11 @@ brin_desc(StringInfo buf, XLogReaderState *record) } const char * -brin_identify(uint8 info) +brin_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_BRIN_CREATE_INDEX: id = "CREATE_INDEX"; diff --git a/src/backend/access/rmgrdesc/clogdesc.c b/src/backend/access/rmgrdesc/clogdesc.c index 915f9945d0..b368af595e 100644 --- a/src/backend/access/rmgrdesc/clogdesc.c +++ b/src/backend/access/rmgrdesc/clogdesc.c @@ -41,11 +41,11 @@ clog_desc(StringInfo buf, XLogReaderState *record) } const char * -clog_identify(uint8 info) +clog_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case CLOG_ZEROPAGE: id = "ZEROPAGE"; diff --git a/src/backend/access/rmgrdesc/dbasedesc.c b/src/backend/access/rmgrdesc/dbasedesc.c index a8e5699371..f0e6ab88b6 100644 --- a/src/backend/access/rmgrdesc/dbasedesc.c +++ b/src/backend/access/rmgrdesc/dbasedesc.c @@ -54,11 +54,11 @@ dbase_desc(StringInfo buf, XLogReaderState *record) } const char * -dbase_identify(uint8 info) +dbase_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_DBASE_CREATE_FILE_COPY: id = "CREATE_FILE_COPY"; diff --git a/src/backend/access/rmgrdesc/gindesc.c b/src/backend/access/rmgrdesc/gindesc.c index 18fc07ba29..ee17eee18a 100644 --- a/src/backend/access/rmgrdesc/gindesc.c +++ b/src/backend/access/rmgrdesc/gindesc.c @@ -179,11 +179,11 @@ gin_desc(StringInfo buf, XLogReaderState *record) } const char * -gin_identify(uint8 info) +gin_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_GIN_CREATE_PTREE: id = "CREATE_PTREE"; diff --git a/src/backend/access/rmgrdesc/gistdesc.c b/src/backend/access/rmgrdesc/gistdesc.c index 6473069c1a..899e12bcc3 100644 --- a/src/backend/access/rmgrdesc/gistdesc.c +++ b/src/backend/access/rmgrdesc/gistdesc.c @@ -86,11 +86,11 @@ gist_desc(StringInfo buf, XLogReaderState *record) } const char * -gist_identify(uint8 info) +gist_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_GIST_PAGE_UPDATE: id = "PAGE_UPDATE"; diff --git a/src/backend/access/rmgrdesc/hashdesc.c b/src/backend/access/rmgrdesc/hashdesc.c index bcb2013048..451251caca 100644 --- a/src/backend/access/rmgrdesc/hashdesc.c +++ b/src/backend/access/rmgrdesc/hashdesc.c @@ -122,11 +122,11 @@ hash_desc(StringInfo buf, XLogReaderState *record) } const char * -hash_identify(uint8 info) +hash_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_HASH_INIT_META_PAGE: id = "INIT_META_PAGE"; diff --git a/src/backend/access/rmgrdesc/heapdesc.c b/src/backend/access/rmgrdesc/heapdesc.c index 15f943051a..5224869af1 100644 --- a/src/backend/access/rmgrdesc/heapdesc.c +++ b/src/backend/access/rmgrdesc/heapdesc.c @@ -302,11 +302,11 @@ heap2_desc(StringInfo buf, XLogReaderState *record) } const char * -heap_identify(uint8 info) +heap_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_HEAP_INSERT: id = "INSERT"; @@ -347,11 +347,11 @@ heap_identify(uint8 info) } const char * -heap2_identify(uint8 info) +heap2_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_HEAP2_PRUNE: id = "PRUNE"; diff --git a/src/backend/access/rmgrdesc/logicalmsgdesc.c b/src/backend/access/rmgrdesc/logicalmsgdesc.c index 2bb6cb06c3..7566630961 100644 --- a/src/backend/access/rmgrdesc/logicalmsgdesc.c +++ b/src/backend/access/rmgrdesc/logicalmsgdesc.c @@ -43,9 +43,9 @@ logicalmsg_desc(StringInfo buf, XLogReaderState *record) } const char * -logicalmsg_identify(uint8 info) +logicalmsg_identify(uint8 rmgrinfo) { - if ((info & ~XLR_INFO_MASK) == XLOG_LOGICAL_MESSAGE) + if (rmgrinfo == XLOG_LOGICAL_MESSAGE) return "MESSAGE"; return NULL; diff --git a/src/backend/access/rmgrdesc/mxactdesc.c b/src/backend/access/rmgrdesc/mxactdesc.c index cdcf8a4f45..7676778239 100644 --- a/src/backend/access/rmgrdesc/mxactdesc.c +++ b/src/backend/access/rmgrdesc/mxactdesc.c @@ -81,11 +81,11 @@ multixact_desc(StringInfo buf, XLogReaderState *record) } const char * -multixact_identify(uint8 info) +multixact_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_MULTIXACT_ZERO_OFF_PAGE: id = "ZERO_OFF_PAGE"; diff --git a/src/backend/access/rmgrdesc/nbtdesc.c b/src/backend/access/rmgrdesc/nbtdesc.c index ede8df72c0..ff9e28ca55 100644 --- a/src/backend/access/rmgrdesc/nbtdesc.c +++ b/src/backend/access/rmgrdesc/nbtdesc.c @@ -134,11 +134,11 @@ btree_desc(StringInfo buf, XLogReaderState *record) } const char * -btree_identify(uint8 info) +btree_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_BTREE_INSERT_LEAF: id = "INSERT_LEAF"; diff --git a/src/backend/access/rmgrdesc/relmapdesc.c b/src/backend/access/rmgrdesc/relmapdesc.c index f4f0e5ce8b..9c9f191700 100644 --- a/src/backend/access/rmgrdesc/relmapdesc.c +++ b/src/backend/access/rmgrdesc/relmapdesc.c @@ -32,11 +32,11 @@ relmap_desc(StringInfo buf, XLogReaderState *record) } const char * -relmap_identify(uint8 info) +relmap_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_RELMAP_UPDATE: id = "UPDATE"; diff --git a/src/backend/access/rmgrdesc/seqdesc.c b/src/backend/access/rmgrdesc/seqdesc.c index 6def7b653b..eed631b5d8 100644 --- a/src/backend/access/rmgrdesc/seqdesc.c +++ b/src/backend/access/rmgrdesc/seqdesc.c @@ -31,11 +31,11 @@ seq_desc(StringInfo buf, XLogReaderState *record) } const char * -seq_identify(uint8 info) +seq_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_SEQ_LOG: id = "LOG"; diff --git a/src/backend/access/rmgrdesc/smgrdesc.c b/src/backend/access/rmgrdesc/smgrdesc.c index 8604bee0c0..653caf58e4 100644 --- a/src/backend/access/rmgrdesc/smgrdesc.c +++ b/src/backend/access/rmgrdesc/smgrdesc.c @@ -43,11 +43,11 @@ smgr_desc(StringInfo buf, XLogReaderState *record) } const char * -smgr_identify(uint8 info) +smgr_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_SMGR_CREATE: id = "CREATE"; diff --git a/src/backend/access/rmgrdesc/spgdesc.c b/src/backend/access/rmgrdesc/spgdesc.c index 9e1e48240b..0d1af9a5b6 100644 --- a/src/backend/access/rmgrdesc/spgdesc.c +++ b/src/backend/access/rmgrdesc/spgdesc.c @@ -128,11 +128,11 @@ spg_desc(StringInfo buf, XLogReaderState *record) } const char * -spg_identify(uint8 info) +spg_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_SPGIST_ADD_LEAF: id = "ADD_LEAF"; diff --git a/src/backend/access/rmgrdesc/standbydesc.c b/src/backend/access/rmgrdesc/standbydesc.c index 9a08e10fe1..56acea1407 100644 --- a/src/backend/access/rmgrdesc/standbydesc.c +++ b/src/backend/access/rmgrdesc/standbydesc.c @@ -76,11 +76,11 @@ standby_desc(StringInfo buf, XLogReaderState *record) } const char * -standby_identify(uint8 info) +standby_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_STANDBY_LOCK: id = "LOCK"; diff --git a/src/backend/access/rmgrdesc/tblspcdesc.c b/src/backend/access/rmgrdesc/tblspcdesc.c index c1feb4fa05..c5849ccf7c 100644 --- a/src/backend/access/rmgrdesc/tblspcdesc.c +++ b/src/backend/access/rmgrdesc/tblspcdesc.c @@ -38,11 +38,11 @@ tblspc_desc(StringInfo buf, XLogReaderState *record) } const char * -tblspc_identify(uint8 info) +tblspc_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_TBLSPC_CREATE: id = "CREATE"; diff --git a/src/backend/access/rmgrdesc/xlogdesc.c b/src/backend/access/rmgrdesc/xlogdesc.c index 3692dfb96a..4db4f3331f 100644 --- a/src/backend/access/rmgrdesc/xlogdesc.c +++ b/src/backend/access/rmgrdesc/xlogdesc.c @@ -151,11 +151,11 @@ xlog_desc(StringInfo buf, XLogReaderState *record) } const char * -xlog_identify(uint8 info) +xlog_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_CHECKPOINT_SHUTDOWN: id = "CHECKPOINT_SHUTDOWN"; diff --git a/src/include/access/brin_xlog.h b/src/include/access/brin_xlog.h index ae263a371f..45a688d09b 100644 --- a/src/include/access/brin_xlog.h +++ b/src/include/access/brin_xlog.h @@ -145,7 +145,7 @@ typedef struct xl_brin_desummarize extern void brin_redo(XLogReaderState *record); extern void brin_desc(StringInfo buf, XLogReaderState *record); -extern const char *brin_identify(uint8 info); +extern const char *brin_identify(uint8 rmgrinfo); extern void brin_mask(char *pagedata, BlockNumber blkno); #endif /* BRIN_XLOG_H */ diff --git a/src/include/access/clog.h b/src/include/access/clog.h index d99444f073..224cad4415 100644 --- a/src/include/access/clog.h +++ b/src/include/access/clog.h @@ -58,6 +58,6 @@ extern int clogsyncfiletag(const FileTag *ftag, char *path); extern void clog_redo(XLogReaderState *record); extern void clog_desc(StringInfo buf, XLogReaderState *record); -extern const char *clog_identify(uint8 info); +extern const char *clog_identify(uint8 rmgrinfo); #endif /* CLOG_H */ diff --git a/src/include/access/ginxlog.h b/src/include/access/ginxlog.h index 37095e5f17..adf1f6b6d8 100644 --- a/src/include/access/ginxlog.h +++ b/src/include/access/ginxlog.h @@ -208,7 +208,7 @@ typedef struct ginxlogDeleteListPages extern void gin_redo(XLogReaderState *record); extern void gin_desc(StringInfo buf, XLogReaderState *record); -extern const char *gin_identify(uint8 info); +extern const char *gin_identify(uint8 rmgrinfo); extern void gin_xlog_startup(void); extern void gin_xlog_cleanup(void); extern void gin_mask(char *pagedata, BlockNumber blkno); diff --git a/src/include/access/gistxlog.h b/src/include/access/gistxlog.h index aff2ffbdcc..0ca37f3a07 100644 --- a/src/include/access/gistxlog.h +++ b/src/include/access/gistxlog.h @@ -109,7 +109,7 @@ typedef struct gistxlogPageReuse extern void gist_redo(XLogReaderState *record); extern void gist_desc(StringInfo buf, XLogReaderState *record); -extern const char *gist_identify(uint8 info); +extern const char *gist_identify(uint8 rmgrinfo); extern void gist_xlog_startup(void); extern void gist_xlog_cleanup(void); extern void gist_mask(char *pagedata, BlockNumber blkno); diff --git a/src/include/access/hash_xlog.h b/src/include/access/hash_xlog.h index b93619d1a8..74859a2a47 100644 --- a/src/include/access/hash_xlog.h +++ b/src/include/access/hash_xlog.h @@ -263,7 +263,7 @@ typedef struct xl_hash_vacuum_one_page extern void hash_redo(XLogReaderState *record); extern void hash_desc(StringInfo buf, XLogReaderState *record); -extern const char *hash_identify(uint8 info); +extern const char *hash_identify(uint8 rmgrinfo); extern void hash_mask(char *pagedata, BlockNumber blkno); #endif /* HASH_XLOG_H */ diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h index a038450787..70e9acd350 100644 --- a/src/include/access/heapam_xlog.h +++ b/src/include/access/heapam_xlog.h @@ -406,11 +406,11 @@ extern void HeapTupleHeaderAdvanceConflictHorizon(HeapTupleHeader tuple, extern void heap_redo(XLogReaderState *record); extern void heap_desc(StringInfo buf, XLogReaderState *record); -extern const char *heap_identify(uint8 info); +extern const char *heap_identify(uint8 rmgrinfo); extern void heap_mask(char *pagedata, BlockNumber blkno); extern void heap2_redo(XLogReaderState *record); extern void heap2_desc(StringInfo buf, XLogReaderState *record); -extern const char *heap2_identify(uint8 info); +extern const char *heap2_identify(uint8 rmgrinfo); extern void heap_xlog_logical_rewrite(XLogReaderState *r); extern XLogRecPtr log_heap_visible(Relation rel, Buffer heap_buffer, diff --git a/src/include/access/multixact.h b/src/include/access/multixact.h index 246f757f6a..d52e69aa05 100644 --- a/src/include/access/multixact.h +++ b/src/include/access/multixact.h @@ -158,7 +158,7 @@ extern void multixact_twophase_postabort(TransactionId xid, uint16 info, extern void multixact_redo(XLogReaderState *record); extern void multixact_desc(StringInfo buf, XLogReaderState *record); -extern const char *multixact_identify(uint8 info); +extern const char *multixact_identify(uint8 rmgrinfo); extern char *mxid_to_string(MultiXactId multi, int nmembers, MultiXactMember *members); diff --git a/src/include/access/nbtxlog.h b/src/include/access/nbtxlog.h index 7dd9fd0206..422ac13d56 100644 --- a/src/include/access/nbtxlog.h +++ b/src/include/access/nbtxlog.h @@ -362,6 +362,6 @@ extern void btree_mask(char *pagedata, BlockNumber blkno); * prototypes for functions in nbtdesc.c */ extern void btree_desc(StringInfo buf, XLogReaderState *record); -extern const char *btree_identify(uint8 info); +extern const char *btree_identify(uint8 rmgrinfo); #endif /* NBTXLOG_H */ diff --git a/src/include/access/spgxlog.h b/src/include/access/spgxlog.h index a7f10bf2d3..3533894baf 100644 --- a/src/include/access/spgxlog.h +++ b/src/include/access/spgxlog.h @@ -251,7 +251,7 @@ typedef struct spgxlogVacuumRedirect extern void spg_redo(XLogReaderState *record); extern void spg_desc(StringInfo buf, XLogReaderState *record); -extern const char *spg_identify(uint8 info); +extern const char *spg_identify(uint8 rmgrinfo); extern void spg_xlog_startup(void); extern void spg_xlog_cleanup(void); extern void spg_mask(char *pagedata, BlockNumber blkno); diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h index 48ca852381..9007f559f3 100644 --- a/src/include/access/xlog.h +++ b/src/include/access/xlog.h @@ -211,7 +211,7 @@ extern void XLogSetReplicationSlotMinimumLSN(XLogRecPtr lsn); extern void xlog_redo(struct XLogReaderState *record); extern void xlog_desc(StringInfo buf, struct XLogReaderState *record); -extern const char *xlog_identify(uint8 info); +extern const char *xlog_identify(uint8 rmgrinfo); extern void issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli); diff --git a/src/include/catalog/storage_xlog.h b/src/include/catalog/storage_xlog.h index 6b0a7aa3df..27514163be 100644 --- a/src/include/catalog/storage_xlog.h +++ b/src/include/catalog/storage_xlog.h @@ -54,6 +54,6 @@ extern void log_smgrcreate(const RelFileLocator *rlocator, ForkNumber forkNum); extern void smgr_redo(XLogReaderState *record); extern void smgr_desc(StringInfo buf, XLogReaderState *record); -extern const char *smgr_identify(uint8 info); +extern const char *smgr_identify(uint8 rmgrinfo); #endif /* STORAGE_XLOG_H */ diff --git a/src/include/commands/dbcommands_xlog.h b/src/include/commands/dbcommands_xlog.h index a1433275f5..f3c11060b6 100644 --- a/src/include/commands/dbcommands_xlog.h +++ b/src/include/commands/dbcommands_xlog.h @@ -55,6 +55,6 @@ typedef struct xl_dbase_drop_rec extern void dbase_redo(XLogReaderState *record); extern void dbase_desc(StringInfo buf, XLogReaderState *record); -extern const char *dbase_identify(uint8 info); +extern const char *dbase_identify(uint8 rmgrinfo); #endif /* DBCOMMANDS_XLOG_H */ diff --git a/src/include/commands/sequence.h b/src/include/commands/sequence.h index 7db7b3da7b..fb7059b8f6 100644 --- a/src/include/commands/sequence.h +++ b/src/include/commands/sequence.h @@ -64,7 +64,7 @@ extern void ResetSequenceCaches(void); extern void seq_redo(XLogReaderState *record); extern void seq_desc(StringInfo buf, XLogReaderState *record); -extern const char *seq_identify(uint8 info); +extern const char *seq_identify(uint8 rmgrinfo); extern void seq_mask(char *page, BlockNumber blkno); #endif /* SEQUENCE_H */ diff --git a/src/include/commands/tablespace.h b/src/include/commands/tablespace.h index f1961c1813..db21eb59ee 100644 --- a/src/include/commands/tablespace.h +++ b/src/include/commands/tablespace.h @@ -64,6 +64,6 @@ extern void remove_tablespace_symlink(const char *linkloc); extern void tblspc_redo(XLogReaderState *record); extern void tblspc_desc(StringInfo buf, XLogReaderState *record); -extern const char *tblspc_identify(uint8 info); +extern const char *tblspc_identify(uint8 rmgrinfo); #endif /* TABLESPACE_H */ diff --git a/src/include/replication/message.h b/src/include/replication/message.h index 6ce7f2038b..2fe985a6fe 100644 --- a/src/include/replication/message.h +++ b/src/include/replication/message.h @@ -36,6 +36,6 @@ extern XLogRecPtr LogLogicalMessage(const char *prefix, const char *message, #define XLOG_LOGICAL_MESSAGE 0x00 extern void logicalmsg_redo(XLogReaderState *record); extern void logicalmsg_desc(StringInfo buf, XLogReaderState *record); -extern const char *logicalmsg_identify(uint8 info); +extern const char *logicalmsg_identify(uint8 rmgrinfo); #endif /* PG_LOGICAL_MESSAGE_H */ diff --git a/src/include/storage/standbydefs.h b/src/include/storage/standbydefs.h index 188e348618..84e1f03c8f 100644 --- a/src/include/storage/standbydefs.h +++ b/src/include/storage/standbydefs.h @@ -22,7 +22,7 @@ /* Recovery handlers for the Standby Rmgr (RM_STANDBY_ID) */ extern void standby_redo(XLogReaderState *record); extern void standby_desc(StringInfo buf, XLogReaderState *record); -extern const char *standby_identify(uint8 info); +extern const char *standby_identify(uint8 rmgrinfo); extern void standby_desc_invalidations(StringInfo buf, int nmsgs, SharedInvalidationMessage *msgs, Oid dbId, Oid tsId, diff --git a/src/include/utils/relmapper.h b/src/include/utils/relmapper.h index 5c173bdbb3..e3c41ff093 100644 --- a/src/include/utils/relmapper.h +++ b/src/include/utils/relmapper.h @@ -68,6 +68,6 @@ extern void RestoreRelationMap(char *startAddress); extern void relmap_redo(XLogReaderState *record); extern void relmap_desc(StringInfo buf, XLogReaderState *record); -extern const char *relmap_identify(uint8 info); +extern const char *relmap_identify(uint8 rmgrinfo); #endif /* RELMAPPER_H */ diff --git a/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c b/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c index 6921308937..6b443fb8c5 100644 --- a/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c +++ b/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c @@ -103,9 +103,9 @@ testcustomrmgrs_desc(StringInfo buf, XLogReaderState *record) } const char * -testcustomrmgrs_identify(uint8 info) +testcustomrmgrs_identify(uint8 rmgrinfo) { - if ((info & ~XLR_INFO_MASK) == XLOG_TEST_CUSTOM_RMGRS_MESSAGE) + if (rmgrinfo == XLOG_TEST_CUSTOM_RMGRS_MESSAGE) return "TEST_CUSTOM_RMGRS_MESSAGE"; return NULL; -- 2.39.0 [application/octet-stream] v1-0002-Replace-accesses-to-rmgr-s-XLogRecord-info-bit-ac.patch (44.3K, ../../CAEze2WgcW5SkPO_TGTqirRcCS_FN0xEZqUsXDE6pPz2TYzXQBA@mail.gmail.com/6-v1-0002-Replace-accesses-to-rmgr-s-XLogRecord-info-bit-ac.patch) download | inline diff: From 9456fe9503743390c1244f1db9d926e278813245 Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Mon, 23 Jan 2023 16:24:01 +0100 Subject: [PATCH v1 2/6] Replace accesses to rmgr's XLogRecord info bit accesses with XLogRecGetRmgrInfo This allows refactoring of the info bits into its own struct field, instead of sharing the field with the records own info bits. --- contrib/pg_walinspect/pg_walinspect.c | 8 ++--- src/backend/access/brin/brin_xlog.c | 4 +-- src/backend/access/gin/ginxlog.c | 2 +- src/backend/access/gist/gistxlog.c | 2 +- src/backend/access/hash/hash_xlog.c | 2 +- src/backend/access/heap/heapam.c | 10 +++---- src/backend/access/nbtree/nbtxlog.c | 2 +- src/backend/access/rmgrdesc/brindesc.c | 2 +- src/backend/access/rmgrdesc/clogdesc.c | 2 +- src/backend/access/rmgrdesc/committsdesc.c | 2 +- src/backend/access/rmgrdesc/dbasedesc.c | 2 +- src/backend/access/rmgrdesc/gindesc.c | 2 +- src/backend/access/rmgrdesc/gistdesc.c | 2 +- src/backend/access/rmgrdesc/hashdesc.c | 2 +- src/backend/access/rmgrdesc/heapdesc.c | 6 ++-- src/backend/access/rmgrdesc/logicalmsgdesc.c | 2 +- src/backend/access/rmgrdesc/mxactdesc.c | 2 +- src/backend/access/rmgrdesc/nbtdesc.c | 2 +- src/backend/access/rmgrdesc/relmapdesc.c | 2 +- src/backend/access/rmgrdesc/replorigindesc.c | 2 +- src/backend/access/rmgrdesc/seqdesc.c | 2 +- src/backend/access/rmgrdesc/smgrdesc.c | 2 +- src/backend/access/rmgrdesc/spgdesc.c | 2 +- src/backend/access/rmgrdesc/standbydesc.c | 2 +- src/backend/access/rmgrdesc/tblspcdesc.c | 2 +- src/backend/access/rmgrdesc/xactdesc.c | 30 +++++++++---------- src/backend/access/rmgrdesc/xlogdesc.c | 2 +- src/backend/access/spgist/spgxlog.c | 2 +- src/backend/access/transam/clog.c | 2 +- src/backend/access/transam/commit_ts.c | 2 +- src/backend/access/transam/multixact.c | 2 +- src/backend/access/transam/twophase.c | 2 +- src/backend/access/transam/xact.c | 10 +++---- src/backend/access/transam/xlog.c | 2 +- src/backend/access/transam/xlogrecovery.c | 22 +++++++------- src/backend/access/transam/xlogstats.c | 2 +- src/backend/catalog/storage.c | 2 +- src/backend/commands/dbcommands.c | 2 +- src/backend/commands/sequence.c | 2 +- src/backend/commands/tablespace.c | 2 +- src/backend/replication/logical/decode.c | 20 +++++++------ src/backend/replication/logical/message.c | 2 +- src/backend/replication/logical/origin.c | 2 +- src/backend/storage/ipc/standby.c | 2 +- src/backend/utils/cache/relmapper.c | 2 +- src/bin/pg_rewind/parsexlog.c | 6 ++-- src/bin/pg_waldump/pg_waldump.c | 4 +-- src/include/access/xact.h | 6 ++-- src/include/access/xlogreader.h | 3 +- .../test_custom_rmgrs/test_custom_rmgrs.c | 4 +-- 50 files changed, 105 insertions(+), 102 deletions(-) diff --git a/contrib/pg_walinspect/pg_walinspect.c b/contrib/pg_walinspect/pg_walinspect.c index 1cd3744d5d..de65a84aba 100644 --- a/contrib/pg_walinspect/pg_walinspect.c +++ b/contrib/pg_walinspect/pg_walinspect.c @@ -196,10 +196,10 @@ GetWALRecordInfo(XLogReaderState *record, Datum *values, int i = 0; desc = GetRmgr(XLogRecGetRmid(record)); - record_type = desc.rm_identify(XLogRecGetInfo(record)); + record_type = desc.rm_identify(XLogRecGetRmgrInfo(record)); if (record_type == NULL) - record_type = psprintf("UNKNOWN (%x)", XLogRecGetInfo(record) & ~XLR_INFO_MASK); + record_type = psprintf("UNKNOWN (%x)", XLogRecGetRmgrInfo(record)); initStringInfo(&rec_desc); desc.rm_desc(&rec_desc, record); @@ -258,11 +258,11 @@ GetWALBlockInfo(FunctionCallInfo fcinfo, XLogReaderState *record, Assert(XLogRecHasAnyBlockRefs(record)); desc = GetRmgr(XLogRecGetRmid(record)); - record_type = desc.rm_identify(XLogRecGetInfo(record)); + record_type = desc.rm_identify(XLogRecGetRmgrInfo(record)); if (record_type == NULL) record_type = psprintf("UNKNOWN (%x)", - XLogRecGetInfo(record) & ~XLR_INFO_MASK); + XLogRecGetRmgrInfo(record)); initStringInfo(&rec_desc); desc.rm_desc(&rec_desc, record); diff --git a/src/backend/access/brin/brin_xlog.c b/src/backend/access/brin/brin_xlog.c index 89145b68f6..00dd85bdc5 100644 --- a/src/backend/access/brin/brin_xlog.c +++ b/src/backend/access/brin/brin_xlog.c @@ -56,7 +56,7 @@ brin_xlog_insert_update(XLogReaderState *record, * If we inserted the first and only tuple on the page, re-initialize the * page from scratch. */ - if (XLogRecGetInfo(record) & XLOG_BRIN_INIT_PAGE) + if (XLogRecGetRmgrInfo(record) & XLOG_BRIN_INIT_PAGE) { buffer = XLogInitBufferForRedo(record, 0); page = BufferGetPage(buffer); @@ -308,7 +308,7 @@ brin_xlog_desummarize_page(XLogReaderState *record) void brin_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info & XLOG_BRIN_OPMASK) { diff --git a/src/backend/access/gin/ginxlog.c b/src/backend/access/gin/ginxlog.c index f7c84beef8..feb675ed19 100644 --- a/src/backend/access/gin/ginxlog.c +++ b/src/backend/access/gin/ginxlog.c @@ -725,7 +725,7 @@ ginRedoDeleteListPages(XLogReaderState *record) void gin_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); MemoryContext oldCtx; /* diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index a2ddfd5e69..433f7ee484 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -398,7 +398,7 @@ gistRedoPageReuse(XLogReaderState *record) void gist_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); MemoryContext oldCxt; /* diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index e8e06c62a9..3a45b7ee21 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -1048,7 +1048,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) void hash_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 0124f37911..2a2525cee9 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -9255,7 +9255,7 @@ heap_xlog_insert(XLogReaderState *record) * If we inserted the first and only tuple on the page, re-initialize the * page from scratch. */ - if (XLogRecGetInfo(record) & XLOG_HEAP_INIT_PAGE) + if (XLogRecGetRmgrInfo(record) & XLOG_HEAP_INIT_PAGE) { buffer = XLogInitBufferForRedo(record, 0); page = BufferGetPage(buffer); @@ -9349,7 +9349,7 @@ heap_xlog_multi_insert(XLogReaderState *record) uint32 newlen; Size freespace = 0; int i; - bool isinit = (XLogRecGetInfo(record) & XLOG_HEAP_INIT_PAGE) != 0; + bool isinit = (XLogRecGetRmgrInfo(record) & XLOG_HEAP_INIT_PAGE) != 0; XLogRedoAction action; /* @@ -9597,7 +9597,7 @@ heap_xlog_update(XLogReaderState *record, bool hot_update) nbuffer = obuffer; newaction = oldaction; } - else if (XLogRecGetInfo(record) & XLOG_HEAP_INIT_PAGE) + else if (XLogRecGetRmgrInfo(record) & XLOG_HEAP_INIT_PAGE) { nbuffer = XLogInitBufferForRedo(record, 0); page = (Page) BufferGetPage(nbuffer); @@ -9961,7 +9961,7 @@ heap_xlog_inplace(XLogReaderState *record) void heap_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* * These operations don't overwrite MVCC data so no conflict processing is @@ -10007,7 +10007,7 @@ heap_redo(XLogReaderState *record) void heap2_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info & XLOG_HEAP_OPMASK) { diff --git a/src/backend/access/nbtree/nbtxlog.c b/src/backend/access/nbtree/nbtxlog.c index c87e46ed66..97d8d068c6 100644 --- a/src/backend/access/nbtree/nbtxlog.c +++ b/src/backend/access/nbtree/nbtxlog.c @@ -1015,7 +1015,7 @@ btree_xlog_reuse_page(XLogReaderState *record) void btree_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); MemoryContext oldCtx; oldCtx = MemoryContextSwitchTo(opCtx); diff --git a/src/backend/access/rmgrdesc/brindesc.c b/src/backend/access/rmgrdesc/brindesc.c index 1466a31bbb..1d3e5ee03e 100644 --- a/src/backend/access/rmgrdesc/brindesc.c +++ b/src/backend/access/rmgrdesc/brindesc.c @@ -20,7 +20,7 @@ void brin_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); info &= XLOG_BRIN_OPMASK; if (info == XLOG_BRIN_CREATE_INDEX) diff --git a/src/backend/access/rmgrdesc/clogdesc.c b/src/backend/access/rmgrdesc/clogdesc.c index e60b76f9da..915f9945d0 100644 --- a/src/backend/access/rmgrdesc/clogdesc.c +++ b/src/backend/access/rmgrdesc/clogdesc.c @@ -21,7 +21,7 @@ void clog_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == CLOG_ZEROPAGE) { diff --git a/src/backend/access/rmgrdesc/committsdesc.c b/src/backend/access/rmgrdesc/committsdesc.c index e7155cd507..3d673c2302 100644 --- a/src/backend/access/rmgrdesc/committsdesc.c +++ b/src/backend/access/rmgrdesc/committsdesc.c @@ -22,7 +22,7 @@ void commit_ts_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == COMMIT_TS_ZEROPAGE) { diff --git a/src/backend/access/rmgrdesc/dbasedesc.c b/src/backend/access/rmgrdesc/dbasedesc.c index 7d12e0ef91..a8e5699371 100644 --- a/src/backend/access/rmgrdesc/dbasedesc.c +++ b/src/backend/access/rmgrdesc/dbasedesc.c @@ -22,7 +22,7 @@ void dbase_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_DBASE_CREATE_FILE_COPY) { diff --git a/src/backend/access/rmgrdesc/gindesc.c b/src/backend/access/rmgrdesc/gindesc.c index 9ef4981ad1..18fc07ba29 100644 --- a/src/backend/access/rmgrdesc/gindesc.c +++ b/src/backend/access/rmgrdesc/gindesc.c @@ -74,7 +74,7 @@ void gin_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/access/rmgrdesc/gistdesc.c b/src/backend/access/rmgrdesc/gistdesc.c index 5dc6e1dcee..6473069c1a 100644 --- a/src/backend/access/rmgrdesc/gistdesc.c +++ b/src/backend/access/rmgrdesc/gistdesc.c @@ -60,7 +60,7 @@ void gist_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/access/rmgrdesc/hashdesc.c b/src/backend/access/rmgrdesc/hashdesc.c index b6810a9320..bcb2013048 100644 --- a/src/backend/access/rmgrdesc/hashdesc.c +++ b/src/backend/access/rmgrdesc/hashdesc.c @@ -20,7 +20,7 @@ void hash_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/access/rmgrdesc/heapdesc.c b/src/backend/access/rmgrdesc/heapdesc.c index d73248abdd..15f943051a 100644 --- a/src/backend/access/rmgrdesc/heapdesc.c +++ b/src/backend/access/rmgrdesc/heapdesc.c @@ -95,7 +95,7 @@ void heap_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); info &= XLOG_HEAP_OPMASK; if (info == XLOG_HEAP_INSERT) @@ -172,7 +172,7 @@ void heap2_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); info &= XLOG_HEAP_OPMASK; if (info == XLOG_HEAP2_PRUNE) @@ -265,7 +265,7 @@ heap2_desc(StringInfo buf, XLogReaderState *record) else if (info == XLOG_HEAP2_MULTI_INSERT) { xl_heap_multi_insert *xlrec = (xl_heap_multi_insert *) rec; - bool isinit = (XLogRecGetInfo(record) & XLOG_HEAP_INIT_PAGE) != 0; + bool isinit = (XLogRecGetRmgrInfo(record) & XLOG_HEAP_INIT_PAGE) != 0; appendStringInfo(buf, "ntuples: %d, flags: 0x%02X", xlrec->ntuples, xlrec->flags); diff --git a/src/backend/access/rmgrdesc/logicalmsgdesc.c b/src/backend/access/rmgrdesc/logicalmsgdesc.c index 0578ec5535..2bb6cb06c3 100644 --- a/src/backend/access/rmgrdesc/logicalmsgdesc.c +++ b/src/backend/access/rmgrdesc/logicalmsgdesc.c @@ -19,7 +19,7 @@ void logicalmsg_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_LOGICAL_MESSAGE) { diff --git a/src/backend/access/rmgrdesc/mxactdesc.c b/src/backend/access/rmgrdesc/mxactdesc.c index a2fa1eca18..cdcf8a4f45 100644 --- a/src/backend/access/rmgrdesc/mxactdesc.c +++ b/src/backend/access/rmgrdesc/mxactdesc.c @@ -50,7 +50,7 @@ void multixact_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_MULTIXACT_ZERO_OFF_PAGE || info == XLOG_MULTIXACT_ZERO_MEM_PAGE) diff --git a/src/backend/access/rmgrdesc/nbtdesc.c b/src/backend/access/rmgrdesc/nbtdesc.c index e4fbaa4d5d..ede8df72c0 100644 --- a/src/backend/access/rmgrdesc/nbtdesc.c +++ b/src/backend/access/rmgrdesc/nbtdesc.c @@ -24,7 +24,7 @@ void btree_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/access/rmgrdesc/relmapdesc.c b/src/backend/access/rmgrdesc/relmapdesc.c index 82ad51eb9a..f4f0e5ce8b 100644 --- a/src/backend/access/rmgrdesc/relmapdesc.c +++ b/src/backend/access/rmgrdesc/relmapdesc.c @@ -20,7 +20,7 @@ void relmap_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_RELMAP_UPDATE) { diff --git a/src/backend/access/rmgrdesc/replorigindesc.c b/src/backend/access/rmgrdesc/replorigindesc.c index 8ce4588fda..539dc57939 100644 --- a/src/backend/access/rmgrdesc/replorigindesc.c +++ b/src/backend/access/rmgrdesc/replorigindesc.c @@ -19,7 +19,7 @@ void replorigin_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/access/rmgrdesc/seqdesc.c b/src/backend/access/rmgrdesc/seqdesc.c index ba60544085..6def7b653b 100644 --- a/src/backend/access/rmgrdesc/seqdesc.c +++ b/src/backend/access/rmgrdesc/seqdesc.c @@ -21,7 +21,7 @@ void seq_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); xl_seq_rec *xlrec = (xl_seq_rec *) rec; if (info == XLOG_SEQ_LOG) diff --git a/src/backend/access/rmgrdesc/smgrdesc.c b/src/backend/access/rmgrdesc/smgrdesc.c index bd841b96e8..8604bee0c0 100644 --- a/src/backend/access/rmgrdesc/smgrdesc.c +++ b/src/backend/access/rmgrdesc/smgrdesc.c @@ -21,7 +21,7 @@ void smgr_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_SMGR_CREATE) { diff --git a/src/backend/access/rmgrdesc/spgdesc.c b/src/backend/access/rmgrdesc/spgdesc.c index 87f62f0fb4..9e1e48240b 100644 --- a/src/backend/access/rmgrdesc/spgdesc.c +++ b/src/backend/access/rmgrdesc/spgdesc.c @@ -20,7 +20,7 @@ void spg_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/access/rmgrdesc/standbydesc.c b/src/backend/access/rmgrdesc/standbydesc.c index f2bce9a37a..9a08e10fe1 100644 --- a/src/backend/access/rmgrdesc/standbydesc.c +++ b/src/backend/access/rmgrdesc/standbydesc.c @@ -47,7 +47,7 @@ void standby_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_STANDBY_LOCK) { diff --git a/src/backend/access/rmgrdesc/tblspcdesc.c b/src/backend/access/rmgrdesc/tblspcdesc.c index b8c89f8c54..c1feb4fa05 100644 --- a/src/backend/access/rmgrdesc/tblspcdesc.c +++ b/src/backend/access/rmgrdesc/tblspcdesc.c @@ -21,7 +21,7 @@ void tblspc_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_TBLSPC_CREATE) { diff --git a/src/backend/access/rmgrdesc/xactdesc.c b/src/backend/access/rmgrdesc/xactdesc.c index 01610c5ddc..8903bb32de 100644 --- a/src/backend/access/rmgrdesc/xactdesc.c +++ b/src/backend/access/rmgrdesc/xactdesc.c @@ -32,7 +32,7 @@ */ void -ParseCommitRecord(uint8 info, xl_xact_commit *xlrec, xl_xact_parsed_commit *parsed) +ParseCommitRecord(uint8 rmgrinfo, xl_xact_commit *xlrec, xl_xact_parsed_commit *parsed) { char *data = ((char *) xlrec) + MinSizeOfXactCommit; @@ -43,7 +43,7 @@ ParseCommitRecord(uint8 info, xl_xact_commit *xlrec, xl_xact_parsed_commit *pars parsed->xact_time = xlrec->xact_time; - if (info & XLOG_XACT_HAS_INFO) + if (rmgrinfo & XLOG_XACT_HAS_INFO) { xl_xact_xinfo *xl_xinfo = (xl_xact_xinfo *) data; @@ -138,7 +138,7 @@ ParseCommitRecord(uint8 info, xl_xact_commit *xlrec, xl_xact_parsed_commit *pars } void -ParseAbortRecord(uint8 info, xl_xact_abort *xlrec, xl_xact_parsed_abort *parsed) +ParseAbortRecord(uint8 rmgrinfo, xl_xact_abort *xlrec, xl_xact_parsed_abort *parsed) { char *data = ((char *) xlrec) + MinSizeOfXactAbort; @@ -149,7 +149,7 @@ ParseAbortRecord(uint8 info, xl_xact_abort *xlrec, xl_xact_parsed_abort *parsed) parsed->xact_time = xlrec->xact_time; - if (info & XLOG_XACT_HAS_INFO) + if (rmgrinfo & XLOG_XACT_HAS_INFO) { xl_xact_xinfo *xl_xinfo = (xl_xact_xinfo *) data; @@ -236,7 +236,7 @@ ParseAbortRecord(uint8 info, xl_xact_abort *xlrec, xl_xact_parsed_abort *parsed) * ParsePrepareRecord */ void -ParsePrepareRecord(uint8 info, xl_xact_prepare *xlrec, xl_xact_parsed_prepare *parsed) +ParsePrepareRecord(uint8 rmgrinfo, xl_xact_prepare *xlrec, xl_xact_parsed_prepare *parsed) { char *bufptr; @@ -328,11 +328,11 @@ xact_desc_stats(StringInfo buf, const char *label, } static void -xact_desc_commit(StringInfo buf, uint8 info, xl_xact_commit *xlrec, RepOriginId origin_id) +xact_desc_commit(StringInfo buf, uint8 rmgrinfo, xl_xact_commit *xlrec, RepOriginId origin_id) { xl_xact_parsed_commit parsed; - ParseCommitRecord(info, xlrec, &parsed); + ParseCommitRecord(rmgrinfo, xlrec, &parsed); /* If this is a prepared xact, show the xid of the original xact */ if (TransactionIdIsValid(parsed.twophase_xid)) @@ -364,11 +364,11 @@ xact_desc_commit(StringInfo buf, uint8 info, xl_xact_commit *xlrec, RepOriginId } static void -xact_desc_abort(StringInfo buf, uint8 info, xl_xact_abort *xlrec, RepOriginId origin_id) +xact_desc_abort(StringInfo buf, uint8 rmgrinfo, xl_xact_abort *xlrec, RepOriginId origin_id) { xl_xact_parsed_abort parsed; - ParseAbortRecord(info, xlrec, &parsed); + ParseAbortRecord(rmgrinfo, xlrec, &parsed); /* If this is a prepared xact, show the xid of the original xact */ if (TransactionIdIsValid(parsed.twophase_xid)) @@ -391,11 +391,11 @@ xact_desc_abort(StringInfo buf, uint8 info, xl_xact_abort *xlrec, RepOriginId or } static void -xact_desc_prepare(StringInfo buf, uint8 info, xl_xact_prepare *xlrec, RepOriginId origin_id) +xact_desc_prepare(StringInfo buf, uint8 rmgrinfo, xl_xact_prepare *xlrec, RepOriginId origin_id) { xl_xact_parsed_prepare parsed; - ParsePrepareRecord(info, xlrec, &parsed); + ParsePrepareRecord(rmgrinfo, xlrec, &parsed); appendStringInfo(buf, "gid %s: ", parsed.twophase_gid); appendStringInfoString(buf, timestamptz_to_str(parsed.xact_time)); @@ -436,27 +436,27 @@ void xact_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & XLOG_XACT_OPMASK; + uint8 info = XLogRecGetRmgrInfo(record) & XLOG_XACT_OPMASK; if (info == XLOG_XACT_COMMIT || info == XLOG_XACT_COMMIT_PREPARED) { xl_xact_commit *xlrec = (xl_xact_commit *) rec; - xact_desc_commit(buf, XLogRecGetInfo(record), xlrec, + xact_desc_commit(buf, XLogRecGetRmgrInfo(record), xlrec, XLogRecGetOrigin(record)); } else if (info == XLOG_XACT_ABORT || info == XLOG_XACT_ABORT_PREPARED) { xl_xact_abort *xlrec = (xl_xact_abort *) rec; - xact_desc_abort(buf, XLogRecGetInfo(record), xlrec, + xact_desc_abort(buf, XLogRecGetRmgrInfo(record), xlrec, XLogRecGetOrigin(record)); } else if (info == XLOG_XACT_PREPARE) { xl_xact_prepare *xlrec = (xl_xact_prepare *) rec; - xact_desc_prepare(buf, XLogRecGetInfo(record), xlrec, + xact_desc_prepare(buf, XLogRecGetRmgrInfo(record), xlrec, XLogRecGetOrigin(record)); } else if (info == XLOG_XACT_ASSIGNMENT) diff --git a/src/backend/access/rmgrdesc/xlogdesc.c b/src/backend/access/rmgrdesc/xlogdesc.c index f390c177e4..3692dfb96a 100644 --- a/src/backend/access/rmgrdesc/xlogdesc.c +++ b/src/backend/access/rmgrdesc/xlogdesc.c @@ -37,7 +37,7 @@ void xlog_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_CHECKPOINT_SHUTDOWN || info == XLOG_CHECKPOINT_ONLINE) diff --git a/src/backend/access/spgist/spgxlog.c b/src/backend/access/spgist/spgxlog.c index 459ac929ba..53190a67f9 100644 --- a/src/backend/access/spgist/spgxlog.c +++ b/src/backend/access/spgist/spgxlog.c @@ -936,7 +936,7 @@ spgRedoVacuumRedirect(XLogReaderState *record) void spg_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); MemoryContext oldCxt; oldCxt = MemoryContextSwitchTo(opCtx); diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c index 4a431d5876..11c10b936d 100644 --- a/src/backend/access/transam/clog.c +++ b/src/backend/access/transam/clog.c @@ -984,7 +984,7 @@ WriteTruncateXlogRec(int pageno, TransactionId oldestXact, Oid oldestXactDb) void clog_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in clog records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/backend/access/transam/commit_ts.c b/src/backend/access/transam/commit_ts.c index b897fabc70..a351ee84ce 100644 --- a/src/backend/access/transam/commit_ts.c +++ b/src/backend/access/transam/commit_ts.c @@ -966,7 +966,7 @@ WriteTruncateXlogRec(int pageno, TransactionId oldestXid) void commit_ts_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in commit_ts records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c index fe6698d5ff..4e1244541d 100644 --- a/src/backend/access/transam/multixact.c +++ b/src/backend/access/transam/multixact.c @@ -3232,7 +3232,7 @@ WriteMTruncateXlogRec(Oid oldestMultiDB, void multixact_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in multixact records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c index 068e59bec0..f9ab5a9c7f 100644 --- a/src/backend/access/transam/twophase.c +++ b/src/backend/access/transam/twophase.c @@ -1429,7 +1429,7 @@ XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len) } if (XLogRecGetRmid(xlogreader) != RM_XACT_ID || - (XLogRecGetInfo(xlogreader) & XLOG_XACT_OPMASK) != XLOG_XACT_PREPARE) + (XLogRecGetRmgrInfo(xlogreader) & XLOG_XACT_OPMASK) != XLOG_XACT_PREPARE) ereport(ERROR, (errcode_for_file_access(), errmsg("expected two-phase state data is not present in WAL at %X/%X", diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index 6a837e1539..33dfada659 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -6184,7 +6184,7 @@ xact_redo_abort(xl_xact_parsed_abort *parsed, TransactionId xid, void xact_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & XLOG_XACT_OPMASK; + uint8 info = XLogRecGetRmgrInfo(record) & XLOG_XACT_OPMASK; /* Backup blocks are not used in xact records */ Assert(!XLogRecHasAnyBlockRefs(record)); @@ -6194,7 +6194,7 @@ xact_redo(XLogReaderState *record) xl_xact_commit *xlrec = (xl_xact_commit *) XLogRecGetData(record); xl_xact_parsed_commit parsed; - ParseCommitRecord(XLogRecGetInfo(record), xlrec, &parsed); + ParseCommitRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); xact_redo_commit(&parsed, XLogRecGetXid(record), record->EndRecPtr, XLogRecGetOrigin(record)); } @@ -6203,7 +6203,7 @@ xact_redo(XLogReaderState *record) xl_xact_commit *xlrec = (xl_xact_commit *) XLogRecGetData(record); xl_xact_parsed_commit parsed; - ParseCommitRecord(XLogRecGetInfo(record), xlrec, &parsed); + ParseCommitRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); xact_redo_commit(&parsed, parsed.twophase_xid, record->EndRecPtr, XLogRecGetOrigin(record)); @@ -6217,7 +6217,7 @@ xact_redo(XLogReaderState *record) xl_xact_abort *xlrec = (xl_xact_abort *) XLogRecGetData(record); xl_xact_parsed_abort parsed; - ParseAbortRecord(XLogRecGetInfo(record), xlrec, &parsed); + ParseAbortRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); xact_redo_abort(&parsed, XLogRecGetXid(record), record->EndRecPtr, XLogRecGetOrigin(record)); } @@ -6226,7 +6226,7 @@ xact_redo(XLogReaderState *record) xl_xact_abort *xlrec = (xl_xact_abort *) XLogRecGetData(record); xl_xact_parsed_abort parsed; - ParseAbortRecord(XLogRecGetInfo(record), xlrec, &parsed); + ParseAbortRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); xact_redo_abort(&parsed, parsed.twophase_xid, record->EndRecPtr, XLogRecGetOrigin(record)); diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index bc5a8e0569..d6c00b896b 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -7715,7 +7715,7 @@ UpdateFullPageWrites(void) void xlog_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); XLogRecPtr lsn = record->EndRecPtr; /* diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index 188f6d6f85..6ca898c6e9 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -1995,7 +1995,7 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl static void xlogrecovery_redo(XLogReaderState *record, TimeLineID replayTLI) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); XLogRecPtr lsn = record->EndRecPtr; Assert(XLogRecGetRmid(record) == RM_XLOG_ID); @@ -2213,7 +2213,7 @@ void xlog_outdesc(StringInfo buf, XLogReaderState *record) { RmgrData rmgr = GetRmgr(XLogRecGetRmid(record)); - uint8 info = XLogRecGetInfo(record); + uint8 info = XLogRecGetRmgrInfo(record); const char *id; appendStringInfoString(buf, rmgr.rm_name); @@ -2221,7 +2221,7 @@ xlog_outdesc(StringInfo buf, XLogReaderState *record) id = rmgr.rm_identify(info); if (id == NULL) - appendStringInfo(buf, "UNKNOWN (%X): ", info & ~XLR_INFO_MASK); + appendStringInfo(buf, "UNKNOWN (%X): ", info); else appendStringInfo(buf, "%s: ", id); @@ -2341,7 +2341,7 @@ checkTimeLineSwitch(XLogRecPtr lsn, TimeLineID newTLI, TimeLineID prevTLI, static bool getRecordTimestamp(XLogReaderState *record, TimestampTz *recordXtime) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); uint8 xact_info = info & XLOG_XACT_OPMASK; uint8 rmid = XLogRecGetRmid(record); @@ -2535,7 +2535,7 @@ recoveryStopsBefore(XLogReaderState *record) if (XLogRecGetRmid(record) != RM_XACT_ID) return false; - xact_info = XLogRecGetInfo(record) & XLOG_XACT_OPMASK; + xact_info = XLogRecGetRmgrInfo(record); if (xact_info == XLOG_XACT_COMMIT) { @@ -2548,7 +2548,7 @@ recoveryStopsBefore(XLogReaderState *record) xl_xact_parsed_commit parsed; isCommit = true; - ParseCommitRecord(XLogRecGetInfo(record), + ParseCommitRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); recordXid = parsed.twophase_xid; @@ -2564,7 +2564,7 @@ recoveryStopsBefore(XLogReaderState *record) xl_xact_parsed_abort parsed; isCommit = false; - ParseAbortRecord(XLogRecGetInfo(record), + ParseAbortRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); recordXid = parsed.twophase_xid; @@ -2653,7 +2653,7 @@ recoveryStopsAfter(XLogReaderState *record) if (!ArchiveRecoveryRequested) return false; - info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + info = XLogRecGetRmgrInfo(record); rmid = XLogRecGetRmid(record); /* @@ -2721,7 +2721,7 @@ recoveryStopsAfter(XLogReaderState *record) xl_xact_commit *xlrec = (xl_xact_commit *) XLogRecGetData(record); xl_xact_parsed_commit parsed; - ParseCommitRecord(XLogRecGetInfo(record), + ParseCommitRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); recordXid = parsed.twophase_xid; @@ -2731,7 +2731,7 @@ recoveryStopsAfter(XLogReaderState *record) xl_xact_abort *xlrec = (xl_xact_abort *) XLogRecGetData(record); xl_xact_parsed_abort parsed; - ParseAbortRecord(XLogRecGetInfo(record), + ParseAbortRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); recordXid = parsed.twophase_xid; @@ -2925,7 +2925,7 @@ recoveryApplyDelay(XLogReaderState *record) if (XLogRecGetRmid(record) != RM_XACT_ID) return false; - xact_info = XLogRecGetInfo(record) & XLOG_XACT_OPMASK; + xact_info = XLogRecGetRmgrInfo(record) & XLOG_XACT_OPMASK; if (xact_info != XLOG_XACT_COMMIT && xact_info != XLOG_XACT_COMMIT_PREPARED) diff --git a/src/backend/access/transam/xlogstats.c b/src/backend/access/transam/xlogstats.c index 2d315df67a..267b108943 100644 --- a/src/backend/access/transam/xlogstats.c +++ b/src/backend/access/transam/xlogstats.c @@ -79,7 +79,7 @@ XLogRecStoreStats(XLogStats *stats, XLogReaderState *record) * RmgrId). */ - recid = XLogRecGetInfo(record) >> 4; + recid = XLogRecGetRmgrInfo(record) >> 4; /* * XACT records need to be handled differently. Those records use the diff --git a/src/backend/catalog/storage.c b/src/backend/catalog/storage.c index 2add053489..4d73023a65 100644 --- a/src/backend/catalog/storage.c +++ b/src/backend/catalog/storage.c @@ -958,7 +958,7 @@ void smgr_redo(XLogReaderState *record) { XLogRecPtr lsn = record->EndRecPtr; - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in smgr records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c index 2e242eeff2..528330a0b3 100644 --- a/src/backend/commands/dbcommands.c +++ b/src/backend/commands/dbcommands.c @@ -3102,7 +3102,7 @@ recovery_create_dbdir(char *path, bool only_tblspc) void dbase_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in dbase records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c index ef01449678..eae2cb09b1 100644 --- a/src/backend/commands/sequence.c +++ b/src/backend/commands/sequence.c @@ -1829,7 +1829,7 @@ void seq_redo(XLogReaderState *record) { XLogRecPtr lsn = record->EndRecPtr; - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); Buffer buffer; Page page; Page localpage; diff --git a/src/backend/commands/tablespace.c b/src/backend/commands/tablespace.c index 3dfbf6a917..f251a84628 100644 --- a/src/backend/commands/tablespace.c +++ b/src/backend/commands/tablespace.c @@ -1516,7 +1516,7 @@ get_tablespace_name(Oid spc_oid) void tblspc_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in tblspc records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c index beef399b42..59ed82276a 100644 --- a/src/backend/replication/logical/decode.c +++ b/src/backend/replication/logical/decode.c @@ -132,7 +132,7 @@ void xlog_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) { SnapBuild *builder = ctx->snapshot_builder; - uint8 info = XLogRecGetInfo(buf->record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(buf->record); ReorderBufferProcessXid(ctx->reorder, XLogRecGetXid(buf->record), buf->origptr); @@ -205,7 +205,7 @@ xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) SnapBuild *builder = ctx->snapshot_builder; ReorderBuffer *reorder = ctx->reorder; XLogReaderState *r = buf->record; - uint8 info = XLogRecGetInfo(r) & XLOG_XACT_OPMASK; + uint8 info = XLogRecGetRmgrInfo(r) & XLOG_XACT_OPMASK; /* * If the snapshot isn't yet fully built, we cannot decode anything, so @@ -225,7 +225,9 @@ xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) bool two_phase = false; xlrec = (xl_xact_commit *) XLogRecGetData(r); - ParseCommitRecord(XLogRecGetInfo(buf->record), xlrec, &parsed); + ParseCommitRecord(XLogRecGetRmgrInfo(buf->record), + xlrec, + &parsed); if (!TransactionIdIsValid(parsed.twophase_xid)) xid = XLogRecGetXid(r); @@ -253,7 +255,7 @@ xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) bool two_phase = false; xlrec = (xl_xact_abort *) XLogRecGetData(r); - ParseAbortRecord(XLogRecGetInfo(buf->record), xlrec, &parsed); + ParseAbortRecord(XLogRecGetRmgrInfo(buf->record), xlrec, &parsed); if (!TransactionIdIsValid(parsed.twophase_xid)) xid = XLogRecGetXid(r); @@ -316,7 +318,7 @@ xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) /* ok, parse it */ xlrec = (xl_xact_prepare *) XLogRecGetData(r); - ParsePrepareRecord(XLogRecGetInfo(buf->record), + ParsePrepareRecord(XLogRecGetRmgrInfo(buf->record), xlrec, &parsed); /* @@ -361,7 +363,7 @@ standby_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) { SnapBuild *builder = ctx->snapshot_builder; XLogReaderState *r = buf->record; - uint8 info = XLogRecGetInfo(r) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(r); ReorderBufferProcessXid(ctx->reorder, XLogRecGetXid(r), buf->origptr); @@ -405,7 +407,7 @@ standby_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) void heap2_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) { - uint8 info = XLogRecGetInfo(buf->record) & XLOG_HEAP_OPMASK; + uint8 info = XLogRecGetRmgrInfo(buf->record) & XLOG_HEAP_OPMASK; TransactionId xid = XLogRecGetXid(buf->record); SnapBuild *builder = ctx->snapshot_builder; @@ -465,7 +467,7 @@ heap2_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) void heap_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) { - uint8 info = XLogRecGetInfo(buf->record) & XLOG_HEAP_OPMASK; + uint8 info = XLogRecGetRmgrInfo(buf->record) & XLOG_HEAP_OPMASK; TransactionId xid = XLogRecGetXid(buf->record); SnapBuild *builder = ctx->snapshot_builder; @@ -590,7 +592,7 @@ logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) SnapBuild *builder = ctx->snapshot_builder; XLogReaderState *r = buf->record; TransactionId xid = XLogRecGetXid(r); - uint8 info = XLogRecGetInfo(r) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(r); RepOriginId origin_id = XLogRecGetOrigin(r); Snapshot snapshot = NULL; xl_logical_message *message; diff --git a/src/backend/replication/logical/message.c b/src/backend/replication/logical/message.c index c5de14afc6..c31dc1203f 100644 --- a/src/backend/replication/logical/message.c +++ b/src/backend/replication/logical/message.c @@ -80,7 +80,7 @@ LogLogicalMessage(const char *prefix, const char *message, size_t size, void logicalmsg_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info != XLOG_LOGICAL_MESSAGE) elog(PANIC, "logicalmsg_redo: unknown op code %u", info); diff --git a/src/backend/replication/logical/origin.c b/src/backend/replication/logical/origin.c index 2c04c8707d..00a6a9169c 100644 --- a/src/backend/replication/logical/origin.c +++ b/src/backend/replication/logical/origin.c @@ -826,7 +826,7 @@ StartupReplicationOrigin(void) void replorigin_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c index ffe5e1563f..e2afc0dcc7 100644 --- a/src/backend/storage/ipc/standby.c +++ b/src/backend/storage/ipc/standby.c @@ -1160,7 +1160,7 @@ StandbyReleaseOldLocks(TransactionId oldxid) void standby_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in standby records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/backend/utils/cache/relmapper.c b/src/backend/utils/cache/relmapper.c index 4c21129707..c67fe048f3 100644 --- a/src/backend/utils/cache/relmapper.c +++ b/src/backend/utils/cache/relmapper.c @@ -1084,7 +1084,7 @@ perform_relmap_update(bool shared, const RelMapFile *updates) void relmap_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in relmap records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/bin/pg_rewind/parsexlog.c b/src/bin/pg_rewind/parsexlog.c index 27782237d0..2b6b46f0e3 100644 --- a/src/bin/pg_rewind/parsexlog.c +++ b/src/bin/pg_rewind/parsexlog.c @@ -222,7 +222,7 @@ findLastCheckpoint(const char *datadir, XLogRecPtr forkptr, int tliIndex, * be the latest checkpoint before WAL forked and not the checkpoint * where the primary has been stopped to be rewound. */ - info = XLogRecGetInfo(xlogreader) & ~XLR_INFO_MASK; + info = XLogRecGetRmgrInfo(xlogreader); if (searchptr < forkptr && XLogRecGetRmid(xlogreader) == RM_XLOG_ID && (info == XLOG_CHECKPOINT_SHUTDOWN || @@ -370,7 +370,7 @@ extractPageInfo(XLogReaderState *record) int block_id; RmgrId rmid = XLogRecGetRmid(record); uint8 info = XLogRecGetInfo(record); - uint8 rminfo = info & ~XLR_INFO_MASK; + uint8 rminfo = XLogRecGetRmgrInfo(record); /* Is this a special record type that I recognize? */ @@ -440,7 +440,7 @@ extractPageInfo(XLogReaderState *record) pg_fatal("WAL record modifies a relation, but record type is not recognized: " "lsn: %X/%X, rmid: %d, rmgr: %s, info: %02X", LSN_FORMAT_ARGS(record->ReadRecPtr), - rmid, RmgrName(rmid), info); + rmid, RmgrName(rmid), info | rminfo); } for (block_id = 0; block_id <= XLogRecMaxBlockId(record); block_id++) diff --git a/src/bin/pg_waldump/pg_waldump.c b/src/bin/pg_waldump/pg_waldump.c index c6d3ae6344..e3390f3d39 100644 --- a/src/bin/pg_waldump/pg_waldump.c +++ b/src/bin/pg_waldump/pg_waldump.c @@ -544,7 +544,7 @@ XLogDumpDisplayRecord(XLogDumpConfig *config, XLogReaderState *record) const RmgrDescData *desc = GetRmgrDesc(XLogRecGetRmid(record)); uint32 rec_len; uint32 fpi_len; - uint8 info = XLogRecGetInfo(record); + uint8 info = XLogRecGetRmgrInfo(record); XLogRecPtr xl_prev = XLogRecGetPrev(record); StringInfoData s; @@ -559,7 +559,7 @@ XLogDumpDisplayRecord(XLogDumpConfig *config, XLogReaderState *record) id = desc->rm_identify(info); if (id == NULL) - printf("desc: UNKNOWN (%x) ", info & ~XLR_INFO_MASK); + printf("desc: UNKNOWN (%x) ", info); else printf("desc: %s ", id); diff --git a/src/include/access/xact.h b/src/include/access/xact.h index 7d3b9446e6..b94b264402 100644 --- a/src/include/access/xact.h +++ b/src/include/access/xact.h @@ -519,9 +519,9 @@ extern void xact_desc(StringInfo buf, XLogReaderState *record); extern const char *xact_identify(uint8 info); /* also in xactdesc.c, so they can be shared between front/backend code */ -extern void ParseCommitRecord(uint8 info, xl_xact_commit *xlrec, xl_xact_parsed_commit *parsed); -extern void ParseAbortRecord(uint8 info, xl_xact_abort *xlrec, xl_xact_parsed_abort *parsed); -extern void ParsePrepareRecord(uint8 info, xl_xact_prepare *xlrec, xl_xact_parsed_prepare *parsed); +extern void ParseCommitRecord(uint8 rmgrinfo, xl_xact_commit *xlrec, xl_xact_parsed_commit *parsed); +extern void ParseAbortRecord(uint8 rmgrinfo, xl_xact_abort *xlrec, xl_xact_parsed_abort *parsed); +extern void ParsePrepareRecord(uint8 rmgrinfo, xl_xact_prepare *xlrec, xl_xact_parsed_prepare *parsed); extern void EnterParallelMode(void); extern void ExitParallelMode(void); diff --git a/src/include/access/xlogreader.h b/src/include/access/xlogreader.h index 30d20c323e..d6f49b1a38 100644 --- a/src/include/access/xlogreader.h +++ b/src/include/access/xlogreader.h @@ -406,7 +406,8 @@ extern bool DecodeXLogRecord(XLogReaderState *state, */ #define XLogRecGetTotalLen(decoder) ((decoder)->record->header.xl_tot_len) #define XLogRecGetPrev(decoder) ((decoder)->record->header.xl_prev) -#define XLogRecGetInfo(decoder) ((decoder)->record->header.xl_info) +#define XLogRecGetInfo(decoder) ((decoder)->record->header.xl_info & XLR_INFO_MASK) +#define XLogRecGetRmgrInfo(decoder) (((decoder)->record->header.xl_info) & XLR_RMGR_INFO_MASK) #define XLogRecGetRmid(decoder) ((decoder)->record->header.xl_rmid) #define XLogRecGetXid(decoder) ((decoder)->record->header.xl_xid) #define XLogRecGetOrigin(decoder) ((decoder)->record->record_origin) diff --git a/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c b/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c index 1727910ce7..6921308937 100644 --- a/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c +++ b/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c @@ -81,7 +81,7 @@ _PG_init(void) void testcustomrmgrs_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info != XLOG_TEST_CUSTOM_RMGRS_MESSAGE) elog(PANIC, "testcustomrmgrs_redo: unknown op code %u", info); @@ -91,7 +91,7 @@ void testcustomrmgrs_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_TEST_CUSTOM_RMGRS_MESSAGE) { -- 2.39.0 [application/octet-stream] v1-0006-Reformat-the-XLog-record-header.patch (41.7K, ../../CAEze2WgcW5SkPO_TGTqirRcCS_FN0xEZqUsXDE6pPz2TYzXQBA@mail.gmail.com/7-v1-0006-Reformat-the-XLog-record-header.patch) download | inline diff: From c018d77bb5ab9c5b2fbec3401bbd0bb848f2f7db Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Mon, 19 Jun 2023 17:46:45 +0200 Subject: [PATCH v1 6/6] Reformat the XLog record header Many WAL records don't utilize all bytes of the XLog header, resulting in a lot of wasted bytes. This new XLog header format saves bytes in several ways: - Records that don't need the XID won't include it This potentially saves 4 bytes /record - Records that are small enough don't need 4 bytes to describe their length By varint encoding the length, we save up to 3 bytes /record (up to 4 bytes /record if dataless records are considered) - Only include xl_rmgrinfo if it is non-0 Several RMGRs have a record that they use most often. We save 1 byte/record if that record id is 0. - Lose some alignment losses We save 1 byte/record by removing one byte of alignment losses --- src/backend/access/transam/xlog.c | 127 ++++++++++++++------ src/backend/access/transam/xloginsert.c | 135 ++++++++++++++++------ src/backend/access/transam/xlogreader.c | 93 ++++++++------- src/backend/access/transam/xlogrecovery.c | 4 +- src/bin/pg_resetwal/pg_resetwal.c | 38 +++--- src/include/access/rmgr.h | 2 +- src/include/access/rmgrlist.h | 1 + src/include/access/xlog_internal.h | 102 +++++++++++++++- src/include/access/xlogreader.h | 4 +- src/include/access/xlogrecord.h | 125 +++++++++++++++++--- 10 files changed, 480 insertions(+), 151 deletions(-) diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 70c193c08f..c1a8b925d8 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -737,17 +737,18 @@ XLogInsertRecord(XLogRecData *rdata, XLogCtlInsert *Insert = &XLogCtl->Insert; pg_crc32c rdata_crc; bool inserted; - XLogRecord *rechdr = (XLogRecord *) rdata->data; - uint8 rmgrinfo = rechdr->xl_rmgrinfo; - bool isLogSwitch = (rechdr->xl_rmid == RM_XLOG_ID && - rmgrinfo == XLOG_SWITCH); + XLogRecHdr rechdr; + bool isLogSwitch; XLogRecPtr StartPos; XLogRecPtr EndPos; + XLogRecPtr xl_prev; bool prevDoPageWrites = doPageWrites; TimeLineID insertTLI; + int rec_payload_len, + rec_hdr_len; - /* we assume that all of the record header is in the first chunk */ - Assert(rdata->len >= SizeOfXLogRecord); + /* we assume that all data of the record header is in the first chunk */ + Assert(rdata->len >= XLogRecordMinHdrSize); /* cross-check on whether we should be here or not */ if (!XLogInsertAllowed()) @@ -758,6 +759,17 @@ XLogInsertRecord(XLogRecData *rdata, * change, so we can read it without a lock. */ insertTLI = XLogCtl->InsertTimeLineID; + rechdr = (XLogRecHdr) rdata->data; + + { + XLogRecord record; + XLogReadRecHdrInto(rechdr, (Size) rdata->len, &record); + + isLogSwitch = (record.xl_rmid == RM_XLOG_ID && + record.xl_rmgrinfo == XLOG_SWITCH); + rec_hdr_len = (int) XLogRecordHdrLen(rechdr->xl_info); + rec_payload_len = (int) record.xl_payload_len; + } /*---------- * @@ -834,34 +846,44 @@ XLogInsertRecord(XLogRecData *rdata, } /* - * Reserve space for the record in the WAL. This also sets the xl_prev - * pointer. + * Reserve space for the record in the WAL. */ if (isLogSwitch) - inserted = ReserveXLogSwitch(&StartPos, &EndPos, &rechdr->xl_prev); + inserted = ReserveXLogSwitch(&StartPos, &EndPos, &xl_prev); else { - ReserveXLogInsertLocation(rechdr->xl_tot_len, &StartPos, &EndPos, - &rechdr->xl_prev); + ReserveXLogInsertLocation(rec_hdr_len + rec_payload_len, + &StartPos, &EndPos, &xl_prev); inserted = true; } if (inserted) { + char *rec = (char *) rechdr; + + /* fill in xl_prev */ + memcpy(rec + rec_hdr_len - sizeof(pg_crc32c) - sizeof(XLogRecPtr), + &xl_prev, sizeof(XLogRecPtr)); + /* * Now that xl_prev has been filled in, calculate CRC of the record * header. */ - rdata_crc = rechdr->xl_crc; - COMP_CRC32C(rdata_crc, rechdr, offsetof(XLogRecord, xl_crc)); + + memcpy(&rdata_crc, rec + rec_hdr_len - sizeof(pg_crc32c), + sizeof(pg_crc32c)); + + COMP_CRC32C(rdata_crc, rec, rec_hdr_len - sizeof(pg_crc32c)); FIN_CRC32C(rdata_crc); - rechdr->xl_crc = rdata_crc; + + memcpy(rec + rec_hdr_len - sizeof(pg_crc32c), + &rdata_crc, sizeof(pg_crc32c)); /* * All the record data, including the header, is now ready to be * inserted. Copy the record in the space reserved. */ - CopyXLogRecordToWAL(rechdr->xl_tot_len, isLogSwitch, rdata, + CopyXLogRecordToWAL(rec_hdr_len + rec_payload_len, isLogSwitch, rdata, StartPos, EndPos, insertTLI); /* @@ -932,7 +954,8 @@ XLogInsertRecord(XLogRecData *rdata, */ if (inserted) { - EndPos = StartPos + SizeOfXLogRecord; + /* xlog switch is minimal record header, plus a byte for rmgrinfo */ + EndPos = StartPos + XLogRecordMinHdrSize + sizeof(uint8); if (StartPos / XLOG_BLCKSZ != EndPos / XLOG_BLCKSZ) { uint64 offset = XLogSegmentOffset(EndPos, wal_segment_size); @@ -949,7 +972,7 @@ XLogInsertRecord(XLogRecData *rdata, if (XLOG_DEBUG) { static XLogReaderState *debug_reader = NULL; - XLogRecord *record; + XLogRecHdr record; DecodedXLogRecord *decoded; StringInfoData buf; StringInfoData recordBuf; @@ -971,9 +994,9 @@ XLogInsertRecord(XLogRecData *rdata, appendBinaryStringInfo(&recordBuf, rdata->data, rdata->len); /* We also need temporary space to decode the record. */ - record = (XLogRecord *) recordBuf.data; + record = (XLogRecHdr) recordBuf.data; decoded = (DecodedXLogRecord *) - palloc(DecodeXLogRecordRequiredSpace(record->xl_tot_len)); + palloc(DecodeXLogRecordRequiredSpace(recordBuf.len)); if (!debug_reader) debug_reader = XLogReaderAllocate(wal_segment_size, NULL, @@ -1018,7 +1041,7 @@ XLogInsertRecord(XLogRecData *rdata, /* Report WAL traffic to the instrumentation. */ if (inserted) { - pgWalUsage.wal_bytes += rechdr->xl_tot_len; + pgWalUsage.wal_bytes += rec_hdr_len + rec_payload_len; pgWalUsage.wal_records++; pgWalUsage.wal_fpi += num_fpi; } @@ -1049,10 +1072,10 @@ ReserveXLogInsertLocation(int size, XLogRecPtr *StartPos, XLogRecPtr *EndPos, uint64 endbytepos; uint64 prevbytepos; - size = MAXALIGN(size); + size = XLP_ALIGN(size); /* All (non xlog-switch) records should contain data. */ - Assert(size > SizeOfXLogRecord); + Assert(size > XLogRecordMinHdrSize); /* * The duration the spinlock needs to be held is minimized by minimizing @@ -1103,7 +1126,7 @@ ReserveXLogSwitch(XLogRecPtr *StartPos, XLogRecPtr *EndPos, XLogRecPtr *PrevPtr) uint64 startbytepos; uint64 endbytepos; uint64 prevbytepos; - uint32 size = MAXALIGN(SizeOfXLogRecord); + uint32 size = XLP_ALIGN(XLogRecordMinHdrSize + sizeof(uint8)); XLogRecPtr ptr; uint32 segleft; @@ -1247,7 +1270,7 @@ CopyXLogRecordToWAL(int write_len, bool isLogSwitch, XLogRecData *rdata, if (isLogSwitch && XLogSegmentOffset(CurrPos, wal_segment_size) != 0) { /* An xlog-switch record doesn't contain any data besides the header */ - Assert(write_len == SizeOfXLogRecord); + Assert(write_len == XLogRecordMinHdrSize + sizeof(uint8)); /* Assert that we did reserve the right amount of space */ Assert(XLogSegmentOffset(EndPos, wal_segment_size) == 0); @@ -4649,11 +4672,15 @@ BootStrapXLOG(void) char *buffer; XLogPageHeader page; XLogLongPageHeader longpage; - XLogRecord *record; + XLogRecHdr rec_hdr; + char *baserecptr; char *recptr; uint64 sysidentifier; struct timeval tv; pg_crc32c crc; + const int rec_hdr_len = offsetof(XLogRecHdrData, xl_hdrdata) + + sizeof(uint8) + sizeof(XLogRecPtr) + sizeof(pg_crc32c); + const int rec_payload_len = sizeof(uint8) * 2 + sizeof(CheckPoint); /* allow ordinary WAL segment creation, like StartupXLOG() would */ SetInstallXLogFileSegmentActive(); @@ -4724,28 +4751,52 @@ BootStrapXLOG(void) longpage->xlp_seg_size = wal_segment_size; longpage->xlp_xlog_blcksz = XLOG_BLCKSZ; + /* if this changes, we need to update the code below */ + Assert(XLogLengthToSizeClass(SizeOfXLogRecordDataHeaderShort + sizeof(checkPoint), XLS_UINT32) == XLS_UINT8); + /* if this changes, we need to add XLR_HAS_RMGRINFO */ + Assert(XLOG_CHECKPOINT_SHUTDOWN == 0); + /* Insert the initial checkpoint record */ - recptr = ((char *) page + SizeOfXLogLongPHD); - record = (XLogRecord *) recptr; - record->xl_prev = 0; - record->xl_xid = InvalidTransactionId; - record->xl_tot_len = SizeOfXLogRecord + SizeOfXLogRecordDataHeaderShort + sizeof(checkPoint); - record->xl_info = 0; - record->xl_rmgrinfo = XLOG_CHECKPOINT_SHUTDOWN; - record->xl_rmid = RM_XLOG_ID; - recptr += SizeOfXLogRecord; + baserecptr = recptr = (((char *) page) + SizeOfXLogLongPHD); + + rec_hdr = (XLogRecHdr) baserecptr; + rec_hdr->xl_info = (char) XLS_UINT8; + rec_hdr->xl_rmid = (char) RM_XLOG_ID; + recptr += offsetof(XLogRecHdrData, xl_hdrdata); + + recptr += XLogWriteLength(SizeOfXLogRecordDataHeaderShort + sizeof(checkPoint), + XLS_UINT8, XLS_UINT8, recptr); + + /* include prevptr */ + { + XLogRecPtr prevptr = 0; + memcpy(recptr, &prevptr, sizeof(XLogRecPtr)); + recptr += sizeof(XLogRecPtr); + } + + /* reserve location of crc */ + recptr += sizeof(pg_crc32c); + + Assert(recptr - baserecptr == XLogRecordHdrLen(rec_hdr->xl_info)); + Assert(recptr - baserecptr == rec_hdr_len); + /* fill the XLogRecordDataHeaderShort struct */ *(recptr++) = (char) XLR_BLOCK_ID_DATA_SHORT; *(recptr++) = sizeof(checkPoint); memcpy(recptr, &checkPoint, sizeof(checkPoint)); recptr += sizeof(checkPoint); - Assert(recptr - (char *) record == record->xl_tot_len); + + /* Assert length of record matches expectations */ + Assert(recptr - baserecptr == XLogRecordTotalLength(rec_hdr)); + Assert((rec_hdr)->xl_info == XLS_UINT8); INIT_CRC32C(crc); - COMP_CRC32C(crc, ((char *) record) + SizeOfXLogRecord, record->xl_tot_len - SizeOfXLogRecord); - COMP_CRC32C(crc, (char *) record, offsetof(XLogRecord, xl_crc)); + COMP_CRC32C(crc, baserecptr + rec_hdr_len, rec_payload_len); + COMP_CRC32C(crc, baserecptr, rec_hdr_len - sizeof(pg_crc32c)); FIN_CRC32C(crc); - record->xl_crc = crc; + + memcpy(baserecptr + rec_hdr_len - sizeof(pg_crc32c), + &crc, sizeof(pg_crc32c)); /* Create first XLOG segment file */ openLogTLI = BootstrapTimeLineID; diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c index b105d90e3f..434f29b089 100644 --- a/src/backend/access/transam/xloginsert.c +++ b/src/backend/access/transam/xloginsert.c @@ -545,12 +545,16 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP XLogRecPtr *fpw_lsn, int *num_fpi, bool *topxid_included) { XLogRecData *rdt; - uint64 total_len = 0; + uint64 payload_len = 0; + XLogSizeClass payload_sizeclass = XLS_EMPTY; int block_id; pg_crc32c rdata_crc; registered_buffer *prev_regbuf = NULL; XLogRecData *rdt_datas_last; - XLogRecord *rechdr; + TransactionId xid; + XLogRecHdr rechdr; + uint8 xlr_flags = 0; + uint32 rec_hdr_len; char *scratch = hdr_scratch; /* @@ -558,9 +562,10 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP * All the modifications we do to the rdata chains below must handle that. */ - /* The record begins with the fixed-size header */ - rechdr = (XLogRecord *) scratch; - scratch += SizeOfXLogRecord; + /* The record begins with the variable-size header */ + rechdr = (XLogRecHdr) scratch; + + scratch += XLogRecordMaxHdrSize; hdr_rdt.next = NULL; rdt_datas_last = &hdr_rdt; @@ -779,7 +784,7 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP } } - total_len += bimg.length; + payload_len += bimg.length; } if (needs_data) @@ -797,7 +802,7 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP bkpb.fork_flags |= BKPBLOCK_HAS_DATA; data_length = (uint16) regbuf->rdata_len; data_sizeclass = XLogLengthToSizeClass(data_length, XLS_UINT16); - total_len += regbuf->rdata_len; + payload_len += regbuf->rdata_len; rdt_datas_last->next = regbuf->rdata_head; rdt_datas_last = regbuf->rdata_tail; @@ -851,16 +856,33 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP } /* followed by toplevel XID, if not already included in previous record */ - if (curinsert_flags & XLOG_INCLUDE_XID && IsSubxactTopXidLogPending()) + if (curinsert_flags & XLOG_INCLUDE_XID) { - TransactionId xid = GetTopTransactionIdIfAny(); + xid = GetCurrentTransactionIdIfAny(); + + if (IsSubxactTopXidLogPending()) + { + TransactionId txid = GetTopTransactionIdIfAny(); + + xlr_flags |= XLR_HAS_XID; + Assert(TransactionIdIsValid(xid)); - /* Set the flag that the top xid is included in the WAL */ - *topxid_included = true; + /* Set the flag that the top xid is included in the WAL */ + *topxid_included = true; - *(scratch++) = (char) XLR_BLOCK_ID_TOPLEVEL_XID; - memcpy(scratch, &xid, sizeof(TransactionId)); - scratch += sizeof(TransactionId); + *(scratch++) = (char) XLR_BLOCK_ID_TOPLEVEL_XID; + memcpy(scratch, &txid, sizeof(TransactionId)); + scratch += sizeof(TransactionId); + } + else if (TransactionIdIsValid(xid)) + { + xlr_flags |= XLR_HAS_XID; + } + } + else + { + xid = InvalidTransactionId; + Assert((xlr_flags & XLR_HAS_XID) == 0); } /* followed by main data, if any */ @@ -889,12 +911,64 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP } rdt_datas_last->next = mainrdata_head; rdt_datas_last = mainrdata_last; - total_len += mainrdata_len; + payload_len += mainrdata_len; } rdt_datas_last->next = NULL; - hdr_rdt.len = (scratch - hdr_scratch); - total_len += hdr_rdt.len; + /* Add the block headers section's length to the payload */ + payload_len += scratch - (hdr_scratch + XLogRecordMaxHdrSize); + + /* + * Fill in the fields in the record header. Prev-link is filled in later, + * once we know where in the WAL the record will be inserted. The CRC does + * not include the record header yet. + */ + payload_sizeclass = XLogLengthToSizeClass(payload_len, XLS_UINT32); + + xlr_flags |= payload_sizeclass; + + if (rmgr_info != 0) + xlr_flags |= XLR_HAS_RMGRINFO; + + /* Set up the xlog header. and xl_rmgr */ + rechdr->xl_info = xlr_flags; + rechdr->xl_rmid = rmid; + + rec_hdr_len = 0; + + /* next, xl_payload_len */ + rec_hdr_len += XLogWriteLength(payload_len, payload_sizeclass, + XLS_UINT32, + &rechdr->xl_hdrdata[rec_hdr_len]); + + if (xlr_flags & XLR_HAS_RMGRINFO) + rechdr->xl_hdrdata[rec_hdr_len++] = (char) rmgr_info; + + if (xlr_flags & XLR_HAS_XID) + { + Assert(curinsert_flags & XLOG_INCLUDE_XID); + Assert(TransactionIdIsValid(xid)); + + memcpy(&rechdr->xl_hdrdata[rec_hdr_len], &xid, sizeof(TransactionId)); + rec_hdr_len += sizeof(TransactionId); + } + + /* reserve space for XLogRecPtr and checksum */ + rec_hdr_len += sizeof(XLogRecPtr); + rec_hdr_len += sizeof(pg_crc32c); + /* Add static header length */ + rec_hdr_len += offsetof(XLogRecHdrData, xl_hdrdata); + + Assert(rec_hdr_len == XLogRecordHdrLen(rechdr->xl_info)); + + /* move the record to be placed the rest of the payload */ + memmove(hdr_scratch + XLogRecordMaxHdrSize - rec_hdr_len, + hdr_scratch, rec_hdr_len); + + rechdr = (XLogRecHdr) (hdr_scratch + XLogRecordMaxHdrSize - rec_hdr_len); + + hdr_rdt.data = (char *) rechdr; + hdr_rdt.len = (scratch - hdr_rdt.data); /* * Calculate CRC of the data @@ -905,10 +979,14 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP * header. */ INIT_CRC32C(rdata_crc); - COMP_CRC32C(rdata_crc, hdr_scratch + SizeOfXLogRecord, hdr_rdt.len - SizeOfXLogRecord); + COMP_CRC32C(rdata_crc, hdr_scratch + XLogRecordMaxHdrSize, + hdr_rdt.len - rec_hdr_len); for (rdt = hdr_rdt.next; rdt != NULL; rdt = rdt->next) COMP_CRC32C(rdata_crc, rdt->data, rdt->len); + memcpy(hdr_rdt.data + rec_hdr_len - sizeof(pg_crc32c), + &rdata_crc, sizeof(pg_crc32c)); + /* * Ensure that the XLogRecord is not too large. * @@ -917,28 +995,11 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP * not emit records larger than the sizes advertised to be supported. * This cap is based on DecodeXLogRecordRequiredSpace(). */ - if (total_len >= XLogRecordMaxSize) + if (payload_len + rec_hdr_len >= XLogRecordMaxSize) ereport(ERROR, (errmsg_internal("oversized WAL record"), errdetail_internal("WAL record would be %llu bytes (of maximum %u bytes); rmid %u flags %u.", - (unsigned long long) total_len, XLogRecordMaxSize, rmid, info))); - - /* - * Fill in the fields in the record header. Prev-link is filled in later, - * once we know where in the WAL the record will be inserted. The CRC does - * not include the record header yet. - */ - if (curinsert_flags & XLOG_INCLUDE_XID) - rechdr->xl_xid = GetCurrentTransactionIdIfAny(); - else - rechdr->xl_xid = InvalidTransactionId; - - rechdr->xl_tot_len = (uint32) total_len; - rechdr->xl_info = info; - rechdr->xl_rmid = rmid; - rechdr->xl_rmgrinfo = rmgr_info; - rechdr->xl_prev = InvalidXLogRecPtr; - rechdr->xl_crc = rdata_crc; + (unsigned long long) payload_len, XLogRecordMaxSize, rmid, info))); return &hdr_rdt; } diff --git a/src/backend/access/transam/xlogreader.c b/src/backend/access/transam/xlogreader.c index e15b0f9580..0ce443b291 100644 --- a/src/backend/access/transam/xlogreader.c +++ b/src/backend/access/transam/xlogreader.c @@ -49,8 +49,9 @@ static int ReadPageInternal(XLogReaderState *state, XLogRecPtr pageptr, static void XLogReaderInvalReadState(XLogReaderState *state); static XLogPageReadResult XLogDecodeNextRecord(XLogReaderState *state, bool nonblocking); static bool ValidXLogRecordHeader(XLogReaderState *state, XLogRecPtr RecPtr, - XLogRecPtr PrevRecPtr, XLogRecord *record, bool randAccess); -static bool ValidXLogRecord(XLogReaderState *state, XLogRecord *record, + XLogRecPtr PrevRecPtr, XLogRecHdr rechdr, + bool randAccess); +static bool ValidXLogRecord(XLogReaderState *state, XLogRecHdr record, XLogRecPtr recptr); static void ResetDecoder(XLogReaderState *state); static void WALOpenSegmentInit(WALOpenSegment *seg, WALSegmentContext *segcxt, @@ -535,7 +536,7 @@ static XLogPageReadResult XLogDecodeNextRecord(XLogReaderState *state, bool nonblocking) { XLogRecPtr RecPtr; - XLogRecord *record; + XLogRecHdr record; XLogRecPtr targetPagePtr; bool randAccess; uint32 len, @@ -602,7 +603,7 @@ restart: * fits on the same page. */ readOff = ReadPageInternal(state, targetPagePtr, - Min(targetRecOff + SizeOfXLogRecord, XLOG_BLCKSZ)); + Min(targetRecOff + XLogRecordMaxHdrSize, XLOG_BLCKSZ)); if (readOff == XLREAD_WOULDBLOCK) return XLREAD_WOULDBLOCK; else if (readOff < 0) @@ -644,13 +645,13 @@ restart: * Read the record length. * * NB: Even though we use an XLogRecord pointer here, the whole record - * header might not fit on this page. xl_tot_len is the first field of the - * struct, so it must be on this page (the records are MAXALIGNed), but we - * cannot access any other fields until we've verified that we got the - * whole header. + * header might not fit on this page. xl_payload_len is the third field of + * the data and starts at the third byte, so it must be on this page (the + * records are aligned to 8 bytes), but we cannot access most other fields + * until we've verified that we got the whole header. */ - record = (XLogRecord *) (state->readBuf + RecPtr % XLOG_BLCKSZ); - total_len = record->xl_tot_len; + record = (XLogRecHdr) (state->readBuf + (RecPtr % XLOG_BLCKSZ)); + total_len = XLogRecordTotalLength(record); /* * If the whole record header is on this page, validate it immediately. @@ -660,7 +661,7 @@ restart: * record" code path below; otherwise we might fail to apply * ValidXLogRecordHeader at all. */ - if (targetRecOff <= XLOG_BLCKSZ - SizeOfXLogRecord) + if (targetRecOff <= XLOG_BLCKSZ - XLogRecordHdrLen(record->xl_info)) { if (!ValidXLogRecordHeader(state, RecPtr, state->DecodeRecPtr, record, randAccess)) @@ -670,12 +671,12 @@ restart: else { /* XXX: more validation should be done here */ - if (total_len < SizeOfXLogRecord) + if (total_len < XLogRecordMinHdrSize) { report_invalid_record(state, "invalid record length at %X/%X: expected at least %u, got %u", LSN_FORMAT_ARGS(RecPtr), - (uint32) SizeOfXLogRecord, total_len); + (uint32) XLogRecordMinHdrSize, total_len); goto err; } gotheader = false; @@ -818,7 +819,7 @@ restart: /* If we just reassembled the record header, validate it. */ if (!gotheader) { - record = (XLogRecord *) state->readRecordBuf; + record = (XLogRecHdr) state->readRecordBuf; if (!ValidXLogRecordHeader(state, RecPtr, state->DecodeRecPtr, record, randAccess)) goto err; @@ -828,7 +829,7 @@ restart: Assert(gotheader); - record = (XLogRecord *) state->readRecordBuf; + record = (XLogRecHdr) state->readRecordBuf; if (!ValidXLogRecord(state, record, RecPtr)) goto err; @@ -860,7 +861,7 @@ restart: * Special processing if it's an XLOG SWITCH record */ if (record->xl_rmid == RM_XLOG_ID && - record->xl_rmgrinfo == XLOG_SWITCH) + XLRHdrGetRmgrInfo(record) == XLOG_SWITCH) { /* Pretend it extends to end of segment */ state->NextRecPtr += state->segcxt.ws_segsize - 1; @@ -1114,22 +1115,25 @@ XLogReaderInvalReadState(XLogReaderState *state) */ static bool ValidXLogRecordHeader(XLogReaderState *state, XLogRecPtr RecPtr, - XLogRecPtr PrevRecPtr, XLogRecord *record, + XLogRecPtr PrevRecPtr, XLogRecHdr rechdr, bool randAccess) { - if (record->xl_tot_len < SizeOfXLogRecord) + XLogRecord record = {0}; + XLogReadRecHdrInto(rechdr, XLogRecordHdrLen(rechdr->xl_info), &record); + + if (XLogRecordTotalLength(rechdr) < XLogRecordMinHdrSize) { report_invalid_record(state, "invalid record length at %X/%X: expected at least %u, got %u", LSN_FORMAT_ARGS(RecPtr), - (uint32) SizeOfXLogRecord, record->xl_tot_len); + (uint32) XLogRecordMinHdrSize, record.xl_payload_len); return false; } - if (!RmgrIdIsValid(record->xl_rmid)) + if (!RmgrIdIsValid(rechdr->xl_rmid)) { report_invalid_record(state, "invalid resource manager ID %u at %X/%X", - record->xl_rmid, LSN_FORMAT_ARGS(RecPtr)); + rechdr->xl_rmid, LSN_FORMAT_ARGS(RecPtr)); return false; } if (randAccess) @@ -1138,12 +1142,14 @@ ValidXLogRecordHeader(XLogReaderState *state, XLogRecPtr RecPtr, * We can't exactly verify the prev-link, but surely it should be less * than the record's own address. */ - if (!(record->xl_prev < RecPtr)) + if (!(record.xl_prev < RecPtr)) { report_invalid_record(state, - "record with incorrect prev-link %X/%X at %X/%X", - LSN_FORMAT_ARGS(record->xl_prev), - LSN_FORMAT_ARGS(RecPtr)); + "record with incorrect prev-link %X/%X at %X/%X, info %u, rmgr %u", + LSN_FORMAT_ARGS(record.xl_prev), + LSN_FORMAT_ARGS(RecPtr), + (uint32) record.xl_info, + (uint32) record.xl_rmid); return false; } } @@ -1154,12 +1160,13 @@ ValidXLogRecordHeader(XLogReaderState *state, XLogRecPtr RecPtr, * check guards against torn WAL pages where a stale but valid-looking * WAL record starts on a sector boundary. */ - if (record->xl_prev != PrevRecPtr) + if (record.xl_prev != PrevRecPtr) { report_invalid_record(state, - "record with incorrect prev-link %X/%X at %X/%X", - LSN_FORMAT_ARGS(record->xl_prev), - LSN_FORMAT_ARGS(RecPtr)); + "record with incorrect prev-link %X/%X at %X/%X, expected %X/%X", + LSN_FORMAT_ARGS(record.xl_prev), + LSN_FORMAT_ARGS(RecPtr), + LSN_FORMAT_ARGS(PrevRecPtr)); return false; } } @@ -1179,19 +1186,26 @@ ValidXLogRecordHeader(XLogReaderState *state, XLogRecPtr RecPtr, * SizeOfXLogRecord. */ static bool -ValidXLogRecord(XLogReaderState *state, XLogRecord *record, XLogRecPtr recptr) +ValidXLogRecord(XLogReaderState *state, XLogRecHdr record, XLogRecPtr recptr) { - pg_crc32c crc; + pg_crc32c crc, + hdr_crc; + Size hdr_len = XLogRecordHdrLen(record->xl_info); + Size rec_length = XLogRecordTotalLength(record); /* Calculate the CRC */ INIT_CRC32C(crc); - COMP_CRC32C(crc, ((char *) record) + SizeOfXLogRecord, record->xl_tot_len - SizeOfXLogRecord); + COMP_CRC32C(crc, ((char *) record) + hdr_len, rec_length - hdr_len); /* include the record header last */ - COMP_CRC32C(crc, (char *) record, offsetof(XLogRecord, xl_crc)); + COMP_CRC32C(crc, (char *) record, hdr_len - sizeof(pg_crc32c)); FIN_CRC32C(crc); - if (!EQ_CRC32C(record->xl_crc, crc)) + memcpy(&hdr_crc, ((char *) record) + hdr_len - sizeof(pg_crc32c), + sizeof(pg_crc32c)); + + if (!EQ_CRC32C(hdr_crc, crc)) { + Assert(false); report_invalid_record(state, "incorrect resource manager data checksum in record at %X/%X", LSN_FORMAT_ARGS(recptr)); @@ -1651,7 +1665,7 @@ DecodeXLogRecordRequiredSpace(size_t xl_tot_len) bool DecodeXLogRecord(XLogReaderState *state, DecodedXLogRecord *decoded, - XLogRecord *record, + XLogRecHdr record, XLogRecPtr lsn, char **errormsg) { @@ -1674,7 +1688,7 @@ DecodeXLogRecord(XLogReaderState *state, RelFileLocator *rlocator = NULL; uint8 block_id; - decoded->header = *record; + XLogReadRecHdrInto(record, XLogRecordHdrLen(record->xl_info), &decoded->header); decoded->lsn = lsn; decoded->next = NULL; decoded->record_origin = InvalidRepOriginId; @@ -1683,8 +1697,8 @@ DecodeXLogRecord(XLogReaderState *state, decoded->main_data_len = 0; decoded->max_block_id = -1; ptr = (char *) record; - ptr += SizeOfXLogRecord; - remaining = record->xl_tot_len - SizeOfXLogRecord; + ptr += XLogRecordHdrLen(record->xl_info); + remaining = decoded->header.xl_payload_len; /* Decode the headers */ datatotal = 0; @@ -1960,7 +1974,7 @@ DecodeXLogRecord(XLogReaderState *state, /* Report the actual size we used. */ decoded->size = MAXALIGN(out - (char *) decoded); - Assert(DecodeXLogRecordRequiredSpace(record->xl_tot_len) >= + Assert(DecodeXLogRecordRequiredSpace(decoded->header.xl_payload_len) >= decoded->size); return true; @@ -1989,6 +2003,7 @@ XLogRecGetBlockTag(XLogReaderState *record, uint8 block_id, if (!XLogRecGetBlockTagExtended(record, block_id, rlocator, forknum, blknum, NULL)) { + Assert(false); #ifndef FRONTEND elog(ERROR, "could not locate backup block with ID %d in WAL record", block_id); diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index aa9f21ca5c..89334c45ed 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -4000,10 +4000,10 @@ ReadCheckpointRecord(XLogPrefetcher *xlogprefetcher, XLogRecPtr RecPtr, (errmsg("invalid xl_rmgrinfo in checkpoint record"))); return NULL; } - if (record->xl_tot_len != SizeOfXLogRecord + SizeOfXLogRecordDataHeaderShort + sizeof(CheckPoint)) + if (record->xl_payload_len != SizeOfXLogRecordDataHeaderShort + sizeof(CheckPoint)) { ereport(LOG, - (errmsg("invalid length of checkpoint record"))); + (errmsg("invalid payload length of checkpoint record"))); return NULL; } return record; diff --git a/src/bin/pg_resetwal/pg_resetwal.c b/src/bin/pg_resetwal/pg_resetwal.c index 80cb9a5bc4..4e171eed1f 100644 --- a/src/bin/pg_resetwal/pg_resetwal.c +++ b/src/bin/pg_resetwal/pg_resetwal.c @@ -1040,12 +1040,15 @@ WriteEmptyXLOG(void) PGAlignedXLogBlock buffer; XLogPageHeader page; XLogLongPageHeader longpage; - XLogRecord *record; pg_crc32c crc; char path[MAXPGPATH]; int fd; int nbytes; char *recptr; + char *baserecptr; + const int rec_hdr_len = offsetof(XLogRecHdrData, xl_hdrdata) + + sizeof(uint8) + sizeof(XLogRecPtr) + sizeof(pg_crc32c); + const int rec_payload_len = sizeof(uint8) * 2 + sizeof(CheckPoint); memset(buffer.data, 0, XLOG_BLCKSZ); @@ -1061,26 +1064,33 @@ WriteEmptyXLOG(void) longpage->xlp_xlog_blcksz = XLOG_BLCKSZ; /* Insert the initial checkpoint record */ - recptr = (char *) page + SizeOfXLogLongPHD; - record = (XLogRecord *) recptr; - record->xl_prev = 0; - record->xl_xid = InvalidTransactionId; - record->xl_tot_len = SizeOfXLogRecord + SizeOfXLogRecordDataHeaderShort + sizeof(CheckPoint); - record->xl_info = 0; - record->xl_rmgrinfo = XLOG_CHECKPOINT_SHUTDOWN; - record->xl_rmid = RM_XLOG_ID; - - recptr += SizeOfXLogRecord; + baserecptr = recptr = (char *) page + SizeOfXLogLongPHD; + *(recptr++) = (char) XLS_UINT8; + *(recptr++) = (char) RM_XLOG_ID; + recptr += XLogWriteLength(SizeOfXLogRecordDataHeaderShort + sizeof(CheckPoint), + XLS_UINT8, XLS_UINT8, recptr); + + /* include prevptr */ + { + XLogRecPtr prevptr = 0; + memcpy(recptr, &prevptr, sizeof(XLogRecPtr)); + recptr += sizeof(XLogRecPtr); + } + /* reserve location of crc */ + recptr += sizeof(pg_crc32c); + *(recptr++) = (char) XLR_BLOCK_ID_DATA_SHORT; *(recptr++) = sizeof(CheckPoint); memcpy(recptr, &ControlFile.checkPointCopy, sizeof(CheckPoint)); INIT_CRC32C(crc); - COMP_CRC32C(crc, ((char *) record) + SizeOfXLogRecord, record->xl_tot_len - SizeOfXLogRecord); - COMP_CRC32C(crc, (char *) record, offsetof(XLogRecord, xl_crc)); + COMP_CRC32C(crc, baserecptr + rec_hdr_len, rec_payload_len); + COMP_CRC32C(crc, baserecptr, rec_hdr_len - sizeof(pg_crc32c)); FIN_CRC32C(crc); - record->xl_crc = crc; + + memcpy(baserecptr + rec_hdr_len - sizeof(pg_crc32c), + &crc, sizeof(pg_crc32c)); /* Write the first page */ XLogFilePath(path, ControlFile.checkPointCopy.ThisTimeLineID, diff --git a/src/include/access/rmgr.h b/src/include/access/rmgr.h index 3b6a497e1b..7038ba0833 100644 --- a/src/include/access/rmgr.h +++ b/src/include/access/rmgr.h @@ -41,7 +41,7 @@ typedef enum RmgrIds static inline bool RmgrIdIsBuiltin(int rmid) { - return rmid <= RM_MAX_BUILTIN_ID; + return rmid <= RM_MAX_BUILTIN_ID && rmid != RM_INVALID_ID; } static inline bool diff --git a/src/include/access/rmgrlist.h b/src/include/access/rmgrlist.h index 463bcb67c5..b53d731f56 100644 --- a/src/include/access/rmgrlist.h +++ b/src/include/access/rmgrlist.h @@ -25,6 +25,7 @@ */ /* symbol name, textual name, redo, desc, identify, startup, cleanup, mask, decode */ +PG_RMGR(RM_INVALID_ID, "invalid", NULL, NULL, NULL, NULL, NULL, NULL, NULL) PG_RMGR(RM_XLOG_ID, "XLOG", xlog_redo, xlog_desc, xlog_identify, NULL, NULL, NULL, xlog_decode) PG_RMGR(RM_XACT_ID, "Transaction", xact_redo, xact_desc, xact_identify, NULL, NULL, NULL, xact_decode) PG_RMGR(RM_SMGR_ID, "Storage", smgr_redo, smgr_desc, smgr_identify, NULL, NULL, NULL, NULL) diff --git a/src/include/access/xlog_internal.h b/src/include/access/xlog_internal.h index 876e2790f9..04bea1cfed 100644 --- a/src/include/access/xlog_internal.h +++ b/src/include/access/xlog_internal.h @@ -26,12 +26,20 @@ #include "pgtime.h" #include "storage/block.h" #include "storage/relfilelocator.h" +#include "transam.h" +/* + * WAL records (and other XLogPage page-level content) are aligned to 8 bytes. + * + * Note that the contents of the records is not aligned (!) + */ +#define ALIGNOF_XLP_CONTENT 8 +#define XLP_ALIGN(LEN) TYPEALIGN(ALIGNOF_XLP_CONTENT, LEN) /* * Each page of XLOG file has a header like this: */ -#define XLOG_PAGE_MAGIC 0xD113 /* can be used as WAL version indicator */ +#define XLOG_PAGE_MAGIC 0xD114 /* can be used as WAL version indicator */ typedef struct XLogPageHeaderData { @@ -49,7 +57,7 @@ typedef struct XLogPageHeaderData uint32 xlp_rem_len; /* total len of remaining data for record */ } XLogPageHeaderData; -#define SizeOfXLogShortPHD MAXALIGN(sizeof(XLogPageHeaderData)) +#define SizeOfXLogShortPHD XLP_ALIGN(sizeof(XLogPageHeaderData)) typedef XLogPageHeaderData *XLogPageHeader; @@ -66,7 +74,7 @@ typedef struct XLogLongPageHeaderData uint32 xlp_xlog_blcksz; /* just as a cross-check */ } XLogLongPageHeaderData; -#define SizeOfXLogLongPHD MAXALIGN(sizeof(XLogLongPageHeaderData)) +#define SizeOfXLogLongPHD XLP_ALIGN(sizeof(XLogLongPageHeaderData)) typedef XLogLongPageHeaderData *XLogLongPageHeader; @@ -381,7 +389,7 @@ static inline Size XLogWriteLength(uint32 length, XLogSizeClass sizeClass, case caseSizeClass: \ if ((caseSizeClass) <= maxSizeClass) \ { \ - field_type typedLength = length; \ + field_type typedLength = (field_type) length; \ memcpy(output, &typedLength, sizeof(field_type)); \ written = sizeof(field_type); \ } \ @@ -394,6 +402,9 @@ static inline Size XLogWriteLength(uint32 length, XLogSizeClass sizeClass, WRITE_OP(XLS_UINT8, uint8); WRITE_OP(XLS_UINT16, uint16); WRITE_OP(XLS_UINT32, uint32); + default: + Assert(false); + pg_unreachable(); } #undef WRITE_OP @@ -436,12 +447,95 @@ static inline Size XLogReadLength(uint32 *length, XLogSizeClass sizeClass, READ_OP(XLS_UINT8, uint8); READ_OP(XLS_UINT16, uint16); READ_OP(XLS_UINT32, uint32); + default: + Assert(false); + pg_unreachable(); } #undef READ_OP return readSize; } + +inline static uint8 XLRHdrGetRmgrInfo(XLogRecHdr record) +{ + XLogSizeClass recSizeClass; + int offset; + + if (!(record->xl_info & XLR_HAS_RMGRINFO)) + return 0; + + recSizeClass = XLR_SIZECLASS(record->xl_info); + /* xl_rmgrinfo is located immediately behind the xl_payload_len field */ + offset = XLogSizeClassToByteLength(recSizeClass); + + return (uint8) record->xl_hdrdata[offset]; +} + + +/* Works on any partial record */ +inline static Size XLogRecordTotalLength(XLogRecHdr record) +{ + uint8 xl_info = record->xl_info; + XLogSizeClass sizeClass; + uint32 length = 0; + sizeClass = XLR_SIZECLASS(xl_info); + + XLogReadLength(&length, sizeClass, XLS_UINT32, + &record->xl_hdrdata[0], 6); + + return (Size) length + XLogRecordHdrLen(xl_info); +} + +inline static void XLogReadRecHdrInto(XLogRecHdr recdata, Size length, + XLogRecord *record) +{ + Size offset = 0; + Size hdr_size PG_USED_FOR_ASSERTS_ONLY = 0; + XLogSizeClass sizeClass; + + Assert(length >= XLogRecordMinHdrSize); + + record->xl_info = recdata->xl_info; + + hdr_size = XLogRecordHdrLen(record->xl_info); + Assert(length >= hdr_size); + + record->xl_rmid = recdata->xl_rmid; + + sizeClass = XLR_SIZECLASS(record->xl_info); + offset += XLogReadLength(&record->xl_payload_len, sizeClass, + XLS_UINT32, &recdata->xl_hdrdata[offset], length - offset); + + if (record->xl_info & XLR_HAS_RMGRINFO) + { + record->xl_rmgrinfo = recdata->xl_hdrdata[offset]; + offset += sizeof(uint8); + } + else + { + record->xl_rmgrinfo = 0; + } + + if (record->xl_info & XLR_HAS_XID) + { + memcpy(&record->xl_xid, &recdata->xl_hdrdata[offset], sizeof(TransactionId)); + offset += sizeof(TransactionId); + } + else + { + record->xl_xid = InvalidTransactionId; + } + + memcpy(&record->xl_prev, &recdata->xl_hdrdata[offset], sizeof(XLogRecPtr)); + offset += sizeof(XLogRecPtr); + + memcpy(&record->xl_crc, &recdata->xl_hdrdata[offset], sizeof(pg_crc32c)); + offset += sizeof(pg_crc32c); + + Assert(hdr_size - 2 == offset); +} + /* * Method table for resource managers. * diff --git a/src/include/access/xlogreader.h b/src/include/access/xlogreader.h index cec7930b19..b2cc31beed 100644 --- a/src/include/access/xlogreader.h +++ b/src/include/access/xlogreader.h @@ -396,7 +396,7 @@ extern bool WALRead(XLogReaderState *state, extern size_t DecodeXLogRecordRequiredSpace(size_t xl_tot_len); extern bool DecodeXLogRecord(XLogReaderState *state, DecodedXLogRecord *decoded, - XLogRecord *record, + XLogRecHdr record, XLogRecPtr lsn, char **errormsg); @@ -404,7 +404,7 @@ extern bool DecodeXLogRecord(XLogReaderState *state, * Macros that provide access to parts of the record most recently returned by * XLogReadRecord() or XLogNextRecord(). */ -#define XLogRecGetTotalLen(decoder) ((decoder)->record->header.xl_tot_len) +#define XLogRecGetTotalLen(decoder) ((decoder)->record->header.xl_payload_len) #define XLogRecGetPrev(decoder) ((decoder)->record->header.xl_prev) #define XLogRecGetInfo(decoder) ((decoder)->record->header.xl_info) #define XLogRecGetRmgrInfo(decoder) ((decoder)->record->header.xl_rmgrinfo) diff --git a/src/include/access/xlogrecord.h b/src/include/access/xlogrecord.h index 42b06f163e..af615e2866 100644 --- a/src/include/access/xlogrecord.h +++ b/src/include/access/xlogrecord.h @@ -38,9 +38,38 @@ typedef enum XLogSizeClass { XLS_UINT32 = 3 /* length <= UINT32_MAX; stored in uint32 (4B) */ } XLogSizeClass; +static inline int XLogSizeClassToByteLength(XLogSizeClass sz) +{ + switch (sz) { + case XLS_EMPTY: + return 0; + case XLS_UINT8: + return sizeof(uint8); + case XLS_UINT16: + return sizeof(uint16); + case XLS_UINT32: + return sizeof(uint32); + default: + pg_unreachable(); + } +} + +typedef struct XLogRecord +{ + uint8 xl_info; /* flag bits, see below */ + RmgrId xl_rmid; /* resource manager for this record */ + uint32 xl_payload_len; /* total len of entire record */ + uint8 xl_rmgrinfo; /* rmgr flag bits, see below */ + TransactionId xl_xid; /* xact id */ + XLogRecPtr xl_prev; /* ptr to previous record in log */ + pg_crc32c xl_crc; /* CRC for this record */ +} XLogRecord; + +#define SizeOfXLogRecord (offsetof(XLogRecord, xl_crc) + sizeof(pg_crc32c)) + /* * The overall layout of an XLOG record is: - * Fixed-size header (XLogRecord struct) + * Variable-size header (containing the data of the XLogRecord struct) * XLogRecordBlockHeader struct * XLogRecordBlockHeader struct * ... @@ -50,6 +79,15 @@ typedef enum XLogSizeClass { * ... * main data * + * Different xlog record headers need to store different header fields, so + * depending on flags in the xl_info field the layout may change. + * + * Note that for records with a payload larger than 0xFFFFFFF, the size field + * will be 4 bytes long, which can be larger than the minimum MAXALIGN quantum. + * To still be able to read the record correctly, we always align WAL to 8 + * bytes, so that the length bytes are always present in the first aligned + * quantum of data. + * * There can be zero or more XLogRecordBlockHeaders, and 0 or more bytes of * rmgr-specific data not associated with a block. XLogRecord structs * always start on MAXALIGN boundaries in the WAL files, but the rest of @@ -59,22 +97,25 @@ typedef enum XLogSizeClass { * XLogRecordDataHeaderLong structs all begin with a single 'id' byte. It's * used to distinguish between block references, and the main data structs. */ -typedef struct XLogRecord +typedef struct XLogRecHdrData { - uint32 xl_tot_len; /* total len of entire record */ - TransactionId xl_xid; /* xact id */ - XLogRecPtr xl_prev; /* ptr to previous record in log */ uint8 xl_info; /* flag bits, see below */ RmgrId xl_rmid; /* resource manager for this record */ - uint8 xl_rmgrinfo; /* rmgr flag bits, see below */ - /* 1 byte of padding here, initialize to zero */ - pg_crc32c xl_crc; /* CRC for this record */ - - /* XLogRecordBlockHeaders and XLogRecordDataHeader follow, no padding */ + char xl_hdrdata[FLEXIBLE_ARRAY_MEMBER]; /* variable length record data */ -} XLogRecord; + /* + * Payload is as follows, in order: + * XLogRecord data encoded + * xl_payload_len; 0, 1, 2 or 4 bytes + * xl_rmgrinfo; 0 or 1 byte + * xl_xid; 0 or 4 bytes + * xl_prev; 8 bytes + * xl_prev follows; 8 bytes + * record payload, if present. + */ +} XLogRecHdrData; -#define SizeOfXLogRecord (offsetof(XLogRecord, xl_crc) + sizeof(pg_crc32c)) +typedef XLogRecHdrData *XLogRecHdr; /* * XLogReader needs to allocate all the data of a WAL record in a single @@ -87,13 +128,28 @@ typedef struct XLogRecord */ #define XLogRecordMaxSize (1020 * 1024 * 1024) +#define XLR_SIZECLASS_MASK 0x03 +#define XLR_SIZECLASS(xl_info) ((XLogSizeClass) ((xl_info) & XLR_SIZECLASS_MASK)) + +/* + * There are several rmgrs which don't use info bits, so we omit that + * byte whenever possible. + */ +#define XLR_HAS_RMGRINFO 0x04 + +/* + * If a WAL record uses the current transaction ID, that will be included + * in the record header. In all other cases we omit the XID to save bytes. + */ +#define XLR_HAS_XID 0x08 + /* * If a WAL record modifies any relation files, in ways not covered by the * usual block references, this flag is set. This is not used for anything * by PostgreSQL itself, but it allows external tools that read WAL and keep * track of modified blocks to recognize such special record types. */ -#define XLR_SPECIAL_REL_UPDATE 0x01 +#define XLR_SPECIAL_REL_UPDATE 0x10 /* * Enforces consistency checks of replayed WAL at recovery. If enabled, @@ -102,7 +158,48 @@ typedef struct XLogRecord * of XLogInsert can use this value if necessary, but if * wal_consistency_checking is enabled for a rmgr this is set unconditionally. */ -#define XLR_CHECK_CONSISTENCY 0x02 +#define XLR_CHECK_CONSISTENCY (0x20) + +#define XLogRecordMaxHdrSize ( \ + sizeof(uint8) /* xl_info */ + \ + sizeof(uint8) /* xl_rmid */ + \ + sizeof(uint32) /* xl_payload_len */ + \ + sizeof(uint8) /* xl_rmgrinfo */ + \ + sizeof(TransactionId) /* xl_xid */ + \ + sizeof(XLogRecPtr) /* xl_prev */ + \ + sizeof(pg_crc32c) /* xl_crc */ \ +) + +#define XLogRecordMinHdrSize ( \ + sizeof(uint8) /* xl_info */ + \ + sizeof(uint8) /* xl_rmid */ + \ + 0 /* xl_payload_len */ + \ + 0 /* xl_rmgrinfo */ + \ + 0 /* xl_xid */ + \ + sizeof(XLogRecPtr) /* xl_prev */ + \ + sizeof(pg_crc32c) /* xl_crc */ \ +) + +static inline Size XLogRecordHdrLen(uint8 info) +{ + Size size = 0; + /* xl_info */ + size += sizeof(uint8); + /* xl_rmid */ + size += sizeof(uint8); + /* xl_payload_len */ + size += XLogSizeClassToByteLength(XLR_SIZECLASS(info)); + /* xl_rmgrinfo */ + size += ((info) & XLR_HAS_RMGRINFO) ? sizeof(uint8) : 0; + /* xl_xid */ + size += ((info) & XLR_HAS_XID) ? sizeof(TransactionId) : 0; + /* xl_prev */ + size += sizeof(XLogRecPtr); + /* xl_crc */ + size += sizeof(pg_crc32c); + + return size; +} /* * Header info for block data appended to an XLOG record. -- 2.39.0 ^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: XLog size reductions: Reduced XLog record header size for PG17 @ 2023-06-30 15:36 Matthias van de Meent <[email protected]> parent: Matthias van de Meent <[email protected]> 0 siblings, 1 reply; 35+ messages in thread From: Matthias van de Meent @ 2023-06-30 15:36 UTC (permalink / raw) To: pgsql-hackers; +Cc: Heikki Linnakangas <[email protected]>; Robert Haas <[email protected]>; Dilip Kumar <[email protected]>; Andres Freund <[email protected]> Hi, The attached v2 patchset contains some small fixes for the failing cfbot 32-bit tests - at least locally it does so. I'd overlooked one remaining use of MAXALIGN64() in xlog.c in the last patch of the set, which has now been updated to XLP_ALIGN as well. Additionally, XLP_ALIGN has been updated to use TYPEALIGN64 instead of TYPEALIGN so that we don't lose bits of the aligned value in 32-bit systems. Kind regards, Matthias van de Meent Neon (https://neon.tech/) Attachments: [application/octet-stream] v2-0004-Move-rmgr-info-bits-into-their-own-field-in-the-x.patch (26.1K, ../../CAEze2Wj+L1AGNAwjuQFYMcu2be0NYEw1oSg8Sa-HvVbC7iHZLQ@mail.gmail.com/2-v2-0004-Move-rmgr-info-bits-into-their-own-field-in-the-x.patch) download | inline diff: From 8956da9e1a6bc55a6256272ee4438e8693a87f18 Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Mon, 23 Jan 2023 17:21:12 +0100 Subject: [PATCH v2 4/6] Move rmgr info bits into their own field in the xlog record header. This takes one byte of the 2 previously lost to alignment, and allows each rmgr to use more info bits (useful for e.g. merging the heap rmgrs back into one). --- contrib/pg_walinspect/pg_walinspect.c | 7 +++--- src/backend/access/transam/xact.c | 22 +++++++++-------- src/backend/access/transam/xlog.c | 7 +++--- src/backend/access/transam/xloginsert.c | 27 ++++++++++++--------- src/backend/access/transam/xlogprefetcher.c | 2 +- src/backend/access/transam/xlogreader.c | 2 +- src/backend/access/transam/xlogrecovery.c | 10 ++++---- src/backend/access/transam/xlogstats.c | 12 ++++----- src/backend/catalog/storage.c | 7 +++--- src/backend/commands/dbcommands.c | 20 +++++++++------ src/backend/utils/probes.d | 2 +- src/bin/pg_resetwal/pg_resetwal.c | 3 ++- src/bin/pg_rewind/parsexlog.c | 4 +-- src/bin/pg_waldump/pg_waldump.c | 7 +++--- src/include/access/brin_xlog.h | 3 +-- src/include/access/heapam_xlog.h | 4 +-- src/include/access/nbtxlog.h | 3 +-- src/include/access/xact.h | 6 ++--- src/include/access/xlog_internal.h | 2 +- src/include/access/xloginsert.h | 3 ++- src/include/access/xlogreader.h | 4 +-- src/include/access/xlogrecord.h | 11 ++------- src/include/access/xlogstats.h | 5 ++-- 23 files changed, 89 insertions(+), 84 deletions(-) diff --git a/contrib/pg_walinspect/pg_walinspect.c b/contrib/pg_walinspect/pg_walinspect.c index 1b6f4cb178..909700f9a1 100644 --- a/contrib/pg_walinspect/pg_walinspect.c +++ b/contrib/pg_walinspect/pg_walinspect.c @@ -697,7 +697,7 @@ GetXLogSummaryStats(XLogStats *stats, ReturnSetInfo *rsinfo, { int rj; - for (rj = 0; rj < MAX_XLINFO_TYPES; rj++) + for (rj = 0; rj < MAX_XLRMGRINFO_TYPES; rj++) { const char *id; @@ -712,10 +712,9 @@ GetXLogSummaryStats(XLogStats *stats, ReturnSetInfo *rsinfo, old_cxt = MemoryContextSwitchTo(tmp_cxt); - /* the upper four bits in xl_info are the rmgr's */ - id = desc.rm_identify(rj << 4); + id = desc.rm_identify(rj); if (id == NULL) - id = psprintf("UNKNOWN (%x)", rj << 4); + id = psprintf("UNKNOWN (%x)", rj); FillXLogStatsRow(psprintf("%s/%s", desc.rm_name, id), count, total_count, rec_len, total_rec_len, fpi_len, diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c560f07390..da0ac6c8d3 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -5650,7 +5650,8 @@ XactLogCommitRecord(TimestampTz commit_time, xl_xact_invals xl_invals; xl_xact_twophase xl_twophase; xl_xact_origin xl_origin; - uint8 info; + uint8 info = 0; + uint8 rmgr_info; Assert(CritSectionCount > 0); @@ -5658,9 +5659,9 @@ XactLogCommitRecord(TimestampTz commit_time, /* decide between a plain and 2pc commit */ if (!TransactionIdIsValid(twophase_xid)) - info = XLOG_XACT_COMMIT; + rmgr_info = XLOG_XACT_COMMIT; else - info = XLOG_XACT_COMMIT_PREPARED; + rmgr_info = XLOG_XACT_COMMIT_PREPARED; /* First figure out and collect all the information needed */ @@ -5736,7 +5737,7 @@ XactLogCommitRecord(TimestampTz commit_time, } if (xl_xinfo.xinfo != 0) - info |= XLOG_XACT_HAS_INFO; + rmgr_info |= XLOG_XACT_HAS_INFO; /* Then include all the collected data into the commit record. */ @@ -5794,7 +5795,7 @@ XactLogCommitRecord(TimestampTz commit_time, /* we allow filtering by xacts */ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); - return XLogInsert(RM_XACT_ID, info); + return XLogInsertExtended(RM_XACT_ID, info, rmgr_info); } /* @@ -5820,7 +5821,8 @@ XactLogAbortRecord(TimestampTz abort_time, xl_xact_dbinfo xl_dbinfo; xl_xact_origin xl_origin; - uint8 info; + uint8 info = 0; + uint8 rmgr_info; Assert(CritSectionCount > 0); @@ -5828,9 +5830,9 @@ XactLogAbortRecord(TimestampTz abort_time, /* decide between a plain and 2pc abort */ if (!TransactionIdIsValid(twophase_xid)) - info = XLOG_XACT_ABORT; + rmgr_info = XLOG_XACT_ABORT; else - info = XLOG_XACT_ABORT_PREPARED; + rmgr_info = XLOG_XACT_ABORT_PREPARED; /* First figure out and collect all the information needed */ @@ -5889,7 +5891,7 @@ XactLogAbortRecord(TimestampTz abort_time, } if (xl_xinfo.xinfo != 0) - info |= XLOG_XACT_HAS_INFO; + rmgr_info |= XLOG_XACT_HAS_INFO; /* Then include all the collected data into the abort record. */ @@ -5940,7 +5942,7 @@ XactLogAbortRecord(TimestampTz abort_time, /* Include the replication origin */ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); - return XLogInsert(RM_XACT_ID, info); + return XLogInsertExtended(RM_XACT_ID, info, rmgr_info); } /* diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index b01cd6cc23..4b1ff0d1aa 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -738,9 +738,9 @@ XLogInsertRecord(XLogRecData *rdata, pg_crc32c rdata_crc; bool inserted; XLogRecord *rechdr = (XLogRecord *) rdata->data; - uint8 info = rechdr->xl_info & ~XLR_INFO_MASK; + uint8 rmgrinfo = rechdr->xl_rmgrinfo; bool isLogSwitch = (rechdr->xl_rmid == RM_XLOG_ID && - info == XLOG_SWITCH); + rmgrinfo == XLOG_SWITCH); XLogRecPtr StartPos; XLogRecPtr EndPos; bool prevDoPageWrites = doPageWrites; @@ -4730,7 +4730,8 @@ BootStrapXLOG(void) record->xl_prev = 0; record->xl_xid = InvalidTransactionId; record->xl_tot_len = SizeOfXLogRecord + SizeOfXLogRecordDataHeaderShort + sizeof(checkPoint); - record->xl_info = XLOG_CHECKPOINT_SHUTDOWN; + record->xl_info = 0; + record->xl_rmgrinfo = XLOG_CHECKPOINT_SHUTDOWN; record->xl_rmid = RM_XLOG_ID; recptr += SizeOfXLogRecord; /* fill the XLogRecordDataHeaderShort struct */ diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c index 0fce063ca4..9a3eb4fb4b 100644 --- a/src/backend/access/transam/xloginsert.c +++ b/src/backend/access/transam/xloginsert.c @@ -135,7 +135,7 @@ static bool begininsert_called = false; /* Memory context to hold the registered buffer and data references. */ static MemoryContext xloginsert_cxt; -static XLogRecData *XLogRecordAssemble(RmgrId rmid, uint8 info, +static XLogRecData *XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecPtr, bool doPageWrites, XLogRecPtr *fpw_lsn, int *num_fpi, bool *topxid_included); @@ -447,9 +447,9 @@ XLogSetRecordFlags(uint8 flags) } /* - * Insert an XLOG record having the specified RMID and info bytes, with the - * body of the record being the data and buffer references registered earlier - * with XLogRegister* calls. + * Insert an XLOG record having the specified RMID and rmgr_info bytes, with + * the body of the record being the data and buffer references registered + * earlier with XLogRegister* calls. * * Returns XLOG pointer to end of record (beginning of next record). * This can be used as LSN for data pages affected by the logged action. @@ -458,7 +458,13 @@ XLogSetRecordFlags(uint8 flags) * WAL rule "write the log before the data".) */ XLogRecPtr -XLogInsert(RmgrId rmid, uint8 info) +XLogInsert(RmgrId rmid, uint8 rmgr_info) +{ + return XLogInsertExtended(rmid, 0, rmgr_info); +} + +XLogRecPtr +XLogInsertExtended(RmgrId rmid, uint8 info, uint8 rmgr_info) { XLogRecPtr EndPos; @@ -470,12 +476,11 @@ XLogInsert(RmgrId rmid, uint8 info) * The caller can set rmgr bits, XLR_SPECIAL_REL_UPDATE and * XLR_CHECK_CONSISTENCY; the rest are reserved for use by me. */ - if ((info & ~(XLR_RMGR_INFO_MASK | - XLR_SPECIAL_REL_UPDATE | + if ((info & ~(XLR_SPECIAL_REL_UPDATE | XLR_CHECK_CONSISTENCY)) != 0) elog(PANIC, "invalid xlog info mask %02X", info); - TRACE_POSTGRESQL_WAL_INSERT(rmid, info); + TRACE_POSTGRESQL_WAL_INSERT(rmid, info, rmgr_info); /* * In bootstrap mode, we don't actually log anything but XLOG resources; @@ -504,7 +509,7 @@ XLogInsert(RmgrId rmid, uint8 info) */ GetFullPageWriteInfo(&RedoRecPtr, &doPageWrites); - rdt = XLogRecordAssemble(rmid, info, RedoRecPtr, doPageWrites, + rdt = XLogRecordAssemble(rmid, info, rmgr_info, RedoRecPtr, doPageWrites, &fpw_lsn, &num_fpi, &topxid_included); EndPos = XLogInsertRecord(rdt, fpw_lsn, curinsert_flags, num_fpi, @@ -532,8 +537,7 @@ XLogInsert(RmgrId rmid, uint8 info) * current subtransaction. */ static XLogRecData * -XLogRecordAssemble(RmgrId rmid, uint8 info, - XLogRecPtr RedoRecPtr, bool doPageWrites, +XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecPtr, bool doPageWrites, XLogRecPtr *fpw_lsn, int *num_fpi, bool *topxid_included) { XLogRecData *rdt; @@ -924,6 +928,7 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, rechdr->xl_tot_len = (uint32) total_len; rechdr->xl_info = info; rechdr->xl_rmid = rmid; + rechdr->xl_rmgrinfo = rmgr_info; rechdr->xl_prev = InvalidXLogRecPtr; rechdr->xl_crc = rdata_crc; diff --git a/src/backend/access/transam/xlogprefetcher.c b/src/backend/access/transam/xlogprefetcher.c index 539928cb85..dd4ca866a4 100644 --- a/src/backend/access/transam/xlogprefetcher.c +++ b/src/backend/access/transam/xlogprefetcher.c @@ -536,7 +536,7 @@ XLogPrefetcherNextBlock(uintptr_t pgsr_private, XLogRecPtr *lsn) if (replaying_lsn < record->lsn) { uint8 rmid = record->header.xl_rmid; - uint8 record_type = record->header.xl_info & ~XLR_INFO_MASK; + uint8 record_type = record->header.xl_rmgrinfo; if (rmid == RM_XLOG_ID) { diff --git a/src/backend/access/transam/xlogreader.c b/src/backend/access/transam/xlogreader.c index 24c6c88b2f..733de8a8dc 100644 --- a/src/backend/access/transam/xlogreader.c +++ b/src/backend/access/transam/xlogreader.c @@ -860,7 +860,7 @@ restart: * Special processing if it's an XLOG SWITCH record */ if (record->xl_rmid == RM_XLOG_ID && - (record->xl_info & ~XLR_INFO_MASK) == XLOG_SWITCH) + record->xl_rmgrinfo == XLOG_SWITCH) { /* Pretend it extends to end of segment */ state->NextRecPtr += state->segcxt.ws_segsize - 1; diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index 6508246999..56fa2f74a2 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -632,7 +632,7 @@ InitWalRecovery(ControlFileData *ControlFile, bool *wasShutdown_ptr, if (record != NULL) { memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint)); - wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN); + wasShutdown = (record->xl_rmgrinfo == XLOG_CHECKPOINT_SHUTDOWN); ereport(DEBUG1, (errmsg_internal("checkpoint record is at %X/%X", LSN_FORMAT_ARGS(CheckPointLoc)))); @@ -786,7 +786,7 @@ InitWalRecovery(ControlFileData *ControlFile, bool *wasShutdown_ptr, (errmsg("could not locate a valid checkpoint record"))); } memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint)); - wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN); + wasShutdown = (record->xl_rmgrinfo == XLOG_CHECKPOINT_SHUTDOWN); } /* @@ -1857,7 +1857,7 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl { TimeLineID newReplayTLI = *replayTLI; TimeLineID prevReplayTLI = *replayTLI; - uint8 info = record->xl_info & ~XLR_INFO_MASK; + uint8 info = record->xl_rmgrinfo; if (info == XLOG_CHECKPOINT_SHUTDOWN) { @@ -3992,12 +3992,12 @@ ReadCheckpointRecord(XLogPrefetcher *xlogprefetcher, XLogRecPtr RecPtr, (errmsg("invalid resource manager ID in checkpoint record"))); return NULL; } - info = record->xl_info & ~XLR_INFO_MASK; + info = record->xl_rmgrinfo; if (info != XLOG_CHECKPOINT_SHUTDOWN && info != XLOG_CHECKPOINT_ONLINE) { ereport(LOG, - (errmsg("invalid xl_info in checkpoint record"))); + (errmsg("invalid xl_rmgrinfo in checkpoint record"))); return NULL; } if (record->xl_tot_len != SizeOfXLogRecord + SizeOfXLogRecordDataHeaderShort + sizeof(CheckPoint)) diff --git a/src/backend/access/transam/xlogstats.c b/src/backend/access/transam/xlogstats.c index 267b108943..28d6d971ac 100644 --- a/src/backend/access/transam/xlogstats.c +++ b/src/backend/access/transam/xlogstats.c @@ -74,21 +74,21 @@ XLogRecStoreStats(XLogStats *stats, XLogReaderState *record) /* * Update per-record statistics, where the record is identified by a - * combination of the RmgrId and the four bits of the xl_info field that - * are the rmgr's domain (resulting in sixteen possible entries per + * combination of the RmgrId and the eight bits of the xl_rmgrinfo field + * that are the rmgr's domain (resulting in 256 possible entries per * RmgrId). */ - recid = XLogRecGetRmgrInfo(record) >> 4; + recid = XLogRecGetRmgrInfo(record); /* * XACT records need to be handled differently. Those records use the * first bit of those four bits for an optional flag variable and the - * following three bits for the opcode. We filter opcode out of xl_info - * and use it as the identifier of the record. + * following three bits for the opcode. We filter opcode out of + * xl_rmgrinfo and use it as the identifier of the record. */ if (rmid == RM_XACT_ID) - recid &= 0x07; + recid &= 0x70; stats->record_stats[rmid][recid].count++; stats->record_stats[rmid][recid].rec_len += rec_len; diff --git a/src/backend/catalog/storage.c b/src/backend/catalog/storage.c index 4d73023a65..5b48b24e7b 100644 --- a/src/backend/catalog/storage.c +++ b/src/backend/catalog/storage.c @@ -194,7 +194,7 @@ log_smgrcreate(const RelFileLocator *rlocator, ForkNumber forkNum) XLogBeginInsert(); XLogRegisterData((char *) &xlrec, sizeof(xlrec)); - XLogInsert(RM_SMGR_ID, XLOG_SMGR_CREATE | XLR_SPECIAL_REL_UPDATE); + XLogInsertExtended(RM_SMGR_ID, XLR_SPECIAL_REL_UPDATE, XLOG_SMGR_CREATE); } /* @@ -375,8 +375,9 @@ RelationTruncate(Relation rel, BlockNumber nblocks) XLogBeginInsert(); XLogRegisterData((char *) &xlrec, sizeof(xlrec)); - lsn = XLogInsert(RM_SMGR_ID, - XLOG_SMGR_TRUNCATE | XLR_SPECIAL_REL_UPDATE); + lsn = XLogInsertExtended(RM_SMGR_ID, + XLR_SPECIAL_REL_UPDATE, + XLOG_SMGR_TRUNCATE); /* * Flush, because otherwise the truncation of the main relation might diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c index dffb683e74..fe89540aed 100644 --- a/src/backend/commands/dbcommands.c +++ b/src/backend/commands/dbcommands.c @@ -624,8 +624,9 @@ CreateDatabaseUsingFileCopy(Oid src_dboid, Oid dst_dboid, Oid src_tsid, XLogRegisterData((char *) &xlrec, sizeof(xl_dbase_create_file_copy_rec)); - (void) XLogInsert(RM_DBASE_ID, - XLOG_DBASE_CREATE_FILE_COPY | XLR_SPECIAL_REL_UPDATE); + (void) XLogInsertExtended(RM_DBASE_ID, + XLR_SPECIAL_REL_UPDATE, + XLOG_DBASE_CREATE_FILE_COPY); } pfree(srcpath); pfree(dstpath); @@ -2086,8 +2087,9 @@ movedb(const char *dbname, const char *tblspcname) XLogRegisterData((char *) &xlrec, sizeof(xl_dbase_create_file_copy_rec)); - (void) XLogInsert(RM_DBASE_ID, - XLOG_DBASE_CREATE_FILE_COPY | XLR_SPECIAL_REL_UPDATE); + (void) XLogInsertExtended(RM_DBASE_ID, + XLR_SPECIAL_REL_UPDATE, + XLOG_DBASE_CREATE_FILE_COPY); } /* @@ -2180,8 +2182,9 @@ movedb(const char *dbname, const char *tblspcname) XLogRegisterData((char *) &xlrec, sizeof(xl_dbase_drop_rec)); XLogRegisterData((char *) &src_tblspcoid, sizeof(Oid)); - (void) XLogInsert(RM_DBASE_ID, - XLOG_DBASE_DROP | XLR_SPECIAL_REL_UPDATE); + (void) XLogInsertExtended(RM_DBASE_ID, + XLR_SPECIAL_REL_UPDATE, + XLOG_DBASE_DROP); } /* Now it's safe to release the database lock */ @@ -2903,8 +2906,9 @@ remove_dbtablespaces(Oid db_id) XLogRegisterData((char *) &xlrec, MinSizeOfDbaseDropRec); XLogRegisterData((char *) tablespace_ids, ntblspc * sizeof(Oid)); - (void) XLogInsert(RM_DBASE_ID, - XLOG_DBASE_DROP | XLR_SPECIAL_REL_UPDATE); + (void) XLogInsertExtended(RM_DBASE_ID, + XLR_SPECIAL_REL_UPDATE, + XLOG_DBASE_DROP); } list_free(ltblspc); diff --git a/src/backend/utils/probes.d b/src/backend/utils/probes.d index 0af275587b..131067b5ac 100644 --- a/src/backend/utils/probes.d +++ b/src/backend/utils/probes.d @@ -87,7 +87,7 @@ provider postgresql { probe smgr__md__write__start(ForkNumber, BlockNumber, Oid, Oid, Oid, int); probe smgr__md__write__done(ForkNumber, BlockNumber, Oid, Oid, Oid, int, int, int); - probe wal__insert(unsigned char, unsigned char); + probe wal__insert(unsigned char, unsigned char, unsigned char); probe wal__switch(); probe wal__buffer__write__dirty__start(); probe wal__buffer__write__dirty__done(); diff --git a/src/bin/pg_resetwal/pg_resetwal.c b/src/bin/pg_resetwal/pg_resetwal.c index e7ef2b8bd0..80cb9a5bc4 100644 --- a/src/bin/pg_resetwal/pg_resetwal.c +++ b/src/bin/pg_resetwal/pg_resetwal.c @@ -1066,7 +1066,8 @@ WriteEmptyXLOG(void) record->xl_prev = 0; record->xl_xid = InvalidTransactionId; record->xl_tot_len = SizeOfXLogRecord + SizeOfXLogRecordDataHeaderShort + sizeof(CheckPoint); - record->xl_info = XLOG_CHECKPOINT_SHUTDOWN; + record->xl_info = 0; + record->xl_rmgrinfo = XLOG_CHECKPOINT_SHUTDOWN; record->xl_rmid = RM_XLOG_ID; recptr += SizeOfXLogRecord; diff --git a/src/bin/pg_rewind/parsexlog.c b/src/bin/pg_rewind/parsexlog.c index 2b6b46f0e3..aa03fe8031 100644 --- a/src/bin/pg_rewind/parsexlog.c +++ b/src/bin/pg_rewind/parsexlog.c @@ -438,9 +438,9 @@ extractPageInfo(XLogReaderState *record) * track that change. */ pg_fatal("WAL record modifies a relation, but record type is not recognized: " - "lsn: %X/%X, rmid: %d, rmgr: %s, info: %02X", + "lsn: %X/%X, rmid: %d, rmgr: %s, info: %02X, rmgrinfo: %02X", LSN_FORMAT_ARGS(record->ReadRecPtr), - rmid, RmgrName(rmid), info | rminfo); + rmid, RmgrName(rmid), info, rminfo); } for (block_id = 0; block_id <= XLogRecMaxBlockId(record); block_id++) diff --git a/src/bin/pg_waldump/pg_waldump.c b/src/bin/pg_waldump/pg_waldump.c index 85694b3d5c..713c526385 100644 --- a/src/bin/pg_waldump/pg_waldump.c +++ b/src/bin/pg_waldump/pg_waldump.c @@ -694,7 +694,7 @@ XLogDumpDisplayStats(XLogDumpConfig *config, XLogStats *stats) } else { - for (rj = 0; rj < MAX_XLINFO_TYPES; rj++) + for (rj = 0; rj < MAX_XLRMGRINFO_TYPES; rj++) { const char *id; @@ -707,10 +707,9 @@ XLogDumpDisplayStats(XLogDumpConfig *config, XLogStats *stats) if (count == 0) continue; - /* the upper four bits in xl_info are the rmgr's */ - id = desc->rm_identify(rj << 4); + id = desc->rm_identify(rj); if (id == NULL) - id = psprintf("UNKNOWN (%x)", rj << 4); + id = psprintf("UNKNOWN (%x)", rj); XLogDumpStatsRow(psprintf("%s/%s", desc->rm_name, id), count, total_count, rec_len, total_rec_len, diff --git a/src/include/access/brin_xlog.h b/src/include/access/brin_xlog.h index 45a688d09b..92190688c9 100644 --- a/src/include/access/brin_xlog.h +++ b/src/include/access/brin_xlog.h @@ -25,8 +25,7 @@ /* * WAL record definitions for BRIN's WAL operations * - * XLOG allows to store some information in high 4 bits of log - * record xl_info field. + * XLOG allows to store some information in the 8-bit xl_rmgrinfo field. */ #define XLOG_BRIN_CREATE_INDEX 0x00 #define XLOG_BRIN_INSERT 0x10 diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h index 70e9acd350..6705c8a316 100644 --- a/src/include/access/heapam_xlog.h +++ b/src/include/access/heapam_xlog.h @@ -26,8 +26,8 @@ /* * WAL record definitions for heapam.c's WAL operations * - * XLOG allows to store some information in high 4 bits of log - * record xl_info field. We use 3 for opcode and one for init bit. + * XLOG allows to store some information in the 8-bit xl_rmgrinfo field. + * We use 3 for opcode and one for init bit. */ #define XLOG_HEAP_INSERT 0x00 #define XLOG_HEAP_DELETE 0x10 diff --git a/src/include/access/nbtxlog.h b/src/include/access/nbtxlog.h index 422ac13d56..6d1231a01e 100644 --- a/src/include/access/nbtxlog.h +++ b/src/include/access/nbtxlog.h @@ -21,8 +21,7 @@ /* * XLOG records for btree operations * - * XLOG allows to store some information in high 4 bits of log - * record xl_info field + * XLOG allows to store some information in the 8-bit xl_rmgrinfo field. */ #define XLOG_BTREE_INSERT_LEAF 0x00 /* add index tuple without split */ #define XLOG_BTREE_INSERT_UPPER 0x10 /* same, on a non-leaf page */ diff --git a/src/include/access/xact.h b/src/include/access/xact.h index b94b264402..2a9a708279 100644 --- a/src/include/access/xact.h +++ b/src/include/access/xact.h @@ -163,8 +163,8 @@ typedef struct SavedTransactionCharacteristics */ /* - * XLOG allows to store some information in high 4 bits of log record xl_info - * field. We use 3 for the opcode, and one about an optional flag variable. + * XLOG allows to store some information in the 8-bit xl_rmgrinfo field. + * We use 3 for the opcode, and one about an optional flag variable. */ #define XLOG_XACT_COMMIT 0x00 #define XLOG_XACT_PREPARE 0x10 @@ -232,7 +232,7 @@ typedef struct xl_xact_assignment * A minimal commit/abort record only consists of a xl_xact_commit/abort * struct. The presence of additional information is indicated by bits set in * 'xl_xact_xinfo->xinfo'. The presence of the xinfo field itself is signaled - * by a set XLOG_XACT_HAS_INFO bit in the xl_info field. + * by a set XLOG_XACT_HAS_INFO bit in the xl_rmgrinfo field. * * NB: All the individual data chunks should be sized to multiples of * sizeof(int) and only require int32 alignment. If they require bigger diff --git a/src/include/access/xlog_internal.h b/src/include/access/xlog_internal.h index 42b3c66547..876e2790f9 100644 --- a/src/include/access/xlog_internal.h +++ b/src/include/access/xlog_internal.h @@ -448,7 +448,7 @@ static inline Size XLogReadLength(uint32 *length, XLogSizeClass sizeClass, * This struct must be kept in sync with the PG_RMGR definition in * rmgr.c. * - * rm_identify must return a name for the record based on xl_info (without + * rm_identify must return a name for the record based on xl_rmgrinfo (without * reference to the rmid). For example, XLOG_BTREE_VACUUM would be named * "VACUUM". rm_desc can then be called to obtain additional detail for the * record, if available (e.g. the last block). diff --git a/src/include/access/xloginsert.h b/src/include/access/xloginsert.h index 31785dc578..c6c46e5409 100644 --- a/src/include/access/xloginsert.h +++ b/src/include/access/xloginsert.h @@ -41,7 +41,8 @@ /* prototypes for public functions in xloginsert.c: */ extern void XLogBeginInsert(void); extern void XLogSetRecordFlags(uint8 flags); -extern XLogRecPtr XLogInsert(RmgrId rmid, uint8 info); +extern XLogRecPtr XLogInsertExtended(RmgrId rmid, uint8 info, uint8 rmgr_info); +extern XLogRecPtr XLogInsert(RmgrId rmid, uint8 rmgr_info); extern void XLogEnsureRecordSpace(int max_block_id, int ndatas); extern void XLogRegisterData(char *data, uint32 len); extern void XLogRegisterBuffer(uint8 block_id, Buffer buffer, uint8 flags); diff --git a/src/include/access/xlogreader.h b/src/include/access/xlogreader.h index 2b976dcc03..a1d0216404 100644 --- a/src/include/access/xlogreader.h +++ b/src/include/access/xlogreader.h @@ -407,8 +407,8 @@ extern bool DecodeXLogRecord(XLogReaderState *state, */ #define XLogRecGetTotalLen(decoder) ((decoder)->record->header.xl_tot_len) #define XLogRecGetPrev(decoder) ((decoder)->record->header.xl_prev) -#define XLogRecGetInfo(decoder) ((decoder)->record->header.xl_info & XLR_INFO_MASK) -#define XLogRecGetRmgrInfo(decoder) (((decoder)->record->header.xl_info) & XLR_RMGR_INFO_MASK) +#define XLogRecGetInfo(decoder) ((decoder)->record->header.xl_info) +#define XLogRecGetRmgrInfo(decoder) ((decoder)->record->header.xl_rmgrinfo) #define XLogRecGetRmid(decoder) ((decoder)->record->header.xl_rmid) #define XLogRecGetXid(decoder) ((decoder)->record->header.xl_xid) #define XLogRecGetOrigin(decoder) ((decoder)->record->record_origin) diff --git a/src/include/access/xlogrecord.h b/src/include/access/xlogrecord.h index b0aada1031..42b06f163e 100644 --- a/src/include/access/xlogrecord.h +++ b/src/include/access/xlogrecord.h @@ -66,7 +66,8 @@ typedef struct XLogRecord XLogRecPtr xl_prev; /* ptr to previous record in log */ uint8 xl_info; /* flag bits, see below */ RmgrId xl_rmid; /* resource manager for this record */ - /* 2 bytes of padding here, initialize to zero */ + uint8 xl_rmgrinfo; /* rmgr flag bits, see below */ + /* 1 byte of padding here, initialize to zero */ pg_crc32c xl_crc; /* CRC for this record */ /* XLogRecordBlockHeaders and XLogRecordDataHeader follow, no padding */ @@ -75,14 +76,6 @@ typedef struct XLogRecord #define SizeOfXLogRecord (offsetof(XLogRecord, xl_crc) + sizeof(pg_crc32c)) -/* - * The high 4 bits in xl_info may be used freely by rmgr. The - * XLR_SPECIAL_REL_UPDATE and XLR_CHECK_CONSISTENCY bits can be passed by - * XLogInsert caller. The rest are set internally by XLogInsert. - */ -#define XLR_INFO_MASK 0x0F -#define XLR_RMGR_INFO_MASK 0xF0 - /* * XLogReader needs to allocate all the data of a WAL record in a single * chunk. This means that a single XLogRecord cannot exceed MaxAllocSize diff --git a/src/include/access/xlogstats.h b/src/include/access/xlogstats.h index 89410ce92b..30974f5ba7 100644 --- a/src/include/access/xlogstats.h +++ b/src/include/access/xlogstats.h @@ -16,7 +16,8 @@ #include "access/rmgr.h" #include "access/xlogreader.h" -#define MAX_XLINFO_TYPES 16 +/* all 8 bits of xl_rmgrinfo are available */ +#define MAX_XLRMGRINFO_TYPES 256 typedef struct XLogRecStats { @@ -33,7 +34,7 @@ typedef struct XLogStats XLogRecPtr endptr; #endif XLogRecStats rmgr_stats[RM_MAX_ID + 1]; - XLogRecStats record_stats[RM_MAX_ID + 1][MAX_XLINFO_TYPES]; + XLogRecStats record_stats[RM_MAX_ID + 1][MAX_XLRMGRINFO_TYPES]; } XLogStats; extern void XLogRecGetLen(XLogReaderState *record, uint32 *rec_len, -- 2.40.1 [application/octet-stream] v2-0003-Rename-rmgr_identify-info-bits-argument-to-rmgrin.patch (21.6K, ../../CAEze2Wj+L1AGNAwjuQFYMcu2be0NYEw1oSg8Sa-HvVbC7iHZLQ@mail.gmail.com/3-v2-0003-Rename-rmgr_identify-info-bits-argument-to-rmgrin.patch) download | inline diff: From 09bfc689208d7aea8d875f4638bdccd6d77c1293 Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Mon, 23 Jan 2023 17:31:18 +0100 Subject: [PATCH v2 3/6] Rename rmgr_identify 'info'-bits argument to rmgrinfo This indicates the provided bits are fully owned by the rmgr, and none of the xlog record's owned bits are shared with the rmgr anymore. --- src/backend/access/rmgrdesc/brindesc.c | 4 ++-- src/backend/access/rmgrdesc/clogdesc.c | 4 ++-- src/backend/access/rmgrdesc/dbasedesc.c | 4 ++-- src/backend/access/rmgrdesc/gindesc.c | 4 ++-- src/backend/access/rmgrdesc/gistdesc.c | 4 ++-- src/backend/access/rmgrdesc/hashdesc.c | 4 ++-- src/backend/access/rmgrdesc/heapdesc.c | 8 ++++---- src/backend/access/rmgrdesc/logicalmsgdesc.c | 4 ++-- src/backend/access/rmgrdesc/mxactdesc.c | 4 ++-- src/backend/access/rmgrdesc/nbtdesc.c | 4 ++-- src/backend/access/rmgrdesc/relmapdesc.c | 4 ++-- src/backend/access/rmgrdesc/seqdesc.c | 4 ++-- src/backend/access/rmgrdesc/smgrdesc.c | 4 ++-- src/backend/access/rmgrdesc/spgdesc.c | 4 ++-- src/backend/access/rmgrdesc/standbydesc.c | 4 ++-- src/backend/access/rmgrdesc/tblspcdesc.c | 4 ++-- src/backend/access/rmgrdesc/xlogdesc.c | 4 ++-- src/include/access/brin_xlog.h | 2 +- src/include/access/clog.h | 2 +- src/include/access/ginxlog.h | 2 +- src/include/access/gistxlog.h | 2 +- src/include/access/hash_xlog.h | 2 +- src/include/access/heapam_xlog.h | 4 ++-- src/include/access/multixact.h | 2 +- src/include/access/nbtxlog.h | 2 +- src/include/access/spgxlog.h | 2 +- src/include/access/xlog.h | 2 +- src/include/catalog/storage_xlog.h | 2 +- src/include/commands/dbcommands_xlog.h | 2 +- src/include/commands/sequence.h | 2 +- src/include/commands/tablespace.h | 2 +- src/include/replication/message.h | 2 +- src/include/storage/standbydefs.h | 2 +- src/include/utils/relmapper.h | 2 +- src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c | 4 ++-- 35 files changed, 56 insertions(+), 56 deletions(-) diff --git a/src/backend/access/rmgrdesc/brindesc.c b/src/backend/access/rmgrdesc/brindesc.c index 1d3e5ee03e..fa272f0720 100644 --- a/src/backend/access/rmgrdesc/brindesc.c +++ b/src/backend/access/rmgrdesc/brindesc.c @@ -71,11 +71,11 @@ brin_desc(StringInfo buf, XLogReaderState *record) } const char * -brin_identify(uint8 info) +brin_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_BRIN_CREATE_INDEX: id = "CREATE_INDEX"; diff --git a/src/backend/access/rmgrdesc/clogdesc.c b/src/backend/access/rmgrdesc/clogdesc.c index 915f9945d0..b368af595e 100644 --- a/src/backend/access/rmgrdesc/clogdesc.c +++ b/src/backend/access/rmgrdesc/clogdesc.c @@ -41,11 +41,11 @@ clog_desc(StringInfo buf, XLogReaderState *record) } const char * -clog_identify(uint8 info) +clog_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case CLOG_ZEROPAGE: id = "ZEROPAGE"; diff --git a/src/backend/access/rmgrdesc/dbasedesc.c b/src/backend/access/rmgrdesc/dbasedesc.c index a9529bde05..a405c0738a 100644 --- a/src/backend/access/rmgrdesc/dbasedesc.c +++ b/src/backend/access/rmgrdesc/dbasedesc.c @@ -54,11 +54,11 @@ dbase_desc(StringInfo buf, XLogReaderState *record) } const char * -dbase_identify(uint8 info) +dbase_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_DBASE_CREATE_FILE_COPY: id = "CREATE_FILE_COPY"; diff --git a/src/backend/access/rmgrdesc/gindesc.c b/src/backend/access/rmgrdesc/gindesc.c index 08fc7ba49b..e7ed7a6094 100644 --- a/src/backend/access/rmgrdesc/gindesc.c +++ b/src/backend/access/rmgrdesc/gindesc.c @@ -179,11 +179,11 @@ gin_desc(StringInfo buf, XLogReaderState *record) } const char * -gin_identify(uint8 info) +gin_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_GIN_CREATE_PTREE: id = "CREATE_PTREE"; diff --git a/src/backend/access/rmgrdesc/gistdesc.c b/src/backend/access/rmgrdesc/gistdesc.c index 6473069c1a..899e12bcc3 100644 --- a/src/backend/access/rmgrdesc/gistdesc.c +++ b/src/backend/access/rmgrdesc/gistdesc.c @@ -86,11 +86,11 @@ gist_desc(StringInfo buf, XLogReaderState *record) } const char * -gist_identify(uint8 info) +gist_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_GIST_PAGE_UPDATE: id = "PAGE_UPDATE"; diff --git a/src/backend/access/rmgrdesc/hashdesc.c b/src/backend/access/rmgrdesc/hashdesc.c index bcb2013048..451251caca 100644 --- a/src/backend/access/rmgrdesc/hashdesc.c +++ b/src/backend/access/rmgrdesc/hashdesc.c @@ -122,11 +122,11 @@ hash_desc(StringInfo buf, XLogReaderState *record) } const char * -hash_identify(uint8 info) +hash_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_HASH_INIT_META_PAGE: id = "INIT_META_PAGE"; diff --git a/src/backend/access/rmgrdesc/heapdesc.c b/src/backend/access/rmgrdesc/heapdesc.c index 15f943051a..5224869af1 100644 --- a/src/backend/access/rmgrdesc/heapdesc.c +++ b/src/backend/access/rmgrdesc/heapdesc.c @@ -302,11 +302,11 @@ heap2_desc(StringInfo buf, XLogReaderState *record) } const char * -heap_identify(uint8 info) +heap_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_HEAP_INSERT: id = "INSERT"; @@ -347,11 +347,11 @@ heap_identify(uint8 info) } const char * -heap2_identify(uint8 info) +heap2_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_HEAP2_PRUNE: id = "PRUNE"; diff --git a/src/backend/access/rmgrdesc/logicalmsgdesc.c b/src/backend/access/rmgrdesc/logicalmsgdesc.c index 2bb6cb06c3..7566630961 100644 --- a/src/backend/access/rmgrdesc/logicalmsgdesc.c +++ b/src/backend/access/rmgrdesc/logicalmsgdesc.c @@ -43,9 +43,9 @@ logicalmsg_desc(StringInfo buf, XLogReaderState *record) } const char * -logicalmsg_identify(uint8 info) +logicalmsg_identify(uint8 rmgrinfo) { - if ((info & ~XLR_INFO_MASK) == XLOG_LOGICAL_MESSAGE) + if (rmgrinfo == XLOG_LOGICAL_MESSAGE) return "MESSAGE"; return NULL; diff --git a/src/backend/access/rmgrdesc/mxactdesc.c b/src/backend/access/rmgrdesc/mxactdesc.c index cdcf8a4f45..7676778239 100644 --- a/src/backend/access/rmgrdesc/mxactdesc.c +++ b/src/backend/access/rmgrdesc/mxactdesc.c @@ -81,11 +81,11 @@ multixact_desc(StringInfo buf, XLogReaderState *record) } const char * -multixact_identify(uint8 info) +multixact_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_MULTIXACT_ZERO_OFF_PAGE: id = "ZERO_OFF_PAGE"; diff --git a/src/backend/access/rmgrdesc/nbtdesc.c b/src/backend/access/rmgrdesc/nbtdesc.c index ede8df72c0..ff9e28ca55 100644 --- a/src/backend/access/rmgrdesc/nbtdesc.c +++ b/src/backend/access/rmgrdesc/nbtdesc.c @@ -134,11 +134,11 @@ btree_desc(StringInfo buf, XLogReaderState *record) } const char * -btree_identify(uint8 info) +btree_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_BTREE_INSERT_LEAF: id = "INSERT_LEAF"; diff --git a/src/backend/access/rmgrdesc/relmapdesc.c b/src/backend/access/rmgrdesc/relmapdesc.c index f4f0e5ce8b..9c9f191700 100644 --- a/src/backend/access/rmgrdesc/relmapdesc.c +++ b/src/backend/access/rmgrdesc/relmapdesc.c @@ -32,11 +32,11 @@ relmap_desc(StringInfo buf, XLogReaderState *record) } const char * -relmap_identify(uint8 info) +relmap_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_RELMAP_UPDATE: id = "UPDATE"; diff --git a/src/backend/access/rmgrdesc/seqdesc.c b/src/backend/access/rmgrdesc/seqdesc.c index 6def7b653b..eed631b5d8 100644 --- a/src/backend/access/rmgrdesc/seqdesc.c +++ b/src/backend/access/rmgrdesc/seqdesc.c @@ -31,11 +31,11 @@ seq_desc(StringInfo buf, XLogReaderState *record) } const char * -seq_identify(uint8 info) +seq_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_SEQ_LOG: id = "LOG"; diff --git a/src/backend/access/rmgrdesc/smgrdesc.c b/src/backend/access/rmgrdesc/smgrdesc.c index 8604bee0c0..653caf58e4 100644 --- a/src/backend/access/rmgrdesc/smgrdesc.c +++ b/src/backend/access/rmgrdesc/smgrdesc.c @@ -43,11 +43,11 @@ smgr_desc(StringInfo buf, XLogReaderState *record) } const char * -smgr_identify(uint8 info) +smgr_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_SMGR_CREATE: id = "CREATE"; diff --git a/src/backend/access/rmgrdesc/spgdesc.c b/src/backend/access/rmgrdesc/spgdesc.c index 9e1e48240b..0d1af9a5b6 100644 --- a/src/backend/access/rmgrdesc/spgdesc.c +++ b/src/backend/access/rmgrdesc/spgdesc.c @@ -128,11 +128,11 @@ spg_desc(StringInfo buf, XLogReaderState *record) } const char * -spg_identify(uint8 info) +spg_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_SPGIST_ADD_LEAF: id = "ADD_LEAF"; diff --git a/src/backend/access/rmgrdesc/standbydesc.c b/src/backend/access/rmgrdesc/standbydesc.c index 9a08e10fe1..56acea1407 100644 --- a/src/backend/access/rmgrdesc/standbydesc.c +++ b/src/backend/access/rmgrdesc/standbydesc.c @@ -76,11 +76,11 @@ standby_desc(StringInfo buf, XLogReaderState *record) } const char * -standby_identify(uint8 info) +standby_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_STANDBY_LOCK: id = "LOCK"; diff --git a/src/backend/access/rmgrdesc/tblspcdesc.c b/src/backend/access/rmgrdesc/tblspcdesc.c index c1feb4fa05..c5849ccf7c 100644 --- a/src/backend/access/rmgrdesc/tblspcdesc.c +++ b/src/backend/access/rmgrdesc/tblspcdesc.c @@ -38,11 +38,11 @@ tblspc_desc(StringInfo buf, XLogReaderState *record) } const char * -tblspc_identify(uint8 info) +tblspc_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_TBLSPC_CREATE: id = "CREATE"; diff --git a/src/backend/access/rmgrdesc/xlogdesc.c b/src/backend/access/rmgrdesc/xlogdesc.c index 3692dfb96a..4db4f3331f 100644 --- a/src/backend/access/rmgrdesc/xlogdesc.c +++ b/src/backend/access/rmgrdesc/xlogdesc.c @@ -151,11 +151,11 @@ xlog_desc(StringInfo buf, XLogReaderState *record) } const char * -xlog_identify(uint8 info) +xlog_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_CHECKPOINT_SHUTDOWN: id = "CHECKPOINT_SHUTDOWN"; diff --git a/src/include/access/brin_xlog.h b/src/include/access/brin_xlog.h index ae263a371f..45a688d09b 100644 --- a/src/include/access/brin_xlog.h +++ b/src/include/access/brin_xlog.h @@ -145,7 +145,7 @@ typedef struct xl_brin_desummarize extern void brin_redo(XLogReaderState *record); extern void brin_desc(StringInfo buf, XLogReaderState *record); -extern const char *brin_identify(uint8 info); +extern const char *brin_identify(uint8 rmgrinfo); extern void brin_mask(char *pagedata, BlockNumber blkno); #endif /* BRIN_XLOG_H */ diff --git a/src/include/access/clog.h b/src/include/access/clog.h index d99444f073..224cad4415 100644 --- a/src/include/access/clog.h +++ b/src/include/access/clog.h @@ -58,6 +58,6 @@ extern int clogsyncfiletag(const FileTag *ftag, char *path); extern void clog_redo(XLogReaderState *record); extern void clog_desc(StringInfo buf, XLogReaderState *record); -extern const char *clog_identify(uint8 info); +extern const char *clog_identify(uint8 rmgrinfo); #endif /* CLOG_H */ diff --git a/src/include/access/ginxlog.h b/src/include/access/ginxlog.h index 37095e5f17..adf1f6b6d8 100644 --- a/src/include/access/ginxlog.h +++ b/src/include/access/ginxlog.h @@ -208,7 +208,7 @@ typedef struct ginxlogDeleteListPages extern void gin_redo(XLogReaderState *record); extern void gin_desc(StringInfo buf, XLogReaderState *record); -extern const char *gin_identify(uint8 info); +extern const char *gin_identify(uint8 rmgrinfo); extern void gin_xlog_startup(void); extern void gin_xlog_cleanup(void); extern void gin_mask(char *pagedata, BlockNumber blkno); diff --git a/src/include/access/gistxlog.h b/src/include/access/gistxlog.h index aff2ffbdcc..0ca37f3a07 100644 --- a/src/include/access/gistxlog.h +++ b/src/include/access/gistxlog.h @@ -109,7 +109,7 @@ typedef struct gistxlogPageReuse extern void gist_redo(XLogReaderState *record); extern void gist_desc(StringInfo buf, XLogReaderState *record); -extern const char *gist_identify(uint8 info); +extern const char *gist_identify(uint8 rmgrinfo); extern void gist_xlog_startup(void); extern void gist_xlog_cleanup(void); extern void gist_mask(char *pagedata, BlockNumber blkno); diff --git a/src/include/access/hash_xlog.h b/src/include/access/hash_xlog.h index b93619d1a8..74859a2a47 100644 --- a/src/include/access/hash_xlog.h +++ b/src/include/access/hash_xlog.h @@ -263,7 +263,7 @@ typedef struct xl_hash_vacuum_one_page extern void hash_redo(XLogReaderState *record); extern void hash_desc(StringInfo buf, XLogReaderState *record); -extern const char *hash_identify(uint8 info); +extern const char *hash_identify(uint8 rmgrinfo); extern void hash_mask(char *pagedata, BlockNumber blkno); #endif /* HASH_XLOG_H */ diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h index a038450787..70e9acd350 100644 --- a/src/include/access/heapam_xlog.h +++ b/src/include/access/heapam_xlog.h @@ -406,11 +406,11 @@ extern void HeapTupleHeaderAdvanceConflictHorizon(HeapTupleHeader tuple, extern void heap_redo(XLogReaderState *record); extern void heap_desc(StringInfo buf, XLogReaderState *record); -extern const char *heap_identify(uint8 info); +extern const char *heap_identify(uint8 rmgrinfo); extern void heap_mask(char *pagedata, BlockNumber blkno); extern void heap2_redo(XLogReaderState *record); extern void heap2_desc(StringInfo buf, XLogReaderState *record); -extern const char *heap2_identify(uint8 info); +extern const char *heap2_identify(uint8 rmgrinfo); extern void heap_xlog_logical_rewrite(XLogReaderState *r); extern XLogRecPtr log_heap_visible(Relation rel, Buffer heap_buffer, diff --git a/src/include/access/multixact.h b/src/include/access/multixact.h index 246f757f6a..d52e69aa05 100644 --- a/src/include/access/multixact.h +++ b/src/include/access/multixact.h @@ -158,7 +158,7 @@ extern void multixact_twophase_postabort(TransactionId xid, uint16 info, extern void multixact_redo(XLogReaderState *record); extern void multixact_desc(StringInfo buf, XLogReaderState *record); -extern const char *multixact_identify(uint8 info); +extern const char *multixact_identify(uint8 rmgrinfo); extern char *mxid_to_string(MultiXactId multi, int nmembers, MultiXactMember *members); diff --git a/src/include/access/nbtxlog.h b/src/include/access/nbtxlog.h index 7dd9fd0206..422ac13d56 100644 --- a/src/include/access/nbtxlog.h +++ b/src/include/access/nbtxlog.h @@ -362,6 +362,6 @@ extern void btree_mask(char *pagedata, BlockNumber blkno); * prototypes for functions in nbtdesc.c */ extern void btree_desc(StringInfo buf, XLogReaderState *record); -extern const char *btree_identify(uint8 info); +extern const char *btree_identify(uint8 rmgrinfo); #endif /* NBTXLOG_H */ diff --git a/src/include/access/spgxlog.h b/src/include/access/spgxlog.h index a7f10bf2d3..3533894baf 100644 --- a/src/include/access/spgxlog.h +++ b/src/include/access/spgxlog.h @@ -251,7 +251,7 @@ typedef struct spgxlogVacuumRedirect extern void spg_redo(XLogReaderState *record); extern void spg_desc(StringInfo buf, XLogReaderState *record); -extern const char *spg_identify(uint8 info); +extern const char *spg_identify(uint8 rmgrinfo); extern void spg_xlog_startup(void); extern void spg_xlog_cleanup(void); extern void spg_mask(char *pagedata, BlockNumber blkno); diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h index 48ca852381..9007f559f3 100644 --- a/src/include/access/xlog.h +++ b/src/include/access/xlog.h @@ -211,7 +211,7 @@ extern void XLogSetReplicationSlotMinimumLSN(XLogRecPtr lsn); extern void xlog_redo(struct XLogReaderState *record); extern void xlog_desc(StringInfo buf, struct XLogReaderState *record); -extern const char *xlog_identify(uint8 info); +extern const char *xlog_identify(uint8 rmgrinfo); extern void issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli); diff --git a/src/include/catalog/storage_xlog.h b/src/include/catalog/storage_xlog.h index 6b0a7aa3df..27514163be 100644 --- a/src/include/catalog/storage_xlog.h +++ b/src/include/catalog/storage_xlog.h @@ -54,6 +54,6 @@ extern void log_smgrcreate(const RelFileLocator *rlocator, ForkNumber forkNum); extern void smgr_redo(XLogReaderState *record); extern void smgr_desc(StringInfo buf, XLogReaderState *record); -extern const char *smgr_identify(uint8 info); +extern const char *smgr_identify(uint8 rmgrinfo); #endif /* STORAGE_XLOG_H */ diff --git a/src/include/commands/dbcommands_xlog.h b/src/include/commands/dbcommands_xlog.h index a1433275f5..f3c11060b6 100644 --- a/src/include/commands/dbcommands_xlog.h +++ b/src/include/commands/dbcommands_xlog.h @@ -55,6 +55,6 @@ typedef struct xl_dbase_drop_rec extern void dbase_redo(XLogReaderState *record); extern void dbase_desc(StringInfo buf, XLogReaderState *record); -extern const char *dbase_identify(uint8 info); +extern const char *dbase_identify(uint8 rmgrinfo); #endif /* DBCOMMANDS_XLOG_H */ diff --git a/src/include/commands/sequence.h b/src/include/commands/sequence.h index 7db7b3da7b..fb7059b8f6 100644 --- a/src/include/commands/sequence.h +++ b/src/include/commands/sequence.h @@ -64,7 +64,7 @@ extern void ResetSequenceCaches(void); extern void seq_redo(XLogReaderState *record); extern void seq_desc(StringInfo buf, XLogReaderState *record); -extern const char *seq_identify(uint8 info); +extern const char *seq_identify(uint8 rmgrinfo); extern void seq_mask(char *page, BlockNumber blkno); #endif /* SEQUENCE_H */ diff --git a/src/include/commands/tablespace.h b/src/include/commands/tablespace.h index f1961c1813..db21eb59ee 100644 --- a/src/include/commands/tablespace.h +++ b/src/include/commands/tablespace.h @@ -64,6 +64,6 @@ extern void remove_tablespace_symlink(const char *linkloc); extern void tblspc_redo(XLogReaderState *record); extern void tblspc_desc(StringInfo buf, XLogReaderState *record); -extern const char *tblspc_identify(uint8 info); +extern const char *tblspc_identify(uint8 rmgrinfo); #endif /* TABLESPACE_H */ diff --git a/src/include/replication/message.h b/src/include/replication/message.h index 6ce7f2038b..2fe985a6fe 100644 --- a/src/include/replication/message.h +++ b/src/include/replication/message.h @@ -36,6 +36,6 @@ extern XLogRecPtr LogLogicalMessage(const char *prefix, const char *message, #define XLOG_LOGICAL_MESSAGE 0x00 extern void logicalmsg_redo(XLogReaderState *record); extern void logicalmsg_desc(StringInfo buf, XLogReaderState *record); -extern const char *logicalmsg_identify(uint8 info); +extern const char *logicalmsg_identify(uint8 rmgrinfo); #endif /* PG_LOGICAL_MESSAGE_H */ diff --git a/src/include/storage/standbydefs.h b/src/include/storage/standbydefs.h index 188e348618..84e1f03c8f 100644 --- a/src/include/storage/standbydefs.h +++ b/src/include/storage/standbydefs.h @@ -22,7 +22,7 @@ /* Recovery handlers for the Standby Rmgr (RM_STANDBY_ID) */ extern void standby_redo(XLogReaderState *record); extern void standby_desc(StringInfo buf, XLogReaderState *record); -extern const char *standby_identify(uint8 info); +extern const char *standby_identify(uint8 rmgrinfo); extern void standby_desc_invalidations(StringInfo buf, int nmsgs, SharedInvalidationMessage *msgs, Oid dbId, Oid tsId, diff --git a/src/include/utils/relmapper.h b/src/include/utils/relmapper.h index 5c173bdbb3..e3c41ff093 100644 --- a/src/include/utils/relmapper.h +++ b/src/include/utils/relmapper.h @@ -68,6 +68,6 @@ extern void RestoreRelationMap(char *startAddress); extern void relmap_redo(XLogReaderState *record); extern void relmap_desc(StringInfo buf, XLogReaderState *record); -extern const char *relmap_identify(uint8 info); +extern const char *relmap_identify(uint8 rmgrinfo); #endif /* RELMAPPER_H */ diff --git a/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c b/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c index b5537a5f5b..21514f8e33 100644 --- a/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c +++ b/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c @@ -103,9 +103,9 @@ testcustomrmgrs_desc(StringInfo buf, XLogReaderState *record) } const char * -testcustomrmgrs_identify(uint8 info) +testcustomrmgrs_identify(uint8 rmgrinfo) { - if ((info & ~XLR_INFO_MASK) == XLOG_TEST_CUSTOM_RMGRS_MESSAGE) + if (rmgrinfo == XLOG_TEST_CUSTOM_RMGRS_MESSAGE) return "TEST_CUSTOM_RMGRS_MESSAGE"; return NULL; -- 2.40.1 [application/octet-stream] v2-0001-Reduce-overhead-of-small-block-data-in-xlog-recor.patch (10.9K, ../../CAEze2Wj+L1AGNAwjuQFYMcu2be0NYEw1oSg8Sa-HvVbC7iHZLQ@mail.gmail.com/4-v2-0001-Reduce-overhead-of-small-block-data-in-xlog-recor.patch) download | inline diff: From 459dc36fde0dd0b3516d4d544aac6cd45a7a0458 Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Tue, 24 Jan 2023 12:37:09 +0100 Subject: [PATCH v2 1/6] Reduce overhead of small block data in xlog records We reserve 2 bits in the block ID header for how large the block's registered data is: 0, 1 or 2 bytes, for 0, 1-255 and 256-UINT16_MAX bytes to the block, respectively. This reduces the minimum size of any block-referencing xlog record from 48 bytes to 46 bytes, and reduces the size of any non-hot heap update record by at least 2 bytes. It includes more infrastructure for future value-dependent encoding of length fields, even though it is currently only used in one place. --- src/backend/access/transam/xloginsert.c | 13 ++- src/backend/access/transam/xlogreader.c | 34 +++++-- src/include/access/xlog_internal.h | 114 ++++++++++++++++++++++++ src/include/access/xlogrecord.h | 43 +++++++-- 4 files changed, 188 insertions(+), 16 deletions(-) diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c index 54247e1d81..0fce063ca4 100644 --- a/src/backend/access/transam/xloginsert.c +++ b/src/backend/access/transam/xloginsert.c @@ -581,6 +581,8 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, XLogRecordBlockHeader bkpb; XLogRecordBlockImageHeader bimg; XLogRecordBlockCompressHeader cbimg = {0}; + int data_length = 0; + XLogSizeClass data_sizeclass = XLS_EMPTY; bool samerel; bool is_compressed = false; bool include_image; @@ -622,7 +624,7 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, bkpb.id = block_id; bkpb.fork_flags = regbuf->forkno; - bkpb.data_length = 0; + data_length = 0; if ((regbuf->flags & REGBUF_WILL_INIT) == REGBUF_WILL_INIT) bkpb.fork_flags |= BKPBLOCK_WILL_INIT; @@ -785,13 +787,16 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, * overall list. */ bkpb.fork_flags |= BKPBLOCK_HAS_DATA; - bkpb.data_length = (uint16) regbuf->rdata_len; + data_length = (uint16) regbuf->rdata_len; + data_sizeclass = XLogLengthToSizeClass(data_length, XLS_UINT16); total_len += regbuf->rdata_len; rdt_datas_last->next = regbuf->rdata_head; rdt_datas_last = regbuf->rdata_tail; } + bkpb.id |= data_sizeclass << XLR_BLOCKID_SZCLASS_SHIFT; + if (prev_regbuf && RelFileLocatorEquals(regbuf->rlocator, prev_regbuf->rlocator)) { samerel = true; @@ -804,6 +809,10 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, /* Ok, copy the header to the scratch buffer */ memcpy(scratch, &bkpb, SizeOfXLogRecordBlockHeader); scratch += SizeOfXLogRecordBlockHeader; + + scratch += XLogWriteLength(data_length, data_sizeclass, + XLS_UINT16, scratch); + if (include_image) { memcpy(scratch, &bimg, SizeOfXLogRecordBlockImageHeader); diff --git a/src/backend/access/transam/xlogreader.c b/src/backend/access/transam/xlogreader.c index 2e7b1ba8e1..24c6c88b2f 100644 --- a/src/backend/access/transam/xlogreader.c +++ b/src/backend/access/transam/xlogreader.c @@ -1660,11 +1660,11 @@ DecodeXLogRecord(XLogReaderState *state, */ #define COPY_HEADER_FIELD(_dst, _size) \ do { \ - if (remaining < _size) \ + if (remaining < (_size)) \ goto shortdata_err; \ - memcpy(_dst, ptr, _size); \ - ptr += _size; \ - remaining -= _size; \ + memcpy((_dst), ptr, (_size)); \ + ptr += (_size); \ + remaining -= (_size); \ } while(0) char *ptr; @@ -1690,8 +1690,13 @@ DecodeXLogRecord(XLogReaderState *state, datatotal = 0; while (remaining > datatotal) { + XLogSizeClass sizeClass; COPY_HEADER_FIELD(&block_id, sizeof(uint8)); + sizeClass = (block_id & XLR_BLOCKID_SZCLASS_MASK) >> + XLR_BLOCKID_SZCLASS_SHIFT; + block_id &= XLR_BLOCK_ID_MASK; + if (block_id == XLR_BLOCK_ID_DATA_SHORT) { /* XLogRecordDataHeaderShort */ @@ -1755,7 +1760,26 @@ DecodeXLogRecord(XLogReaderState *state, blk->prefetch_buffer = InvalidBuffer; - COPY_HEADER_FIELD(&blk->data_len, sizeof(uint16)); + { + Size read; + uint32 length = 0; + read = XLogReadLength(&length, sizeClass, + XLS_UINT16, + ptr, remaining); + + if (read < 0) + { + report_invalid_record(state, + "Could not read length from record at %X/%X", + LSN_FORMAT_ARGS(state->ReadRecPtr)); + goto err; + } + + ptr += read; + remaining -= read; + blk->data_len = length; + } + /* cross-check that the HAS_DATA flag is set iff data_length > 0 */ if (blk->has_data && blk->data_len == 0) { diff --git a/src/include/access/xlog_internal.h b/src/include/access/xlog_internal.h index b0fd338a00..42b3c66547 100644 --- a/src/include/access/xlog_internal.h +++ b/src/include/access/xlog_internal.h @@ -328,6 +328,120 @@ typedef enum struct LogicalDecodingContext; struct XLogRecordBuffer; +/* + * XLogSizeClass interactions + */ + +/* + * XLogLengthToSizeClass + * Turns a length into a size class + * MaxSizeClass is used to allow the compiler to eliminate branches that + * can never be hit in the caller's code path; e.g. when processing uint16. + */ +static inline XLogSizeClass XLogLengthToSizeClass(uint32 length, + const XLogSizeClass maxSizeClass) +{ + XLogSizeClass sizeClass; + + if (length == 0) + sizeClass = XLS_EMPTY; + else if (length <= UINT8_MAX && maxSizeClass >= XLS_UINT8) + sizeClass = XLS_UINT8; + else if (length <= UINT16_MAX && maxSizeClass >= XLS_UINT16) + sizeClass = XLS_UINT16; + else + { + Assert(maxSizeClass == XLS_UINT32); + sizeClass = XLS_UINT32; + } + + Assert(sizeClass <= maxSizeClass); + + return sizeClass; +} + +/* + * XLogWriteLength + * + * Write a length with size determined by sizeClass into the output. + * Returns the Size of bytes written. The user is responsible for making + * sure that out-of-bounds write is impossible. + * Writes at most 4 bytes. + */ +static inline Size XLogWriteLength(uint32 length, XLogSizeClass sizeClass, + const XLogSizeClass maxSizeClass, + char *output) +{ + Size written = -1; + + Assert(sizeClass <= maxSizeClass && + XLogLengthToSizeClass(length, maxSizeClass) == sizeClass); + +#define WRITE_OP(caseSizeClass, field_type) \ + case caseSizeClass: \ + if ((caseSizeClass) <= maxSizeClass) \ + { \ + field_type typedLength = length; \ + memcpy(output, &typedLength, sizeof(field_type)); \ + written = sizeof(field_type); \ + } \ + break + + switch (sizeClass) { + case XLS_EMPTY: + written = 0; + break; + WRITE_OP(XLS_UINT8, uint8); + WRITE_OP(XLS_UINT16, uint16); + WRITE_OP(XLS_UINT32, uint32); + } + +#undef WRITE_OP + return written; +} + +/* + * XLogReadLength + * + * Read a length with size determined by sizeClass from the input into + * *length. Returns the Size of bytes read, or -1 if the input size was + * too small to read the relevant length field. + */ +static inline Size XLogReadLength(uint32 *length, XLogSizeClass sizeClass, + const XLogSizeClass maxSizeClass, + char *input, Size inSize) +{ + Size readSize = -1; + + Assert(sizeClass <= maxSizeClass); + +#define READ_OP(caseSizeClass, field_type) \ + case caseSizeClass: \ + if ((caseSizeClass) <= maxSizeClass) \ + { \ + field_type typedLength; \ + if (inSize < sizeof(field_type)) \ + return -1; \ + memcpy(&typedLength, input, sizeof(field_type)); \ + readSize = sizeof(field_type); \ + *length = typedLength; \ + } \ + break + + switch (sizeClass) { + case XLS_EMPTY: + readSize = 0; + *length = 0; + break; + READ_OP(XLS_UINT8, uint8); + READ_OP(XLS_UINT16, uint16); + READ_OP(XLS_UINT32, uint32); + } + +#undef READ_OP + return readSize; +} + /* * Method table for resource managers. * diff --git a/src/include/access/xlogrecord.h b/src/include/access/xlogrecord.h index f355e08e1d..b0aada1031 100644 --- a/src/include/access/xlogrecord.h +++ b/src/include/access/xlogrecord.h @@ -17,6 +17,27 @@ #include "storage/block.h" #include "storage/relfilelocator.h" +/* + * XLogSizeClass + * + * XLog data segments registered to e.g. blocks are often quite small. + * This XLogSizeClass infrastructure allows us to do variable length encoding + * on these fields, so that we may save some bytes in each record on length + * fields. + * + * Note that it is possible to encode smaller sizes in the larger size classes, + * but the reverse is not true. For maximum efficiency, it is important to only + * use + * + * Values > UINT32_MAX are not supported in this scheme. + */ +typedef enum XLogSizeClass { + XLS_EMPTY = 0, /* length == 0 */ + XLS_UINT8 = 1, /* length <= UINT8_MAX; stored in uint8 (1B) */ + XLS_UINT16 = 2, /* length <= UINT16_MAX; stored in uint16 (2B) */ + XLS_UINT32 = 3 /* length <= UINT32_MAX; stored in uint32 (4B) */ +} XLogSizeClass; + /* * The overall layout of an XLOG record is: * Fixed-size header (XLogRecord struct) @@ -104,15 +125,14 @@ typedef struct XLogRecordBlockHeader { uint8 id; /* block reference ID */ uint8 fork_flags; /* fork within the relation, and flags */ - uint16 data_length; /* number of payload bytes (not including page - * image) */ + /* Depending on XLR_BLOCKID_SZCLASS; 0, 1 or 2 bytes of data_length follow */ /* If BKPBLOCK_HAS_IMAGE, an XLogRecordBlockImageHeader struct follows */ /* If BKPBLOCK_SAME_REL is not set, a RelFileLocator follows */ /* BlockNumber follows */ } XLogRecordBlockHeader; -#define SizeOfXLogRecordBlockHeader (offsetof(XLogRecordBlockHeader, data_length) + sizeof(uint16)) +#define SizeOfXLogRecordBlockHeader (offsetof(XLogRecordBlockHeader, fork_flags) + sizeof(uint8)) /* * Additional header information when a full-page image is included @@ -226,6 +246,11 @@ typedef struct XLogRecordDataHeaderLong #define SizeOfXLogRecordDataHeaderLong (sizeof(uint8) + sizeof(uint32)) +#define XLR_BLOCK_ID_MASK 0x3F + +#define XLR_BLOCKID_SZCLASS_SHIFT 6 +#define XLR_BLOCKID_SZCLASS_MASK (3 << XLR_BLOCKID_SZCLASS_SHIFT) + /* * Block IDs used to distinguish different kinds of record fragments. Block * references are numbered from 0 to XLR_MAX_BLOCK_ID. A rmgr is free to use @@ -234,15 +259,15 @@ typedef struct XLogRecordDataHeaderLong * numbers are reserved to denote the "main" data portion of the record, * as well as replication-supporting transaction metadata. * - * The maximum is currently set at 32, quite arbitrarily. Most records only + * The maximum is currently set at 0x20 (32), quite arbitrarily. Most records only * need a handful of block references, but there are a few exceptions that * need more. */ -#define XLR_MAX_BLOCK_ID 32 +#define XLR_MAX_BLOCK_ID 0x20 -#define XLR_BLOCK_ID_DATA_SHORT 255 -#define XLR_BLOCK_ID_DATA_LONG 254 -#define XLR_BLOCK_ID_ORIGIN 253 -#define XLR_BLOCK_ID_TOPLEVEL_XID 252 +#define XLR_BLOCK_ID_DATA_SHORT 0x3F +#define XLR_BLOCK_ID_DATA_LONG 0x3E +#define XLR_BLOCK_ID_ORIGIN 0x3D +#define XLR_BLOCK_ID_TOPLEVEL_XID 0x3C #endif /* XLOGRECORD_H */ -- 2.40.1 [application/octet-stream] v2-0002-Replace-accesses-to-rmgr-s-XLogRecord-info-bit-ac.patch (44.3K, ../../CAEze2Wj+L1AGNAwjuQFYMcu2be0NYEw1oSg8Sa-HvVbC7iHZLQ@mail.gmail.com/5-v2-0002-Replace-accesses-to-rmgr-s-XLogRecord-info-bit-ac.patch) download | inline diff: From c8b3e45614e9227572ae981ee0fb260c8f31005d Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Mon, 23 Jan 2023 16:24:01 +0100 Subject: [PATCH v2 2/6] Replace accesses to rmgr's XLogRecord info bit accesses with XLogRecGetRmgrInfo This allows refactoring of the info bits into its own struct field, instead of sharing the field with the records own info bits. --- contrib/pg_walinspect/pg_walinspect.c | 8 ++--- src/backend/access/brin/brin_xlog.c | 4 +-- src/backend/access/gin/ginxlog.c | 2 +- src/backend/access/gist/gistxlog.c | 2 +- src/backend/access/hash/hash_xlog.c | 2 +- src/backend/access/heap/heapam.c | 10 +++---- src/backend/access/nbtree/nbtxlog.c | 2 +- src/backend/access/rmgrdesc/brindesc.c | 2 +- src/backend/access/rmgrdesc/clogdesc.c | 2 +- src/backend/access/rmgrdesc/committsdesc.c | 2 +- src/backend/access/rmgrdesc/dbasedesc.c | 2 +- src/backend/access/rmgrdesc/gindesc.c | 2 +- src/backend/access/rmgrdesc/gistdesc.c | 2 +- src/backend/access/rmgrdesc/hashdesc.c | 2 +- src/backend/access/rmgrdesc/heapdesc.c | 6 ++-- src/backend/access/rmgrdesc/logicalmsgdesc.c | 2 +- src/backend/access/rmgrdesc/mxactdesc.c | 2 +- src/backend/access/rmgrdesc/nbtdesc.c | 2 +- src/backend/access/rmgrdesc/relmapdesc.c | 2 +- src/backend/access/rmgrdesc/replorigindesc.c | 2 +- src/backend/access/rmgrdesc/seqdesc.c | 2 +- src/backend/access/rmgrdesc/smgrdesc.c | 2 +- src/backend/access/rmgrdesc/spgdesc.c | 2 +- src/backend/access/rmgrdesc/standbydesc.c | 2 +- src/backend/access/rmgrdesc/tblspcdesc.c | 2 +- src/backend/access/rmgrdesc/xactdesc.c | 30 +++++++++---------- src/backend/access/rmgrdesc/xlogdesc.c | 2 +- src/backend/access/spgist/spgxlog.c | 2 +- src/backend/access/transam/clog.c | 2 +- src/backend/access/transam/commit_ts.c | 2 +- src/backend/access/transam/multixact.c | 2 +- src/backend/access/transam/twophase.c | 2 +- src/backend/access/transam/xact.c | 10 +++---- src/backend/access/transam/xlog.c | 2 +- src/backend/access/transam/xlogrecovery.c | 22 +++++++------- src/backend/access/transam/xlogstats.c | 2 +- src/backend/catalog/storage.c | 2 +- src/backend/commands/dbcommands.c | 2 +- src/backend/commands/sequence.c | 2 +- src/backend/commands/tablespace.c | 2 +- src/backend/replication/logical/decode.c | 20 +++++++------ src/backend/replication/logical/message.c | 2 +- src/backend/replication/logical/origin.c | 2 +- src/backend/storage/ipc/standby.c | 2 +- src/backend/utils/cache/relmapper.c | 2 +- src/bin/pg_rewind/parsexlog.c | 6 ++-- src/bin/pg_waldump/pg_waldump.c | 4 +-- src/include/access/xact.h | 6 ++-- src/include/access/xlogreader.h | 3 +- .../test_custom_rmgrs/test_custom_rmgrs.c | 4 +-- 50 files changed, 105 insertions(+), 102 deletions(-) diff --git a/contrib/pg_walinspect/pg_walinspect.c b/contrib/pg_walinspect/pg_walinspect.c index 796a74f322..1b6f4cb178 100644 --- a/contrib/pg_walinspect/pg_walinspect.c +++ b/contrib/pg_walinspect/pg_walinspect.c @@ -196,10 +196,10 @@ GetWALRecordInfo(XLogReaderState *record, Datum *values, int i = 0; desc = GetRmgr(XLogRecGetRmid(record)); - record_type = desc.rm_identify(XLogRecGetInfo(record)); + record_type = desc.rm_identify(XLogRecGetRmgrInfo(record)); if (record_type == NULL) - record_type = psprintf("UNKNOWN (%x)", XLogRecGetInfo(record) & ~XLR_INFO_MASK); + record_type = psprintf("UNKNOWN (%x)", XLogRecGetRmgrInfo(record)); initStringInfo(&rec_desc); desc.rm_desc(&rec_desc, record); @@ -258,11 +258,11 @@ GetWALBlockInfo(FunctionCallInfo fcinfo, XLogReaderState *record, Assert(XLogRecHasAnyBlockRefs(record)); desc = GetRmgr(XLogRecGetRmid(record)); - record_type = desc.rm_identify(XLogRecGetInfo(record)); + record_type = desc.rm_identify(XLogRecGetRmgrInfo(record)); if (record_type == NULL) record_type = psprintf("UNKNOWN (%x)", - XLogRecGetInfo(record) & ~XLR_INFO_MASK); + XLogRecGetRmgrInfo(record)); initStringInfo(&rec_desc); desc.rm_desc(&rec_desc, record); diff --git a/src/backend/access/brin/brin_xlog.c b/src/backend/access/brin/brin_xlog.c index 89145b68f6..00dd85bdc5 100644 --- a/src/backend/access/brin/brin_xlog.c +++ b/src/backend/access/brin/brin_xlog.c @@ -56,7 +56,7 @@ brin_xlog_insert_update(XLogReaderState *record, * If we inserted the first and only tuple on the page, re-initialize the * page from scratch. */ - if (XLogRecGetInfo(record) & XLOG_BRIN_INIT_PAGE) + if (XLogRecGetRmgrInfo(record) & XLOG_BRIN_INIT_PAGE) { buffer = XLogInitBufferForRedo(record, 0); page = BufferGetPage(buffer); @@ -308,7 +308,7 @@ brin_xlog_desummarize_page(XLogReaderState *record) void brin_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info & XLOG_BRIN_OPMASK) { diff --git a/src/backend/access/gin/ginxlog.c b/src/backend/access/gin/ginxlog.c index f7c84beef8..feb675ed19 100644 --- a/src/backend/access/gin/ginxlog.c +++ b/src/backend/access/gin/ginxlog.c @@ -725,7 +725,7 @@ ginRedoDeleteListPages(XLogReaderState *record) void gin_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); MemoryContext oldCtx; /* diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index 15249aa921..a49210b64f 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -398,7 +398,7 @@ gistRedoPageReuse(XLogReaderState *record) void gist_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); MemoryContext oldCxt; /* diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index e8e06c62a9..3a45b7ee21 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -1048,7 +1048,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) void hash_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 7ed72abe59..1512f22c76 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -9255,7 +9255,7 @@ heap_xlog_insert(XLogReaderState *record) * If we inserted the first and only tuple on the page, re-initialize the * page from scratch. */ - if (XLogRecGetInfo(record) & XLOG_HEAP_INIT_PAGE) + if (XLogRecGetRmgrInfo(record) & XLOG_HEAP_INIT_PAGE) { buffer = XLogInitBufferForRedo(record, 0); page = BufferGetPage(buffer); @@ -9349,7 +9349,7 @@ heap_xlog_multi_insert(XLogReaderState *record) uint32 newlen; Size freespace = 0; int i; - bool isinit = (XLogRecGetInfo(record) & XLOG_HEAP_INIT_PAGE) != 0; + bool isinit = (XLogRecGetRmgrInfo(record) & XLOG_HEAP_INIT_PAGE) != 0; XLogRedoAction action; /* @@ -9597,7 +9597,7 @@ heap_xlog_update(XLogReaderState *record, bool hot_update) nbuffer = obuffer; newaction = oldaction; } - else if (XLogRecGetInfo(record) & XLOG_HEAP_INIT_PAGE) + else if (XLogRecGetRmgrInfo(record) & XLOG_HEAP_INIT_PAGE) { nbuffer = XLogInitBufferForRedo(record, 0); page = (Page) BufferGetPage(nbuffer); @@ -9961,7 +9961,7 @@ heap_xlog_inplace(XLogReaderState *record) void heap_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* * These operations don't overwrite MVCC data so no conflict processing is @@ -10007,7 +10007,7 @@ heap_redo(XLogReaderState *record) void heap2_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info & XLOG_HEAP_OPMASK) { diff --git a/src/backend/access/nbtree/nbtxlog.c b/src/backend/access/nbtree/nbtxlog.c index c87e46ed66..97d8d068c6 100644 --- a/src/backend/access/nbtree/nbtxlog.c +++ b/src/backend/access/nbtree/nbtxlog.c @@ -1015,7 +1015,7 @@ btree_xlog_reuse_page(XLogReaderState *record) void btree_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); MemoryContext oldCtx; oldCtx = MemoryContextSwitchTo(opCtx); diff --git a/src/backend/access/rmgrdesc/brindesc.c b/src/backend/access/rmgrdesc/brindesc.c index 1466a31bbb..1d3e5ee03e 100644 --- a/src/backend/access/rmgrdesc/brindesc.c +++ b/src/backend/access/rmgrdesc/brindesc.c @@ -20,7 +20,7 @@ void brin_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); info &= XLOG_BRIN_OPMASK; if (info == XLOG_BRIN_CREATE_INDEX) diff --git a/src/backend/access/rmgrdesc/clogdesc.c b/src/backend/access/rmgrdesc/clogdesc.c index e60b76f9da..915f9945d0 100644 --- a/src/backend/access/rmgrdesc/clogdesc.c +++ b/src/backend/access/rmgrdesc/clogdesc.c @@ -21,7 +21,7 @@ void clog_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == CLOG_ZEROPAGE) { diff --git a/src/backend/access/rmgrdesc/committsdesc.c b/src/backend/access/rmgrdesc/committsdesc.c index e7155cd507..3d673c2302 100644 --- a/src/backend/access/rmgrdesc/committsdesc.c +++ b/src/backend/access/rmgrdesc/committsdesc.c @@ -22,7 +22,7 @@ void commit_ts_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == COMMIT_TS_ZEROPAGE) { diff --git a/src/backend/access/rmgrdesc/dbasedesc.c b/src/backend/access/rmgrdesc/dbasedesc.c index 3922120d64..a9529bde05 100644 --- a/src/backend/access/rmgrdesc/dbasedesc.c +++ b/src/backend/access/rmgrdesc/dbasedesc.c @@ -22,7 +22,7 @@ void dbase_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_DBASE_CREATE_FILE_COPY) { diff --git a/src/backend/access/rmgrdesc/gindesc.c b/src/backend/access/rmgrdesc/gindesc.c index 246a6a6b85..08fc7ba49b 100644 --- a/src/backend/access/rmgrdesc/gindesc.c +++ b/src/backend/access/rmgrdesc/gindesc.c @@ -74,7 +74,7 @@ void gin_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/access/rmgrdesc/gistdesc.c b/src/backend/access/rmgrdesc/gistdesc.c index 5dc6e1dcee..6473069c1a 100644 --- a/src/backend/access/rmgrdesc/gistdesc.c +++ b/src/backend/access/rmgrdesc/gistdesc.c @@ -60,7 +60,7 @@ void gist_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/access/rmgrdesc/hashdesc.c b/src/backend/access/rmgrdesc/hashdesc.c index b6810a9320..bcb2013048 100644 --- a/src/backend/access/rmgrdesc/hashdesc.c +++ b/src/backend/access/rmgrdesc/hashdesc.c @@ -20,7 +20,7 @@ void hash_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/access/rmgrdesc/heapdesc.c b/src/backend/access/rmgrdesc/heapdesc.c index d73248abdd..15f943051a 100644 --- a/src/backend/access/rmgrdesc/heapdesc.c +++ b/src/backend/access/rmgrdesc/heapdesc.c @@ -95,7 +95,7 @@ void heap_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); info &= XLOG_HEAP_OPMASK; if (info == XLOG_HEAP_INSERT) @@ -172,7 +172,7 @@ void heap2_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); info &= XLOG_HEAP_OPMASK; if (info == XLOG_HEAP2_PRUNE) @@ -265,7 +265,7 @@ heap2_desc(StringInfo buf, XLogReaderState *record) else if (info == XLOG_HEAP2_MULTI_INSERT) { xl_heap_multi_insert *xlrec = (xl_heap_multi_insert *) rec; - bool isinit = (XLogRecGetInfo(record) & XLOG_HEAP_INIT_PAGE) != 0; + bool isinit = (XLogRecGetRmgrInfo(record) & XLOG_HEAP_INIT_PAGE) != 0; appendStringInfo(buf, "ntuples: %d, flags: 0x%02X", xlrec->ntuples, xlrec->flags); diff --git a/src/backend/access/rmgrdesc/logicalmsgdesc.c b/src/backend/access/rmgrdesc/logicalmsgdesc.c index 0578ec5535..2bb6cb06c3 100644 --- a/src/backend/access/rmgrdesc/logicalmsgdesc.c +++ b/src/backend/access/rmgrdesc/logicalmsgdesc.c @@ -19,7 +19,7 @@ void logicalmsg_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_LOGICAL_MESSAGE) { diff --git a/src/backend/access/rmgrdesc/mxactdesc.c b/src/backend/access/rmgrdesc/mxactdesc.c index a2fa1eca18..cdcf8a4f45 100644 --- a/src/backend/access/rmgrdesc/mxactdesc.c +++ b/src/backend/access/rmgrdesc/mxactdesc.c @@ -50,7 +50,7 @@ void multixact_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_MULTIXACT_ZERO_OFF_PAGE || info == XLOG_MULTIXACT_ZERO_MEM_PAGE) diff --git a/src/backend/access/rmgrdesc/nbtdesc.c b/src/backend/access/rmgrdesc/nbtdesc.c index e4fbaa4d5d..ede8df72c0 100644 --- a/src/backend/access/rmgrdesc/nbtdesc.c +++ b/src/backend/access/rmgrdesc/nbtdesc.c @@ -24,7 +24,7 @@ void btree_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/access/rmgrdesc/relmapdesc.c b/src/backend/access/rmgrdesc/relmapdesc.c index 82ad51eb9a..f4f0e5ce8b 100644 --- a/src/backend/access/rmgrdesc/relmapdesc.c +++ b/src/backend/access/rmgrdesc/relmapdesc.c @@ -20,7 +20,7 @@ void relmap_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_RELMAP_UPDATE) { diff --git a/src/backend/access/rmgrdesc/replorigindesc.c b/src/backend/access/rmgrdesc/replorigindesc.c index 8ce4588fda..539dc57939 100644 --- a/src/backend/access/rmgrdesc/replorigindesc.c +++ b/src/backend/access/rmgrdesc/replorigindesc.c @@ -19,7 +19,7 @@ void replorigin_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/access/rmgrdesc/seqdesc.c b/src/backend/access/rmgrdesc/seqdesc.c index ba60544085..6def7b653b 100644 --- a/src/backend/access/rmgrdesc/seqdesc.c +++ b/src/backend/access/rmgrdesc/seqdesc.c @@ -21,7 +21,7 @@ void seq_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); xl_seq_rec *xlrec = (xl_seq_rec *) rec; if (info == XLOG_SEQ_LOG) diff --git a/src/backend/access/rmgrdesc/smgrdesc.c b/src/backend/access/rmgrdesc/smgrdesc.c index bd841b96e8..8604bee0c0 100644 --- a/src/backend/access/rmgrdesc/smgrdesc.c +++ b/src/backend/access/rmgrdesc/smgrdesc.c @@ -21,7 +21,7 @@ void smgr_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_SMGR_CREATE) { diff --git a/src/backend/access/rmgrdesc/spgdesc.c b/src/backend/access/rmgrdesc/spgdesc.c index 87f62f0fb4..9e1e48240b 100644 --- a/src/backend/access/rmgrdesc/spgdesc.c +++ b/src/backend/access/rmgrdesc/spgdesc.c @@ -20,7 +20,7 @@ void spg_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/access/rmgrdesc/standbydesc.c b/src/backend/access/rmgrdesc/standbydesc.c index f2bce9a37a..9a08e10fe1 100644 --- a/src/backend/access/rmgrdesc/standbydesc.c +++ b/src/backend/access/rmgrdesc/standbydesc.c @@ -47,7 +47,7 @@ void standby_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_STANDBY_LOCK) { diff --git a/src/backend/access/rmgrdesc/tblspcdesc.c b/src/backend/access/rmgrdesc/tblspcdesc.c index b8c89f8c54..c1feb4fa05 100644 --- a/src/backend/access/rmgrdesc/tblspcdesc.c +++ b/src/backend/access/rmgrdesc/tblspcdesc.c @@ -21,7 +21,7 @@ void tblspc_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_TBLSPC_CREATE) { diff --git a/src/backend/access/rmgrdesc/xactdesc.c b/src/backend/access/rmgrdesc/xactdesc.c index 01610c5ddc..8903bb32de 100644 --- a/src/backend/access/rmgrdesc/xactdesc.c +++ b/src/backend/access/rmgrdesc/xactdesc.c @@ -32,7 +32,7 @@ */ void -ParseCommitRecord(uint8 info, xl_xact_commit *xlrec, xl_xact_parsed_commit *parsed) +ParseCommitRecord(uint8 rmgrinfo, xl_xact_commit *xlrec, xl_xact_parsed_commit *parsed) { char *data = ((char *) xlrec) + MinSizeOfXactCommit; @@ -43,7 +43,7 @@ ParseCommitRecord(uint8 info, xl_xact_commit *xlrec, xl_xact_parsed_commit *pars parsed->xact_time = xlrec->xact_time; - if (info & XLOG_XACT_HAS_INFO) + if (rmgrinfo & XLOG_XACT_HAS_INFO) { xl_xact_xinfo *xl_xinfo = (xl_xact_xinfo *) data; @@ -138,7 +138,7 @@ ParseCommitRecord(uint8 info, xl_xact_commit *xlrec, xl_xact_parsed_commit *pars } void -ParseAbortRecord(uint8 info, xl_xact_abort *xlrec, xl_xact_parsed_abort *parsed) +ParseAbortRecord(uint8 rmgrinfo, xl_xact_abort *xlrec, xl_xact_parsed_abort *parsed) { char *data = ((char *) xlrec) + MinSizeOfXactAbort; @@ -149,7 +149,7 @@ ParseAbortRecord(uint8 info, xl_xact_abort *xlrec, xl_xact_parsed_abort *parsed) parsed->xact_time = xlrec->xact_time; - if (info & XLOG_XACT_HAS_INFO) + if (rmgrinfo & XLOG_XACT_HAS_INFO) { xl_xact_xinfo *xl_xinfo = (xl_xact_xinfo *) data; @@ -236,7 +236,7 @@ ParseAbortRecord(uint8 info, xl_xact_abort *xlrec, xl_xact_parsed_abort *parsed) * ParsePrepareRecord */ void -ParsePrepareRecord(uint8 info, xl_xact_prepare *xlrec, xl_xact_parsed_prepare *parsed) +ParsePrepareRecord(uint8 rmgrinfo, xl_xact_prepare *xlrec, xl_xact_parsed_prepare *parsed) { char *bufptr; @@ -328,11 +328,11 @@ xact_desc_stats(StringInfo buf, const char *label, } static void -xact_desc_commit(StringInfo buf, uint8 info, xl_xact_commit *xlrec, RepOriginId origin_id) +xact_desc_commit(StringInfo buf, uint8 rmgrinfo, xl_xact_commit *xlrec, RepOriginId origin_id) { xl_xact_parsed_commit parsed; - ParseCommitRecord(info, xlrec, &parsed); + ParseCommitRecord(rmgrinfo, xlrec, &parsed); /* If this is a prepared xact, show the xid of the original xact */ if (TransactionIdIsValid(parsed.twophase_xid)) @@ -364,11 +364,11 @@ xact_desc_commit(StringInfo buf, uint8 info, xl_xact_commit *xlrec, RepOriginId } static void -xact_desc_abort(StringInfo buf, uint8 info, xl_xact_abort *xlrec, RepOriginId origin_id) +xact_desc_abort(StringInfo buf, uint8 rmgrinfo, xl_xact_abort *xlrec, RepOriginId origin_id) { xl_xact_parsed_abort parsed; - ParseAbortRecord(info, xlrec, &parsed); + ParseAbortRecord(rmgrinfo, xlrec, &parsed); /* If this is a prepared xact, show the xid of the original xact */ if (TransactionIdIsValid(parsed.twophase_xid)) @@ -391,11 +391,11 @@ xact_desc_abort(StringInfo buf, uint8 info, xl_xact_abort *xlrec, RepOriginId or } static void -xact_desc_prepare(StringInfo buf, uint8 info, xl_xact_prepare *xlrec, RepOriginId origin_id) +xact_desc_prepare(StringInfo buf, uint8 rmgrinfo, xl_xact_prepare *xlrec, RepOriginId origin_id) { xl_xact_parsed_prepare parsed; - ParsePrepareRecord(info, xlrec, &parsed); + ParsePrepareRecord(rmgrinfo, xlrec, &parsed); appendStringInfo(buf, "gid %s: ", parsed.twophase_gid); appendStringInfoString(buf, timestamptz_to_str(parsed.xact_time)); @@ -436,27 +436,27 @@ void xact_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & XLOG_XACT_OPMASK; + uint8 info = XLogRecGetRmgrInfo(record) & XLOG_XACT_OPMASK; if (info == XLOG_XACT_COMMIT || info == XLOG_XACT_COMMIT_PREPARED) { xl_xact_commit *xlrec = (xl_xact_commit *) rec; - xact_desc_commit(buf, XLogRecGetInfo(record), xlrec, + xact_desc_commit(buf, XLogRecGetRmgrInfo(record), xlrec, XLogRecGetOrigin(record)); } else if (info == XLOG_XACT_ABORT || info == XLOG_XACT_ABORT_PREPARED) { xl_xact_abort *xlrec = (xl_xact_abort *) rec; - xact_desc_abort(buf, XLogRecGetInfo(record), xlrec, + xact_desc_abort(buf, XLogRecGetRmgrInfo(record), xlrec, XLogRecGetOrigin(record)); } else if (info == XLOG_XACT_PREPARE) { xl_xact_prepare *xlrec = (xl_xact_prepare *) rec; - xact_desc_prepare(buf, XLogRecGetInfo(record), xlrec, + xact_desc_prepare(buf, XLogRecGetRmgrInfo(record), xlrec, XLogRecGetOrigin(record)); } else if (info == XLOG_XACT_ASSIGNMENT) diff --git a/src/backend/access/rmgrdesc/xlogdesc.c b/src/backend/access/rmgrdesc/xlogdesc.c index f390c177e4..3692dfb96a 100644 --- a/src/backend/access/rmgrdesc/xlogdesc.c +++ b/src/backend/access/rmgrdesc/xlogdesc.c @@ -37,7 +37,7 @@ void xlog_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_CHECKPOINT_SHUTDOWN || info == XLOG_CHECKPOINT_ONLINE) diff --git a/src/backend/access/spgist/spgxlog.c b/src/backend/access/spgist/spgxlog.c index 459ac929ba..53190a67f9 100644 --- a/src/backend/access/spgist/spgxlog.c +++ b/src/backend/access/spgist/spgxlog.c @@ -936,7 +936,7 @@ spgRedoVacuumRedirect(XLogReaderState *record) void spg_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); MemoryContext oldCxt; oldCxt = MemoryContextSwitchTo(opCtx); diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c index 4a431d5876..11c10b936d 100644 --- a/src/backend/access/transam/clog.c +++ b/src/backend/access/transam/clog.c @@ -984,7 +984,7 @@ WriteTruncateXlogRec(int pageno, TransactionId oldestXact, Oid oldestXactDb) void clog_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in clog records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/backend/access/transam/commit_ts.c b/src/backend/access/transam/commit_ts.c index b897fabc70..a351ee84ce 100644 --- a/src/backend/access/transam/commit_ts.c +++ b/src/backend/access/transam/commit_ts.c @@ -966,7 +966,7 @@ WriteTruncateXlogRec(int pageno, TransactionId oldestXid) void commit_ts_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in commit_ts records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c index abb022e067..c70b6b7358 100644 --- a/src/backend/access/transam/multixact.c +++ b/src/backend/access/transam/multixact.c @@ -3232,7 +3232,7 @@ WriteMTruncateXlogRec(Oid oldestMultiDB, void multixact_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in multixact records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c index 068e59bec0..f9ab5a9c7f 100644 --- a/src/backend/access/transam/twophase.c +++ b/src/backend/access/transam/twophase.c @@ -1429,7 +1429,7 @@ XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len) } if (XLogRecGetRmid(xlogreader) != RM_XACT_ID || - (XLogRecGetInfo(xlogreader) & XLOG_XACT_OPMASK) != XLOG_XACT_PREPARE) + (XLogRecGetRmgrInfo(xlogreader) & XLOG_XACT_OPMASK) != XLOG_XACT_PREPARE) ereport(ERROR, (errcode_for_file_access(), errmsg("expected two-phase state data is not present in WAL at %X/%X", diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index 8daaa535ed..c560f07390 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -6183,7 +6183,7 @@ xact_redo_abort(xl_xact_parsed_abort *parsed, TransactionId xid, void xact_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & XLOG_XACT_OPMASK; + uint8 info = XLogRecGetRmgrInfo(record) & XLOG_XACT_OPMASK; /* Backup blocks are not used in xact records */ Assert(!XLogRecHasAnyBlockRefs(record)); @@ -6193,7 +6193,7 @@ xact_redo(XLogReaderState *record) xl_xact_commit *xlrec = (xl_xact_commit *) XLogRecGetData(record); xl_xact_parsed_commit parsed; - ParseCommitRecord(XLogRecGetInfo(record), xlrec, &parsed); + ParseCommitRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); xact_redo_commit(&parsed, XLogRecGetXid(record), record->EndRecPtr, XLogRecGetOrigin(record)); } @@ -6202,7 +6202,7 @@ xact_redo(XLogReaderState *record) xl_xact_commit *xlrec = (xl_xact_commit *) XLogRecGetData(record); xl_xact_parsed_commit parsed; - ParseCommitRecord(XLogRecGetInfo(record), xlrec, &parsed); + ParseCommitRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); xact_redo_commit(&parsed, parsed.twophase_xid, record->EndRecPtr, XLogRecGetOrigin(record)); @@ -6216,7 +6216,7 @@ xact_redo(XLogReaderState *record) xl_xact_abort *xlrec = (xl_xact_abort *) XLogRecGetData(record); xl_xact_parsed_abort parsed; - ParseAbortRecord(XLogRecGetInfo(record), xlrec, &parsed); + ParseAbortRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); xact_redo_abort(&parsed, XLogRecGetXid(record), record->EndRecPtr, XLogRecGetOrigin(record)); } @@ -6225,7 +6225,7 @@ xact_redo(XLogReaderState *record) xl_xact_abort *xlrec = (xl_xact_abort *) XLogRecGetData(record); xl_xact_parsed_abort parsed; - ParseAbortRecord(XLogRecGetInfo(record), xlrec, &parsed); + ParseAbortRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); xact_redo_abort(&parsed, parsed.twophase_xid, record->EndRecPtr, XLogRecGetOrigin(record)); diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 8b0710abe6..b01cd6cc23 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -7719,7 +7719,7 @@ UpdateFullPageWrites(void) void xlog_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); XLogRecPtr lsn = record->EndRecPtr; /* diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index becc2bda62..6508246999 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -1995,7 +1995,7 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl static void xlogrecovery_redo(XLogReaderState *record, TimeLineID replayTLI) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); XLogRecPtr lsn = record->EndRecPtr; Assert(XLogRecGetRmid(record) == RM_XLOG_ID); @@ -2213,7 +2213,7 @@ void xlog_outdesc(StringInfo buf, XLogReaderState *record) { RmgrData rmgr = GetRmgr(XLogRecGetRmid(record)); - uint8 info = XLogRecGetInfo(record); + uint8 info = XLogRecGetRmgrInfo(record); const char *id; appendStringInfoString(buf, rmgr.rm_name); @@ -2221,7 +2221,7 @@ xlog_outdesc(StringInfo buf, XLogReaderState *record) id = rmgr.rm_identify(info); if (id == NULL) - appendStringInfo(buf, "UNKNOWN (%X): ", info & ~XLR_INFO_MASK); + appendStringInfo(buf, "UNKNOWN (%X): ", info); else appendStringInfo(buf, "%s: ", id); @@ -2341,7 +2341,7 @@ checkTimeLineSwitch(XLogRecPtr lsn, TimeLineID newTLI, TimeLineID prevTLI, static bool getRecordTimestamp(XLogReaderState *record, TimestampTz *recordXtime) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); uint8 xact_info = info & XLOG_XACT_OPMASK; uint8 rmid = XLogRecGetRmid(record); @@ -2535,7 +2535,7 @@ recoveryStopsBefore(XLogReaderState *record) if (XLogRecGetRmid(record) != RM_XACT_ID) return false; - xact_info = XLogRecGetInfo(record) & XLOG_XACT_OPMASK; + xact_info = XLogRecGetRmgrInfo(record); if (xact_info == XLOG_XACT_COMMIT) { @@ -2548,7 +2548,7 @@ recoveryStopsBefore(XLogReaderState *record) xl_xact_parsed_commit parsed; isCommit = true; - ParseCommitRecord(XLogRecGetInfo(record), + ParseCommitRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); recordXid = parsed.twophase_xid; @@ -2564,7 +2564,7 @@ recoveryStopsBefore(XLogReaderState *record) xl_xact_parsed_abort parsed; isCommit = false; - ParseAbortRecord(XLogRecGetInfo(record), + ParseAbortRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); recordXid = parsed.twophase_xid; @@ -2653,7 +2653,7 @@ recoveryStopsAfter(XLogReaderState *record) if (!ArchiveRecoveryRequested) return false; - info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + info = XLogRecGetRmgrInfo(record); rmid = XLogRecGetRmid(record); /* @@ -2721,7 +2721,7 @@ recoveryStopsAfter(XLogReaderState *record) xl_xact_commit *xlrec = (xl_xact_commit *) XLogRecGetData(record); xl_xact_parsed_commit parsed; - ParseCommitRecord(XLogRecGetInfo(record), + ParseCommitRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); recordXid = parsed.twophase_xid; @@ -2731,7 +2731,7 @@ recoveryStopsAfter(XLogReaderState *record) xl_xact_abort *xlrec = (xl_xact_abort *) XLogRecGetData(record); xl_xact_parsed_abort parsed; - ParseAbortRecord(XLogRecGetInfo(record), + ParseAbortRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); recordXid = parsed.twophase_xid; @@ -2925,7 +2925,7 @@ recoveryApplyDelay(XLogReaderState *record) if (XLogRecGetRmid(record) != RM_XACT_ID) return false; - xact_info = XLogRecGetInfo(record) & XLOG_XACT_OPMASK; + xact_info = XLogRecGetRmgrInfo(record) & XLOG_XACT_OPMASK; if (xact_info != XLOG_XACT_COMMIT && xact_info != XLOG_XACT_COMMIT_PREPARED) diff --git a/src/backend/access/transam/xlogstats.c b/src/backend/access/transam/xlogstats.c index 2d315df67a..267b108943 100644 --- a/src/backend/access/transam/xlogstats.c +++ b/src/backend/access/transam/xlogstats.c @@ -79,7 +79,7 @@ XLogRecStoreStats(XLogStats *stats, XLogReaderState *record) * RmgrId). */ - recid = XLogRecGetInfo(record) >> 4; + recid = XLogRecGetRmgrInfo(record) >> 4; /* * XACT records need to be handled differently. Those records use the diff --git a/src/backend/catalog/storage.c b/src/backend/catalog/storage.c index 2add053489..4d73023a65 100644 --- a/src/backend/catalog/storage.c +++ b/src/backend/catalog/storage.c @@ -958,7 +958,7 @@ void smgr_redo(XLogReaderState *record) { XLogRecPtr lsn = record->EndRecPtr; - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in smgr records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c index 09f1ab41ad..dffb683e74 100644 --- a/src/backend/commands/dbcommands.c +++ b/src/backend/commands/dbcommands.c @@ -3109,7 +3109,7 @@ recovery_create_dbdir(char *path, bool only_tblspc) void dbase_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in dbase records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c index ef01449678..eae2cb09b1 100644 --- a/src/backend/commands/sequence.c +++ b/src/backend/commands/sequence.c @@ -1829,7 +1829,7 @@ void seq_redo(XLogReaderState *record) { XLogRecPtr lsn = record->EndRecPtr; - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); Buffer buffer; Page page; Page localpage; diff --git a/src/backend/commands/tablespace.c b/src/backend/commands/tablespace.c index 13b0dee146..1de6803df7 100644 --- a/src/backend/commands/tablespace.c +++ b/src/backend/commands/tablespace.c @@ -1516,7 +1516,7 @@ get_tablespace_name(Oid spc_oid) void tblspc_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in tblspc records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c index d91055a440..419ab527a8 100644 --- a/src/backend/replication/logical/decode.c +++ b/src/backend/replication/logical/decode.c @@ -132,7 +132,7 @@ void xlog_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) { SnapBuild *builder = ctx->snapshot_builder; - uint8 info = XLogRecGetInfo(buf->record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(buf->record); ReorderBufferProcessXid(ctx->reorder, XLogRecGetXid(buf->record), buf->origptr); @@ -205,7 +205,7 @@ xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) SnapBuild *builder = ctx->snapshot_builder; ReorderBuffer *reorder = ctx->reorder; XLogReaderState *r = buf->record; - uint8 info = XLogRecGetInfo(r) & XLOG_XACT_OPMASK; + uint8 info = XLogRecGetRmgrInfo(r) & XLOG_XACT_OPMASK; /* * If the snapshot isn't yet fully built, we cannot decode anything, so @@ -225,7 +225,9 @@ xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) bool two_phase = false; xlrec = (xl_xact_commit *) XLogRecGetData(r); - ParseCommitRecord(XLogRecGetInfo(buf->record), xlrec, &parsed); + ParseCommitRecord(XLogRecGetRmgrInfo(buf->record), + xlrec, + &parsed); if (!TransactionIdIsValid(parsed.twophase_xid)) xid = XLogRecGetXid(r); @@ -253,7 +255,7 @@ xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) bool two_phase = false; xlrec = (xl_xact_abort *) XLogRecGetData(r); - ParseAbortRecord(XLogRecGetInfo(buf->record), xlrec, &parsed); + ParseAbortRecord(XLogRecGetRmgrInfo(buf->record), xlrec, &parsed); if (!TransactionIdIsValid(parsed.twophase_xid)) xid = XLogRecGetXid(r); @@ -316,7 +318,7 @@ xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) /* ok, parse it */ xlrec = (xl_xact_prepare *) XLogRecGetData(r); - ParsePrepareRecord(XLogRecGetInfo(buf->record), + ParsePrepareRecord(XLogRecGetRmgrInfo(buf->record), xlrec, &parsed); /* @@ -361,7 +363,7 @@ standby_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) { SnapBuild *builder = ctx->snapshot_builder; XLogReaderState *r = buf->record; - uint8 info = XLogRecGetInfo(r) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(r); ReorderBufferProcessXid(ctx->reorder, XLogRecGetXid(r), buf->origptr); @@ -405,7 +407,7 @@ standby_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) void heap2_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) { - uint8 info = XLogRecGetInfo(buf->record) & XLOG_HEAP_OPMASK; + uint8 info = XLogRecGetRmgrInfo(buf->record) & XLOG_HEAP_OPMASK; TransactionId xid = XLogRecGetXid(buf->record); SnapBuild *builder = ctx->snapshot_builder; @@ -465,7 +467,7 @@ heap2_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) void heap_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) { - uint8 info = XLogRecGetInfo(buf->record) & XLOG_HEAP_OPMASK; + uint8 info = XLogRecGetRmgrInfo(buf->record) & XLOG_HEAP_OPMASK; TransactionId xid = XLogRecGetXid(buf->record); SnapBuild *builder = ctx->snapshot_builder; @@ -590,7 +592,7 @@ logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) SnapBuild *builder = ctx->snapshot_builder; XLogReaderState *r = buf->record; TransactionId xid = XLogRecGetXid(r); - uint8 info = XLogRecGetInfo(r) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(r); RepOriginId origin_id = XLogRecGetOrigin(r); Snapshot snapshot = NULL; xl_logical_message *message; diff --git a/src/backend/replication/logical/message.c b/src/backend/replication/logical/message.c index c5de14afc6..c31dc1203f 100644 --- a/src/backend/replication/logical/message.c +++ b/src/backend/replication/logical/message.c @@ -80,7 +80,7 @@ LogLogicalMessage(const char *prefix, const char *message, size_t size, void logicalmsg_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info != XLOG_LOGICAL_MESSAGE) elog(PANIC, "logicalmsg_redo: unknown op code %u", info); diff --git a/src/backend/replication/logical/origin.c b/src/backend/replication/logical/origin.c index b0255ffd25..4977d2eb9a 100644 --- a/src/backend/replication/logical/origin.c +++ b/src/backend/replication/logical/origin.c @@ -826,7 +826,7 @@ StartupReplicationOrigin(void) void replorigin_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c index 4c06741a69..3fa1ca12a3 100644 --- a/src/backend/storage/ipc/standby.c +++ b/src/backend/storage/ipc/standby.c @@ -1160,7 +1160,7 @@ StandbyReleaseOldLocks(TransactionId oldxid) void standby_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in standby records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/backend/utils/cache/relmapper.c b/src/backend/utils/cache/relmapper.c index 26575cae6c..b825dda353 100644 --- a/src/backend/utils/cache/relmapper.c +++ b/src/backend/utils/cache/relmapper.c @@ -1084,7 +1084,7 @@ perform_relmap_update(bool shared, const RelMapFile *updates) void relmap_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in relmap records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/bin/pg_rewind/parsexlog.c b/src/bin/pg_rewind/parsexlog.c index 27782237d0..2b6b46f0e3 100644 --- a/src/bin/pg_rewind/parsexlog.c +++ b/src/bin/pg_rewind/parsexlog.c @@ -222,7 +222,7 @@ findLastCheckpoint(const char *datadir, XLogRecPtr forkptr, int tliIndex, * be the latest checkpoint before WAL forked and not the checkpoint * where the primary has been stopped to be rewound. */ - info = XLogRecGetInfo(xlogreader) & ~XLR_INFO_MASK; + info = XLogRecGetRmgrInfo(xlogreader); if (searchptr < forkptr && XLogRecGetRmid(xlogreader) == RM_XLOG_ID && (info == XLOG_CHECKPOINT_SHUTDOWN || @@ -370,7 +370,7 @@ extractPageInfo(XLogReaderState *record) int block_id; RmgrId rmid = XLogRecGetRmid(record); uint8 info = XLogRecGetInfo(record); - uint8 rminfo = info & ~XLR_INFO_MASK; + uint8 rminfo = XLogRecGetRmgrInfo(record); /* Is this a special record type that I recognize? */ @@ -440,7 +440,7 @@ extractPageInfo(XLogReaderState *record) pg_fatal("WAL record modifies a relation, but record type is not recognized: " "lsn: %X/%X, rmid: %d, rmgr: %s, info: %02X", LSN_FORMAT_ARGS(record->ReadRecPtr), - rmid, RmgrName(rmid), info); + rmid, RmgrName(rmid), info | rminfo); } for (block_id = 0; block_id <= XLogRecMaxBlockId(record); block_id++) diff --git a/src/bin/pg_waldump/pg_waldump.c b/src/bin/pg_waldump/pg_waldump.c index 96845e1a1a..85694b3d5c 100644 --- a/src/bin/pg_waldump/pg_waldump.c +++ b/src/bin/pg_waldump/pg_waldump.c @@ -545,7 +545,7 @@ XLogDumpDisplayRecord(XLogDumpConfig *config, XLogReaderState *record) const RmgrDescData *desc = GetRmgrDesc(XLogRecGetRmid(record)); uint32 rec_len; uint32 fpi_len; - uint8 info = XLogRecGetInfo(record); + uint8 info = XLogRecGetRmgrInfo(record); XLogRecPtr xl_prev = XLogRecGetPrev(record); StringInfoData s; @@ -560,7 +560,7 @@ XLogDumpDisplayRecord(XLogDumpConfig *config, XLogReaderState *record) id = desc->rm_identify(info); if (id == NULL) - printf("desc: UNKNOWN (%x) ", info & ~XLR_INFO_MASK); + printf("desc: UNKNOWN (%x) ", info); else printf("desc: %s ", id); diff --git a/src/include/access/xact.h b/src/include/access/xact.h index 7d3b9446e6..b94b264402 100644 --- a/src/include/access/xact.h +++ b/src/include/access/xact.h @@ -519,9 +519,9 @@ extern void xact_desc(StringInfo buf, XLogReaderState *record); extern const char *xact_identify(uint8 info); /* also in xactdesc.c, so they can be shared between front/backend code */ -extern void ParseCommitRecord(uint8 info, xl_xact_commit *xlrec, xl_xact_parsed_commit *parsed); -extern void ParseAbortRecord(uint8 info, xl_xact_abort *xlrec, xl_xact_parsed_abort *parsed); -extern void ParsePrepareRecord(uint8 info, xl_xact_prepare *xlrec, xl_xact_parsed_prepare *parsed); +extern void ParseCommitRecord(uint8 rmgrinfo, xl_xact_commit *xlrec, xl_xact_parsed_commit *parsed); +extern void ParseAbortRecord(uint8 rmgrinfo, xl_xact_abort *xlrec, xl_xact_parsed_abort *parsed); +extern void ParsePrepareRecord(uint8 rmgrinfo, xl_xact_prepare *xlrec, xl_xact_parsed_prepare *parsed); extern void EnterParallelMode(void); extern void ExitParallelMode(void); diff --git a/src/include/access/xlogreader.h b/src/include/access/xlogreader.h index da32c7db77..2b976dcc03 100644 --- a/src/include/access/xlogreader.h +++ b/src/include/access/xlogreader.h @@ -407,7 +407,8 @@ extern bool DecodeXLogRecord(XLogReaderState *state, */ #define XLogRecGetTotalLen(decoder) ((decoder)->record->header.xl_tot_len) #define XLogRecGetPrev(decoder) ((decoder)->record->header.xl_prev) -#define XLogRecGetInfo(decoder) ((decoder)->record->header.xl_info) +#define XLogRecGetInfo(decoder) ((decoder)->record->header.xl_info & XLR_INFO_MASK) +#define XLogRecGetRmgrInfo(decoder) (((decoder)->record->header.xl_info) & XLR_RMGR_INFO_MASK) #define XLogRecGetRmid(decoder) ((decoder)->record->header.xl_rmid) #define XLogRecGetXid(decoder) ((decoder)->record->header.xl_xid) #define XLogRecGetOrigin(decoder) ((decoder)->record->record_origin) diff --git a/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c b/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c index a304ba54bb..b5537a5f5b 100644 --- a/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c +++ b/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c @@ -81,7 +81,7 @@ _PG_init(void) void testcustomrmgrs_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info != XLOG_TEST_CUSTOM_RMGRS_MESSAGE) elog(PANIC, "testcustomrmgrs_redo: unknown op code %u", info); @@ -91,7 +91,7 @@ void testcustomrmgrs_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_TEST_CUSTOM_RMGRS_MESSAGE) { -- 2.40.1 [application/octet-stream] v2-0005-Stop-the-logging-of-XIDs-in-records-where-the-XID.patch (10.4K, ../../CAEze2Wj+L1AGNAwjuQFYMcu2be0NYEw1oSg8Sa-HvVbC7iHZLQ@mail.gmail.com/6-v2-0005-Stop-the-logging-of-XIDs-in-records-where-the-XID.patch) download | inline diff: From 56fdf28a10d531984b1765f3bd6ae3fbee8c0006 Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Wed, 25 Jan 2023 15:40:30 +0100 Subject: [PATCH v2 5/6] Stop the logging of XIDs in records where the XID is unused. Many WAL record types do not need to contain an XID. This change prepares for the removal of the xid from the main xlog record header, by requiring the user of the XLog insertion API to explicitly request the inclusion of the current xid in the log record - in all other cases we'll log, assume and track InvalidXid instead. --- src/backend/access/heap/heapam.c | 16 +++++++++------- src/backend/access/heap/pruneheap.c | 1 + src/backend/access/heap/rewriteheap.c | 1 + src/backend/access/transam/generic_xlog.c | 3 +++ src/backend/access/transam/multixact.c | 1 + src/backend/access/transam/xact.c | 4 ++-- src/backend/access/transam/xloginsert.c | 12 ++++++++++-- src/backend/commands/tablecmds.c | 2 +- src/backend/replication/logical/message.c | 2 +- src/backend/utils/cache/inval.c | 1 + src/include/access/xlog.h | 1 + 11 files changed, 31 insertions(+), 13 deletions(-) diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 1512f22c76..30b7b05f54 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -1969,7 +1969,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid, heaptup->t_len - SizeofHeapTupleHeader); /* filtering by origin on a row level is much more efficient */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP_ID, info); @@ -2365,7 +2365,7 @@ heap_multi_insert(Relation relation, TupleTableSlot **slots, int ntuples, XLogRegisterBufData(0, tupledata, totaldatalen); /* filtering by origin on a row level is much more efficient */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP2_ID, info); @@ -2865,7 +2865,7 @@ l1: } /* filtering by origin on a row level is much more efficient */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP_ID, XLOG_HEAP_DELETE); @@ -5535,6 +5535,7 @@ l4: cleared_all_frozen ? XLH_LOCK_ALL_FROZEN_CLEARED : 0; XLogRegisterData((char *) &xlrec, SizeOfHeapLockUpdated); + XLogSetRecordFlags(XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP2_ID, XLOG_HEAP2_LOCK_UPDATED); @@ -5685,7 +5686,7 @@ heap_finish_speculative(Relation relation, ItemPointer tid) XLogBeginInsert(); /* We want the same filtering on this as on a plain insert */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); XLogRegisterData((char *) &xlrec, SizeOfHeapConfirm); XLogRegisterBuffer(0, buffer, REGBUF_STANDARD); @@ -5831,7 +5832,8 @@ heap_abort_speculative(Relation relation, ItemPointer tid) XLogRegisterData((char *) &xlrec, SizeOfHeapDelete); XLogRegisterBuffer(0, buffer, REGBUF_STANDARD); - /* No replica identity & replication origin logged */ + /* No replica identity & replication origin logged, but XID is required */ + XLogSetRecordFlags(XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP_ID, XLOG_HEAP_DELETE); @@ -8563,7 +8565,7 @@ log_heap_update(Relation reln, Buffer oldbuf, } /* filtering by origin on a row level is much more efficient */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP_ID, info); @@ -8638,7 +8640,7 @@ log_heap_new_cid(Relation relation, HeapTuple tup) XLogRegisterData((char *) &xlrec, SizeOfHeapNewCid); /* will be looked at irrespective of origin */ - + XLogSetRecordFlags(XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP2_ID, XLOG_HEAP2_NEW_CID); return recptr; diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index 47b9e20915..24e1bbe1ad 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -445,6 +445,7 @@ heap_page_prune(Relation relation, Buffer buffer, XLogRegisterBufData(0, (char *) prstate.nowunused, prstate.nunused * sizeof(OffsetNumber)); + XLogSetRecordFlags(XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP2_ID, XLOG_HEAP2_PRUNE); PageSetLSN(BufferGetPage(buffer), recptr); diff --git a/src/backend/access/heap/rewriteheap.c b/src/backend/access/heap/rewriteheap.c index 424958912c..9f77537335 100644 --- a/src/backend/access/heap/rewriteheap.c +++ b/src/backend/access/heap/rewriteheap.c @@ -921,6 +921,7 @@ logical_heap_rewrite_flush_mappings(RewriteState state) src->off += len; XLogBeginInsert(); + XLogSetRecordFlags(XLOG_INCLUDE_XID); XLogRegisterData((char *) (&xlrec), sizeof(xlrec)); XLogRegisterData(waldata_start, len); diff --git a/src/backend/access/transam/generic_xlog.c b/src/backend/access/transam/generic_xlog.c index 6c68191ca6..c8983b40ad 100644 --- a/src/backend/access/transam/generic_xlog.c +++ b/src/backend/access/transam/generic_xlog.c @@ -399,6 +399,9 @@ GenericXLogFinish(GenericXLogState *state) } } + /* Assume XID is required */ + XLogSetRecordFlags(XLOG_INCLUDE_XID); + /* Insert xlog record */ lsn = XLogInsert(RM_GENERIC_ID, 0); diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c index c70b6b7358..883da6a33b 100644 --- a/src/backend/access/transam/multixact.c +++ b/src/backend/access/transam/multixact.c @@ -834,6 +834,7 @@ MultiXactIdCreateFromMembers(int nmembers, MultiXactMember *members) * Not clear that it's worth the trouble though. */ XLogBeginInsert(); + XLogSetRecordFlags(XLOG_INCLUDE_XID); XLogRegisterData((char *) (&xlrec), SizeOfMultiXactCreate); XLogRegisterData((char *) members, nmembers * sizeof(MultiXactMember)); diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index da0ac6c8d3..57504594b0 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -5793,7 +5793,7 @@ XactLogCommitRecord(TimestampTz commit_time, XLogRegisterData((char *) (&xl_origin), sizeof(xl_xact_origin)); /* we allow filtering by xacts */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); return XLogInsertExtended(RM_XACT_ID, info, rmgr_info); } @@ -5940,7 +5940,7 @@ XactLogAbortRecord(TimestampTz abort_time, XLogRegisterData((char *) (&xl_origin), sizeof(xl_xact_origin)); /* Include the replication origin */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); return XLogInsertExtended(RM_XACT_ID, info, rmgr_info); } diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c index 9a3eb4fb4b..4816c5284b 100644 --- a/src/backend/access/transam/xloginsert.c +++ b/src/backend/access/transam/xloginsert.c @@ -456,6 +456,10 @@ XLogSetRecordFlags(uint8 flags) * (LSN is the XLOG point up to which the XLOG must be flushed to disk * before the data page can be written out. This implements the basic * WAL rule "write the log before the data".) + * + * Note: To include the current backend's TransactionID in the record, + * you have to set the XLOG_INCLUDE_XID flag using XLogRecordSetFlags + * before calling XLogInsert. */ XLogRecPtr XLogInsert(RmgrId rmid, uint8 rmgr_info) @@ -847,7 +851,7 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP } /* followed by toplevel XID, if not already included in previous record */ - if (IsSubxactTopXidLogPending()) + if (curinsert_flags & XLOG_INCLUDE_XID && IsSubxactTopXidLogPending()) { TransactionId xid = GetTopTransactionIdIfAny(); @@ -924,7 +928,11 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP * once we know where in the WAL the record will be inserted. The CRC does * not include the record header yet. */ - rechdr->xl_xid = GetCurrentTransactionIdIfAny(); + if (curinsert_flags & XLOG_INCLUDE_XID) + rechdr->xl_xid = GetCurrentTransactionIdIfAny(); + else + rechdr->xl_xid = InvalidTransactionId; + rechdr->xl_tot_len = (uint32) total_len; rechdr->xl_info = info; rechdr->xl_rmid = rmid; diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d985278ac6..264a0399b4 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -2132,7 +2132,7 @@ ExecuteTruncateGuts(List *explicit_rels, XLogRegisterData((char *) &xlrec, SizeOfHeapTruncate); XLogRegisterData((char *) logrelids, list_length(relids_logged) * sizeof(Oid)); - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); (void) XLogInsert(RM_HEAP_ID, XLOG_HEAP_TRUNCATE); } diff --git a/src/backend/replication/logical/message.c b/src/backend/replication/logical/message.c index c31dc1203f..af86d18457 100644 --- a/src/backend/replication/logical/message.c +++ b/src/backend/replication/logical/message.c @@ -69,7 +69,7 @@ LogLogicalMessage(const char *prefix, const char *message, size_t size, XLogRegisterData(unconstify(char *, message), size); /* allow origin filtering */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); return XLogInsert(RM_LOGICALMSG_ID, XLOG_LOGICAL_MESSAGE); } diff --git a/src/backend/utils/cache/inval.c b/src/backend/utils/cache/inval.c index 0008826f67..b0afdd6f90 100644 --- a/src/backend/utils/cache/inval.c +++ b/src/backend/utils/cache/inval.c @@ -1632,6 +1632,7 @@ LogLogicalInvalidations(void) ProcessMessageSubGroupMulti(group, RelCacheMsgs, XLogRegisterData((char *) msgs, n * sizeof(SharedInvalidationMessage))); + XLogSetRecordFlags(XLOG_INCLUDE_XID); XLogInsert(RM_XACT_ID, XLOG_XACT_INVALIDATIONS); } } diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h index 9007f559f3..f4dfaeda29 100644 --- a/src/include/access/xlog.h +++ b/src/include/access/xlog.h @@ -150,6 +150,7 @@ extern PGDLLIMPORT bool XLOG_DEBUG; */ #define XLOG_INCLUDE_ORIGIN 0x01 /* include the replication origin */ #define XLOG_MARK_UNIMPORTANT 0x02 /* record not important for durability */ +#define XLOG_INCLUDE_XID 0x04 /* include the transaction ID */ /* Checkpoint statistics */ -- 2.40.1 [application/octet-stream] v2-0006-Reformat-the-XLog-record-header.patch (42.4K, ../../CAEze2Wj+L1AGNAwjuQFYMcu2be0NYEw1oSg8Sa-HvVbC7iHZLQ@mail.gmail.com/7-v2-0006-Reformat-the-XLog-record-header.patch) download | inline diff: From 58c94d4772407125ea7f6a577a047719bced56d8 Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Mon, 19 Jun 2023 17:46:45 +0200 Subject: [PATCH v2 6/6] Reformat the XLog record header Many WAL records don't utilize all bytes of the XLog header, resulting in a lot of wasted bytes. This new XLog header format saves bytes in several ways: - Records that don't need the XID won't include it This potentially saves 4 bytes /record - Records that are small enough don't need 4 bytes to describe their length By varint encoding the length, we save up to 3 bytes /record (up to 4 bytes /record if dataless records are considered) - Only include xl_rmgrinfo if it is non-0 Several RMGRs have a record that they use most often. We save 1 byte/record if that record id is 0. - Lose some alignment losses We save 1 byte/record by removing one byte of alignment losses --- src/backend/access/transam/xlog.c | 133 ++++++++++++++------- src/backend/access/transam/xloginsert.c | 135 ++++++++++++++++------ src/backend/access/transam/xlogreader.c | 93 ++++++++------- src/backend/access/transam/xlogrecovery.c | 4 +- src/bin/pg_resetwal/pg_resetwal.c | 38 +++--- src/include/access/rmgr.h | 2 +- src/include/access/rmgrlist.h | 1 + src/include/access/xlog_internal.h | 102 +++++++++++++++- src/include/access/xlogreader.h | 4 +- src/include/access/xlogrecord.h | 125 +++++++++++++++++--- 10 files changed, 483 insertions(+), 154 deletions(-) diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 4b1ff0d1aa..5937b43f50 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -737,17 +737,18 @@ XLogInsertRecord(XLogRecData *rdata, XLogCtlInsert *Insert = &XLogCtl->Insert; pg_crc32c rdata_crc; bool inserted; - XLogRecord *rechdr = (XLogRecord *) rdata->data; - uint8 rmgrinfo = rechdr->xl_rmgrinfo; - bool isLogSwitch = (rechdr->xl_rmid == RM_XLOG_ID && - rmgrinfo == XLOG_SWITCH); + XLogRecHdr rechdr; + bool isLogSwitch; XLogRecPtr StartPos; XLogRecPtr EndPos; + XLogRecPtr xl_prev; bool prevDoPageWrites = doPageWrites; TimeLineID insertTLI; + int rec_payload_len, + rec_hdr_len; - /* we assume that all of the record header is in the first chunk */ - Assert(rdata->len >= SizeOfXLogRecord); + /* we assume that all data of the record header is in the first chunk */ + Assert(rdata->len >= XLogRecordMinHdrSize); /* cross-check on whether we should be here or not */ if (!XLogInsertAllowed()) @@ -758,6 +759,17 @@ XLogInsertRecord(XLogRecData *rdata, * change, so we can read it without a lock. */ insertTLI = XLogCtl->InsertTimeLineID; + rechdr = (XLogRecHdr) rdata->data; + + { + XLogRecord record = {0}; + XLogReadRecHdrInto(rechdr, (Size) rdata->len, &record); + + isLogSwitch = (record.xl_rmid == RM_XLOG_ID && + record.xl_rmgrinfo == XLOG_SWITCH); + rec_hdr_len = (int) XLogRecordHdrLen(rechdr->xl_info); + rec_payload_len = (int) record.xl_payload_len; + } /*---------- * @@ -834,34 +846,44 @@ XLogInsertRecord(XLogRecData *rdata, } /* - * Reserve space for the record in the WAL. This also sets the xl_prev - * pointer. + * Reserve space for the record in the WAL. */ if (isLogSwitch) - inserted = ReserveXLogSwitch(&StartPos, &EndPos, &rechdr->xl_prev); + inserted = ReserveXLogSwitch(&StartPos, &EndPos, &xl_prev); else { - ReserveXLogInsertLocation(rechdr->xl_tot_len, &StartPos, &EndPos, - &rechdr->xl_prev); + ReserveXLogInsertLocation(rec_hdr_len + rec_payload_len, + &StartPos, &EndPos, &xl_prev); inserted = true; } if (inserted) { + char *rec = (char *) rechdr; + + /* fill in xl_prev */ + memcpy(rec + rec_hdr_len - sizeof(pg_crc32c) - sizeof(XLogRecPtr), + &xl_prev, sizeof(XLogRecPtr)); + /* * Now that xl_prev has been filled in, calculate CRC of the record * header. */ - rdata_crc = rechdr->xl_crc; - COMP_CRC32C(rdata_crc, rechdr, offsetof(XLogRecord, xl_crc)); + + memcpy(&rdata_crc, rec + rec_hdr_len - sizeof(pg_crc32c), + sizeof(pg_crc32c)); + + COMP_CRC32C(rdata_crc, rec, rec_hdr_len - sizeof(pg_crc32c)); FIN_CRC32C(rdata_crc); - rechdr->xl_crc = rdata_crc; + + memcpy(rec + rec_hdr_len - sizeof(pg_crc32c), + &rdata_crc, sizeof(pg_crc32c)); /* * All the record data, including the header, is now ready to be * inserted. Copy the record in the space reserved. */ - CopyXLogRecordToWAL(rechdr->xl_tot_len, isLogSwitch, rdata, + CopyXLogRecordToWAL(rec_hdr_len + rec_payload_len, isLogSwitch, rdata, StartPos, EndPos, insertTLI); /* @@ -932,7 +954,8 @@ XLogInsertRecord(XLogRecData *rdata, */ if (inserted) { - EndPos = StartPos + SizeOfXLogRecord; + /* xlog switch is minimal record header, plus a byte for rmgrinfo */ + EndPos = StartPos + XLogRecordMinHdrSize + sizeof(uint8); if (StartPos / XLOG_BLCKSZ != EndPos / XLOG_BLCKSZ) { uint64 offset = XLogSegmentOffset(EndPos, wal_segment_size); @@ -949,7 +972,7 @@ XLogInsertRecord(XLogRecData *rdata, if (XLOG_DEBUG) { static XLogReaderState *debug_reader = NULL; - XLogRecord *record; + XLogRecHdr record; DecodedXLogRecord *decoded; StringInfoData buf; StringInfoData recordBuf; @@ -971,9 +994,9 @@ XLogInsertRecord(XLogRecData *rdata, appendBinaryStringInfo(&recordBuf, rdata->data, rdata->len); /* We also need temporary space to decode the record. */ - record = (XLogRecord *) recordBuf.data; + record = (XLogRecHdr) recordBuf.data; decoded = (DecodedXLogRecord *) - palloc(DecodeXLogRecordRequiredSpace(record->xl_tot_len)); + palloc(DecodeXLogRecordRequiredSpace(recordBuf.len)); if (!debug_reader) debug_reader = XLogReaderAllocate(wal_segment_size, NULL, @@ -1018,7 +1041,7 @@ XLogInsertRecord(XLogRecData *rdata, /* Report WAL traffic to the instrumentation. */ if (inserted) { - pgWalUsage.wal_bytes += rechdr->xl_tot_len; + pgWalUsage.wal_bytes += rec_hdr_len + rec_payload_len; pgWalUsage.wal_records++; pgWalUsage.wal_fpi += num_fpi; } @@ -1049,10 +1072,10 @@ ReserveXLogInsertLocation(int size, XLogRecPtr *StartPos, XLogRecPtr *EndPos, uint64 endbytepos; uint64 prevbytepos; - size = MAXALIGN(size); + size = XLP_ALIGN(size); /* All (non xlog-switch) records should contain data. */ - Assert(size > SizeOfXLogRecord); + Assert(size > XLogRecordMinHdrSize); /* * The duration the spinlock needs to be held is minimized by minimizing @@ -1103,7 +1126,7 @@ ReserveXLogSwitch(XLogRecPtr *StartPos, XLogRecPtr *EndPos, XLogRecPtr *PrevPtr) uint64 startbytepos; uint64 endbytepos; uint64 prevbytepos; - uint32 size = MAXALIGN(SizeOfXLogRecord); + uint32 size = XLP_ALIGN(XLogRecordMinHdrSize + sizeof(uint8)); XLogRecPtr ptr; uint32 segleft; @@ -1176,10 +1199,10 @@ CopyXLogRecordToWAL(int write_len, bool isLogSwitch, XLogRecData *rdata, freespace = INSERT_FREESPACE(CurrPos); /* - * there should be enough space for at least the first field (xl_tot_len) + * there should be enough space for at least the first XL_ALIGN quantum * on this page. */ - Assert(freespace >= sizeof(uint32)); + Assert(freespace >= XLP_ALIGN(1)); /* Copy record data */ written = 0; @@ -1247,7 +1270,7 @@ CopyXLogRecordToWAL(int write_len, bool isLogSwitch, XLogRecData *rdata, if (isLogSwitch && XLogSegmentOffset(CurrPos, wal_segment_size) != 0) { /* An xlog-switch record doesn't contain any data besides the header */ - Assert(write_len == SizeOfXLogRecord); + Assert(write_len == XLogRecordMinHdrSize + sizeof(uint8)); /* Assert that we did reserve the right amount of space */ Assert(XLogSegmentOffset(EndPos, wal_segment_size) == 0); @@ -1291,7 +1314,7 @@ CopyXLogRecordToWAL(int write_len, bool isLogSwitch, XLogRecData *rdata, else { /* Align the end position, so that the next record starts aligned */ - CurrPos = MAXALIGN64(CurrPos); + CurrPos = XLP_ALIGN(CurrPos); } if (CurrPos != EndPos) @@ -4649,11 +4672,15 @@ BootStrapXLOG(void) char *buffer; XLogPageHeader page; XLogLongPageHeader longpage; - XLogRecord *record; + XLogRecHdr rec_hdr; + char *baserecptr; char *recptr; uint64 sysidentifier; struct timeval tv; pg_crc32c crc; + const int rec_hdr_len = offsetof(XLogRecHdrData, xl_hdrdata) + + sizeof(uint8) + sizeof(XLogRecPtr) + sizeof(pg_crc32c); + const int rec_payload_len = sizeof(uint8) * 2 + sizeof(CheckPoint); /* allow ordinary WAL segment creation, like StartupXLOG() would */ SetInstallXLogFileSegmentActive(); @@ -4724,28 +4751,52 @@ BootStrapXLOG(void) longpage->xlp_seg_size = wal_segment_size; longpage->xlp_xlog_blcksz = XLOG_BLCKSZ; + /* if this changes, we need to update the code below */ + Assert(XLogLengthToSizeClass(SizeOfXLogRecordDataHeaderShort + sizeof(checkPoint), XLS_UINT32) == XLS_UINT8); + /* if this changes, we need to add XLR_HAS_RMGRINFO */ + Assert(XLOG_CHECKPOINT_SHUTDOWN == 0); + /* Insert the initial checkpoint record */ - recptr = ((char *) page + SizeOfXLogLongPHD); - record = (XLogRecord *) recptr; - record->xl_prev = 0; - record->xl_xid = InvalidTransactionId; - record->xl_tot_len = SizeOfXLogRecord + SizeOfXLogRecordDataHeaderShort + sizeof(checkPoint); - record->xl_info = 0; - record->xl_rmgrinfo = XLOG_CHECKPOINT_SHUTDOWN; - record->xl_rmid = RM_XLOG_ID; - recptr += SizeOfXLogRecord; + baserecptr = recptr = (((char *) page) + SizeOfXLogLongPHD); + + rec_hdr = (XLogRecHdr) baserecptr; + rec_hdr->xl_info = (char) XLS_UINT8; + rec_hdr->xl_rmid = (char) RM_XLOG_ID; + recptr += offsetof(XLogRecHdrData, xl_hdrdata); + + recptr += XLogWriteLength(SizeOfXLogRecordDataHeaderShort + sizeof(checkPoint), + XLS_UINT8, XLS_UINT8, recptr); + + /* include prevptr */ + { + XLogRecPtr prevptr = 0; + memcpy(recptr, &prevptr, sizeof(XLogRecPtr)); + recptr += sizeof(XLogRecPtr); + } + + /* reserve location of crc */ + recptr += sizeof(pg_crc32c); + + Assert(recptr - baserecptr == XLogRecordHdrLen(rec_hdr->xl_info)); + Assert(recptr - baserecptr == rec_hdr_len); + /* fill the XLogRecordDataHeaderShort struct */ *(recptr++) = (char) XLR_BLOCK_ID_DATA_SHORT; *(recptr++) = sizeof(checkPoint); memcpy(recptr, &checkPoint, sizeof(checkPoint)); recptr += sizeof(checkPoint); - Assert(recptr - (char *) record == record->xl_tot_len); + + /* Assert length of record matches expectations */ + Assert(recptr - baserecptr == XLogRecordTotalLength(rec_hdr)); + Assert((rec_hdr)->xl_info == XLS_UINT8); INIT_CRC32C(crc); - COMP_CRC32C(crc, ((char *) record) + SizeOfXLogRecord, record->xl_tot_len - SizeOfXLogRecord); - COMP_CRC32C(crc, (char *) record, offsetof(XLogRecord, xl_crc)); + COMP_CRC32C(crc, baserecptr + rec_hdr_len, rec_payload_len); + COMP_CRC32C(crc, baserecptr, rec_hdr_len - sizeof(pg_crc32c)); FIN_CRC32C(crc); - record->xl_crc = crc; + + memcpy(baserecptr + rec_hdr_len - sizeof(pg_crc32c), + &crc, sizeof(pg_crc32c)); /* Create first XLOG segment file */ openLogTLI = BootstrapTimeLineID; diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c index 4816c5284b..194196f596 100644 --- a/src/backend/access/transam/xloginsert.c +++ b/src/backend/access/transam/xloginsert.c @@ -545,12 +545,16 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP XLogRecPtr *fpw_lsn, int *num_fpi, bool *topxid_included) { XLogRecData *rdt; - uint64 total_len = 0; + uint64 payload_len = 0; + XLogSizeClass payload_sizeclass = XLS_EMPTY; int block_id; pg_crc32c rdata_crc; registered_buffer *prev_regbuf = NULL; XLogRecData *rdt_datas_last; - XLogRecord *rechdr; + TransactionId xid; + XLogRecHdr rechdr; + uint8 xlr_flags = 0; + uint32 rec_hdr_len; char *scratch = hdr_scratch; /* @@ -558,9 +562,10 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP * All the modifications we do to the rdata chains below must handle that. */ - /* The record begins with the fixed-size header */ - rechdr = (XLogRecord *) scratch; - scratch += SizeOfXLogRecord; + /* The record begins with the variable-size header */ + rechdr = (XLogRecHdr) scratch; + + scratch += XLogRecordMaxHdrSize; hdr_rdt.next = NULL; rdt_datas_last = &hdr_rdt; @@ -779,7 +784,7 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP } } - total_len += bimg.length; + payload_len += bimg.length; } if (needs_data) @@ -797,7 +802,7 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP bkpb.fork_flags |= BKPBLOCK_HAS_DATA; data_length = (uint16) regbuf->rdata_len; data_sizeclass = XLogLengthToSizeClass(data_length, XLS_UINT16); - total_len += regbuf->rdata_len; + payload_len += regbuf->rdata_len; rdt_datas_last->next = regbuf->rdata_head; rdt_datas_last = regbuf->rdata_tail; @@ -851,16 +856,33 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP } /* followed by toplevel XID, if not already included in previous record */ - if (curinsert_flags & XLOG_INCLUDE_XID && IsSubxactTopXidLogPending()) + if (curinsert_flags & XLOG_INCLUDE_XID) { - TransactionId xid = GetTopTransactionIdIfAny(); + xid = GetCurrentTransactionIdIfAny(); + + if (IsSubxactTopXidLogPending()) + { + TransactionId txid = GetTopTransactionIdIfAny(); + + xlr_flags |= XLR_HAS_XID; + Assert(TransactionIdIsValid(xid)); - /* Set the flag that the top xid is included in the WAL */ - *topxid_included = true; + /* Set the flag that the top xid is included in the WAL */ + *topxid_included = true; - *(scratch++) = (char) XLR_BLOCK_ID_TOPLEVEL_XID; - memcpy(scratch, &xid, sizeof(TransactionId)); - scratch += sizeof(TransactionId); + *(scratch++) = (char) XLR_BLOCK_ID_TOPLEVEL_XID; + memcpy(scratch, &txid, sizeof(TransactionId)); + scratch += sizeof(TransactionId); + } + else if (TransactionIdIsValid(xid)) + { + xlr_flags |= XLR_HAS_XID; + } + } + else + { + xid = InvalidTransactionId; + Assert((xlr_flags & XLR_HAS_XID) == 0); } /* followed by main data, if any */ @@ -889,12 +911,64 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP } rdt_datas_last->next = mainrdata_head; rdt_datas_last = mainrdata_last; - total_len += mainrdata_len; + payload_len += mainrdata_len; } rdt_datas_last->next = NULL; - hdr_rdt.len = (scratch - hdr_scratch); - total_len += hdr_rdt.len; + /* Add the block headers section's length to the payload */ + payload_len += scratch - (hdr_scratch + XLogRecordMaxHdrSize); + + /* + * Fill in the fields in the record header. Prev-link is filled in later, + * once we know where in the WAL the record will be inserted. The CRC does + * not include the record header yet. + */ + payload_sizeclass = XLogLengthToSizeClass(payload_len, XLS_UINT32); + + xlr_flags |= payload_sizeclass; + + if (rmgr_info != 0) + xlr_flags |= XLR_HAS_RMGRINFO; + + /* Set up the xlog header. and xl_rmgr */ + rechdr->xl_info = xlr_flags; + rechdr->xl_rmid = rmid; + + rec_hdr_len = 0; + + /* next, xl_payload_len */ + rec_hdr_len += XLogWriteLength(payload_len, payload_sizeclass, + XLS_UINT32, + &rechdr->xl_hdrdata[rec_hdr_len]); + + if (xlr_flags & XLR_HAS_RMGRINFO) + rechdr->xl_hdrdata[rec_hdr_len++] = (char) rmgr_info; + + if (xlr_flags & XLR_HAS_XID) + { + Assert(curinsert_flags & XLOG_INCLUDE_XID); + Assert(TransactionIdIsValid(xid)); + + memcpy(&rechdr->xl_hdrdata[rec_hdr_len], &xid, sizeof(TransactionId)); + rec_hdr_len += sizeof(TransactionId); + } + + /* reserve space for XLogRecPtr and checksum */ + rec_hdr_len += sizeof(XLogRecPtr); + rec_hdr_len += sizeof(pg_crc32c); + /* Add static header length */ + rec_hdr_len += offsetof(XLogRecHdrData, xl_hdrdata); + + Assert(rec_hdr_len == XLogRecordHdrLen(rechdr->xl_info)); + + /* move the record to be placed the rest of the payload */ + memmove(hdr_scratch + XLogRecordMaxHdrSize - rec_hdr_len, + hdr_scratch, rec_hdr_len); + + rechdr = (XLogRecHdr) (hdr_scratch + XLogRecordMaxHdrSize - rec_hdr_len); + + hdr_rdt.data = (char *) rechdr; + hdr_rdt.len = (scratch - hdr_rdt.data); /* * Calculate CRC of the data @@ -905,10 +979,14 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP * header. */ INIT_CRC32C(rdata_crc); - COMP_CRC32C(rdata_crc, hdr_scratch + SizeOfXLogRecord, hdr_rdt.len - SizeOfXLogRecord); + COMP_CRC32C(rdata_crc, hdr_scratch + XLogRecordMaxHdrSize, + hdr_rdt.len - rec_hdr_len); for (rdt = hdr_rdt.next; rdt != NULL; rdt = rdt->next) COMP_CRC32C(rdata_crc, rdt->data, rdt->len); + memcpy(hdr_rdt.data + rec_hdr_len - sizeof(pg_crc32c), + &rdata_crc, sizeof(pg_crc32c)); + /* * Ensure that the XLogRecord is not too large. * @@ -917,28 +995,11 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP * not emit records larger than the sizes advertised to be supported. This * cap is based on DecodeXLogRecordRequiredSpace(). */ - if (total_len >= XLogRecordMaxSize) + if (payload_len + rec_hdr_len >= XLogRecordMaxSize) ereport(ERROR, (errmsg_internal("oversized WAL record"), errdetail_internal("WAL record would be %llu bytes (of maximum %u bytes); rmid %u flags %u.", - (unsigned long long) total_len, XLogRecordMaxSize, rmid, info))); - - /* - * Fill in the fields in the record header. Prev-link is filled in later, - * once we know where in the WAL the record will be inserted. The CRC does - * not include the record header yet. - */ - if (curinsert_flags & XLOG_INCLUDE_XID) - rechdr->xl_xid = GetCurrentTransactionIdIfAny(); - else - rechdr->xl_xid = InvalidTransactionId; - - rechdr->xl_tot_len = (uint32) total_len; - rechdr->xl_info = info; - rechdr->xl_rmid = rmid; - rechdr->xl_rmgrinfo = rmgr_info; - rechdr->xl_prev = InvalidXLogRecPtr; - rechdr->xl_crc = rdata_crc; + (unsigned long long) payload_len, XLogRecordMaxSize, rmid, info))); return &hdr_rdt; } diff --git a/src/backend/access/transam/xlogreader.c b/src/backend/access/transam/xlogreader.c index 733de8a8dc..13da1a11c9 100644 --- a/src/backend/access/transam/xlogreader.c +++ b/src/backend/access/transam/xlogreader.c @@ -49,8 +49,9 @@ static int ReadPageInternal(XLogReaderState *state, XLogRecPtr pageptr, static void XLogReaderInvalReadState(XLogReaderState *state); static XLogPageReadResult XLogDecodeNextRecord(XLogReaderState *state, bool nonblocking); static bool ValidXLogRecordHeader(XLogReaderState *state, XLogRecPtr RecPtr, - XLogRecPtr PrevRecPtr, XLogRecord *record, bool randAccess); -static bool ValidXLogRecord(XLogReaderState *state, XLogRecord *record, + XLogRecPtr PrevRecPtr, XLogRecHdr rechdr, + bool randAccess); +static bool ValidXLogRecord(XLogReaderState *state, XLogRecHdr record, XLogRecPtr recptr); static void ResetDecoder(XLogReaderState *state); static void WALOpenSegmentInit(WALOpenSegment *seg, WALSegmentContext *segcxt, @@ -535,7 +536,7 @@ static XLogPageReadResult XLogDecodeNextRecord(XLogReaderState *state, bool nonblocking) { XLogRecPtr RecPtr; - XLogRecord *record; + XLogRecHdr record; XLogRecPtr targetPagePtr; bool randAccess; uint32 len, @@ -602,7 +603,7 @@ restart: * fits on the same page. */ readOff = ReadPageInternal(state, targetPagePtr, - Min(targetRecOff + SizeOfXLogRecord, XLOG_BLCKSZ)); + Min(targetRecOff + XLogRecordMaxHdrSize, XLOG_BLCKSZ)); if (readOff == XLREAD_WOULDBLOCK) return XLREAD_WOULDBLOCK; else if (readOff < 0) @@ -644,13 +645,13 @@ restart: * Read the record length. * * NB: Even though we use an XLogRecord pointer here, the whole record - * header might not fit on this page. xl_tot_len is the first field of the - * struct, so it must be on this page (the records are MAXALIGNed), but we - * cannot access any other fields until we've verified that we got the - * whole header. + * header might not fit on this page. xl_payload_len is the third field of + * the data and starts at the third byte, so it must be on this page (the + * records are aligned to 8 bytes), but we cannot access most other fields + * until we've verified that we got the whole header. */ - record = (XLogRecord *) (state->readBuf + RecPtr % XLOG_BLCKSZ); - total_len = record->xl_tot_len; + record = (XLogRecHdr) (state->readBuf + (RecPtr % XLOG_BLCKSZ)); + total_len = XLogRecordTotalLength(record); /* * If the whole record header is on this page, validate it immediately. @@ -660,7 +661,7 @@ restart: * record" code path below; otherwise we might fail to apply * ValidXLogRecordHeader at all. */ - if (targetRecOff <= XLOG_BLCKSZ - SizeOfXLogRecord) + if (targetRecOff <= XLOG_BLCKSZ - XLogRecordHdrLen(record->xl_info)) { if (!ValidXLogRecordHeader(state, RecPtr, state->DecodeRecPtr, record, randAccess)) @@ -670,12 +671,12 @@ restart: else { /* XXX: more validation should be done here */ - if (total_len < SizeOfXLogRecord) + if (total_len < XLogRecordMinHdrSize) { report_invalid_record(state, "invalid record length at %X/%X: expected at least %u, got %u", LSN_FORMAT_ARGS(RecPtr), - (uint32) SizeOfXLogRecord, total_len); + (uint32) XLogRecordMinHdrSize, total_len); goto err; } gotheader = false; @@ -818,7 +819,7 @@ restart: /* If we just reassembled the record header, validate it. */ if (!gotheader) { - record = (XLogRecord *) state->readRecordBuf; + record = (XLogRecHdr) state->readRecordBuf; if (!ValidXLogRecordHeader(state, RecPtr, state->DecodeRecPtr, record, randAccess)) goto err; @@ -828,7 +829,7 @@ restart: Assert(gotheader); - record = (XLogRecord *) state->readRecordBuf; + record = (XLogRecHdr) state->readRecordBuf; if (!ValidXLogRecord(state, record, RecPtr)) goto err; @@ -860,7 +861,7 @@ restart: * Special processing if it's an XLOG SWITCH record */ if (record->xl_rmid == RM_XLOG_ID && - record->xl_rmgrinfo == XLOG_SWITCH) + XLRHdrGetRmgrInfo(record) == XLOG_SWITCH) { /* Pretend it extends to end of segment */ state->NextRecPtr += state->segcxt.ws_segsize - 1; @@ -1114,22 +1115,25 @@ XLogReaderInvalReadState(XLogReaderState *state) */ static bool ValidXLogRecordHeader(XLogReaderState *state, XLogRecPtr RecPtr, - XLogRecPtr PrevRecPtr, XLogRecord *record, + XLogRecPtr PrevRecPtr, XLogRecHdr rechdr, bool randAccess) { - if (record->xl_tot_len < SizeOfXLogRecord) + XLogRecord record = {0}; + XLogReadRecHdrInto(rechdr, XLogRecordHdrLen(rechdr->xl_info), &record); + + if (XLogRecordTotalLength(rechdr) < XLogRecordMinHdrSize) { report_invalid_record(state, "invalid record length at %X/%X: expected at least %u, got %u", LSN_FORMAT_ARGS(RecPtr), - (uint32) SizeOfXLogRecord, record->xl_tot_len); + (uint32) XLogRecordMinHdrSize, record.xl_payload_len); return false; } - if (!RmgrIdIsValid(record->xl_rmid)) + if (!RmgrIdIsValid(rechdr->xl_rmid)) { report_invalid_record(state, "invalid resource manager ID %u at %X/%X", - record->xl_rmid, LSN_FORMAT_ARGS(RecPtr)); + rechdr->xl_rmid, LSN_FORMAT_ARGS(RecPtr)); return false; } if (randAccess) @@ -1138,12 +1142,14 @@ ValidXLogRecordHeader(XLogReaderState *state, XLogRecPtr RecPtr, * We can't exactly verify the prev-link, but surely it should be less * than the record's own address. */ - if (!(record->xl_prev < RecPtr)) + if (!(record.xl_prev < RecPtr)) { report_invalid_record(state, - "record with incorrect prev-link %X/%X at %X/%X", - LSN_FORMAT_ARGS(record->xl_prev), - LSN_FORMAT_ARGS(RecPtr)); + "record with incorrect prev-link %X/%X at %X/%X, info %u, rmgr %u", + LSN_FORMAT_ARGS(record.xl_prev), + LSN_FORMAT_ARGS(RecPtr), + (uint32) record.xl_info, + (uint32) record.xl_rmid); return false; } } @@ -1154,12 +1160,13 @@ ValidXLogRecordHeader(XLogReaderState *state, XLogRecPtr RecPtr, * check guards against torn WAL pages where a stale but valid-looking * WAL record starts on a sector boundary. */ - if (record->xl_prev != PrevRecPtr) + if (record.xl_prev != PrevRecPtr) { report_invalid_record(state, - "record with incorrect prev-link %X/%X at %X/%X", - LSN_FORMAT_ARGS(record->xl_prev), - LSN_FORMAT_ARGS(RecPtr)); + "record with incorrect prev-link %X/%X at %X/%X, expected %X/%X", + LSN_FORMAT_ARGS(record.xl_prev), + LSN_FORMAT_ARGS(RecPtr), + LSN_FORMAT_ARGS(PrevRecPtr)); return false; } } @@ -1179,19 +1186,26 @@ ValidXLogRecordHeader(XLogReaderState *state, XLogRecPtr RecPtr, * SizeOfXLogRecord. */ static bool -ValidXLogRecord(XLogReaderState *state, XLogRecord *record, XLogRecPtr recptr) +ValidXLogRecord(XLogReaderState *state, XLogRecHdr record, XLogRecPtr recptr) { - pg_crc32c crc; + pg_crc32c crc, + hdr_crc; + Size hdr_len = XLogRecordHdrLen(record->xl_info); + Size rec_length = XLogRecordTotalLength(record); /* Calculate the CRC */ INIT_CRC32C(crc); - COMP_CRC32C(crc, ((char *) record) + SizeOfXLogRecord, record->xl_tot_len - SizeOfXLogRecord); + COMP_CRC32C(crc, ((char *) record) + hdr_len, rec_length - hdr_len); /* include the record header last */ - COMP_CRC32C(crc, (char *) record, offsetof(XLogRecord, xl_crc)); + COMP_CRC32C(crc, (char *) record, hdr_len - sizeof(pg_crc32c)); FIN_CRC32C(crc); - if (!EQ_CRC32C(record->xl_crc, crc)) + memcpy(&hdr_crc, ((char *) record) + hdr_len - sizeof(pg_crc32c), + sizeof(pg_crc32c)); + + if (!EQ_CRC32C(hdr_crc, crc)) { + Assert(false); report_invalid_record(state, "incorrect resource manager data checksum in record at %X/%X", LSN_FORMAT_ARGS(recptr)); @@ -1651,7 +1665,7 @@ DecodeXLogRecordRequiredSpace(size_t xl_tot_len) bool DecodeXLogRecord(XLogReaderState *state, DecodedXLogRecord *decoded, - XLogRecord *record, + XLogRecHdr record, XLogRecPtr lsn, char **errormsg) { @@ -1674,7 +1688,7 @@ DecodeXLogRecord(XLogReaderState *state, RelFileLocator *rlocator = NULL; uint8 block_id; - decoded->header = *record; + XLogReadRecHdrInto(record, XLogRecordHdrLen(record->xl_info), &decoded->header); decoded->lsn = lsn; decoded->next = NULL; decoded->record_origin = InvalidRepOriginId; @@ -1683,8 +1697,8 @@ DecodeXLogRecord(XLogReaderState *state, decoded->main_data_len = 0; decoded->max_block_id = -1; ptr = (char *) record; - ptr += SizeOfXLogRecord; - remaining = record->xl_tot_len - SizeOfXLogRecord; + ptr += XLogRecordHdrLen(record->xl_info); + remaining = decoded->header.xl_payload_len; /* Decode the headers */ datatotal = 0; @@ -1960,7 +1974,7 @@ DecodeXLogRecord(XLogReaderState *state, /* Report the actual size we used. */ decoded->size = MAXALIGN(out - (char *) decoded); - Assert(DecodeXLogRecordRequiredSpace(record->xl_tot_len) >= + Assert(DecodeXLogRecordRequiredSpace(decoded->header.xl_payload_len) >= decoded->size); return true; @@ -1989,6 +2003,7 @@ XLogRecGetBlockTag(XLogReaderState *record, uint8 block_id, if (!XLogRecGetBlockTagExtended(record, block_id, rlocator, forknum, blknum, NULL)) { + Assert(false); #ifndef FRONTEND elog(ERROR, "could not locate backup block with ID %d in WAL record", block_id); diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index 56fa2f74a2..0362ba747f 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -4000,10 +4000,10 @@ ReadCheckpointRecord(XLogPrefetcher *xlogprefetcher, XLogRecPtr RecPtr, (errmsg("invalid xl_rmgrinfo in checkpoint record"))); return NULL; } - if (record->xl_tot_len != SizeOfXLogRecord + SizeOfXLogRecordDataHeaderShort + sizeof(CheckPoint)) + if (record->xl_payload_len != SizeOfXLogRecordDataHeaderShort + sizeof(CheckPoint)) { ereport(LOG, - (errmsg("invalid length of checkpoint record"))); + (errmsg("invalid payload length of checkpoint record"))); return NULL; } return record; diff --git a/src/bin/pg_resetwal/pg_resetwal.c b/src/bin/pg_resetwal/pg_resetwal.c index 80cb9a5bc4..4e171eed1f 100644 --- a/src/bin/pg_resetwal/pg_resetwal.c +++ b/src/bin/pg_resetwal/pg_resetwal.c @@ -1040,12 +1040,15 @@ WriteEmptyXLOG(void) PGAlignedXLogBlock buffer; XLogPageHeader page; XLogLongPageHeader longpage; - XLogRecord *record; pg_crc32c crc; char path[MAXPGPATH]; int fd; int nbytes; char *recptr; + char *baserecptr; + const int rec_hdr_len = offsetof(XLogRecHdrData, xl_hdrdata) + + sizeof(uint8) + sizeof(XLogRecPtr) + sizeof(pg_crc32c); + const int rec_payload_len = sizeof(uint8) * 2 + sizeof(CheckPoint); memset(buffer.data, 0, XLOG_BLCKSZ); @@ -1061,26 +1064,33 @@ WriteEmptyXLOG(void) longpage->xlp_xlog_blcksz = XLOG_BLCKSZ; /* Insert the initial checkpoint record */ - recptr = (char *) page + SizeOfXLogLongPHD; - record = (XLogRecord *) recptr; - record->xl_prev = 0; - record->xl_xid = InvalidTransactionId; - record->xl_tot_len = SizeOfXLogRecord + SizeOfXLogRecordDataHeaderShort + sizeof(CheckPoint); - record->xl_info = 0; - record->xl_rmgrinfo = XLOG_CHECKPOINT_SHUTDOWN; - record->xl_rmid = RM_XLOG_ID; - - recptr += SizeOfXLogRecord; + baserecptr = recptr = (char *) page + SizeOfXLogLongPHD; + *(recptr++) = (char) XLS_UINT8; + *(recptr++) = (char) RM_XLOG_ID; + recptr += XLogWriteLength(SizeOfXLogRecordDataHeaderShort + sizeof(CheckPoint), + XLS_UINT8, XLS_UINT8, recptr); + + /* include prevptr */ + { + XLogRecPtr prevptr = 0; + memcpy(recptr, &prevptr, sizeof(XLogRecPtr)); + recptr += sizeof(XLogRecPtr); + } + /* reserve location of crc */ + recptr += sizeof(pg_crc32c); + *(recptr++) = (char) XLR_BLOCK_ID_DATA_SHORT; *(recptr++) = sizeof(CheckPoint); memcpy(recptr, &ControlFile.checkPointCopy, sizeof(CheckPoint)); INIT_CRC32C(crc); - COMP_CRC32C(crc, ((char *) record) + SizeOfXLogRecord, record->xl_tot_len - SizeOfXLogRecord); - COMP_CRC32C(crc, (char *) record, offsetof(XLogRecord, xl_crc)); + COMP_CRC32C(crc, baserecptr + rec_hdr_len, rec_payload_len); + COMP_CRC32C(crc, baserecptr, rec_hdr_len - sizeof(pg_crc32c)); FIN_CRC32C(crc); - record->xl_crc = crc; + + memcpy(baserecptr + rec_hdr_len - sizeof(pg_crc32c), + &crc, sizeof(pg_crc32c)); /* Write the first page */ XLogFilePath(path, ControlFile.checkPointCopy.ThisTimeLineID, diff --git a/src/include/access/rmgr.h b/src/include/access/rmgr.h index 3b6a497e1b..7038ba0833 100644 --- a/src/include/access/rmgr.h +++ b/src/include/access/rmgr.h @@ -41,7 +41,7 @@ typedef enum RmgrIds static inline bool RmgrIdIsBuiltin(int rmid) { - return rmid <= RM_MAX_BUILTIN_ID; + return rmid <= RM_MAX_BUILTIN_ID && rmid != RM_INVALID_ID; } static inline bool diff --git a/src/include/access/rmgrlist.h b/src/include/access/rmgrlist.h index 463bcb67c5..b53d731f56 100644 --- a/src/include/access/rmgrlist.h +++ b/src/include/access/rmgrlist.h @@ -25,6 +25,7 @@ */ /* symbol name, textual name, redo, desc, identify, startup, cleanup, mask, decode */ +PG_RMGR(RM_INVALID_ID, "invalid", NULL, NULL, NULL, NULL, NULL, NULL, NULL) PG_RMGR(RM_XLOG_ID, "XLOG", xlog_redo, xlog_desc, xlog_identify, NULL, NULL, NULL, xlog_decode) PG_RMGR(RM_XACT_ID, "Transaction", xact_redo, xact_desc, xact_identify, NULL, NULL, NULL, xact_decode) PG_RMGR(RM_SMGR_ID, "Storage", smgr_redo, smgr_desc, smgr_identify, NULL, NULL, NULL, NULL) diff --git a/src/include/access/xlog_internal.h b/src/include/access/xlog_internal.h index 876e2790f9..ea32a9ba1b 100644 --- a/src/include/access/xlog_internal.h +++ b/src/include/access/xlog_internal.h @@ -26,12 +26,20 @@ #include "pgtime.h" #include "storage/block.h" #include "storage/relfilelocator.h" +#include "transam.h" +/* + * WAL records (and other XLogPage page-level content) are aligned to 8 bytes. + * + * Note that the contents of the records is not aligned (!) + */ +#define ALIGNOF_XLP_CONTENT 8 +#define XLP_ALIGN(LEN) TYPEALIGN64(ALIGNOF_XLP_CONTENT, LEN) /* * Each page of XLOG file has a header like this: */ -#define XLOG_PAGE_MAGIC 0xD113 /* can be used as WAL version indicator */ +#define XLOG_PAGE_MAGIC 0xD114 /* can be used as WAL version indicator */ typedef struct XLogPageHeaderData { @@ -49,7 +57,7 @@ typedef struct XLogPageHeaderData uint32 xlp_rem_len; /* total len of remaining data for record */ } XLogPageHeaderData; -#define SizeOfXLogShortPHD MAXALIGN(sizeof(XLogPageHeaderData)) +#define SizeOfXLogShortPHD XLP_ALIGN(sizeof(XLogPageHeaderData)) typedef XLogPageHeaderData *XLogPageHeader; @@ -66,7 +74,7 @@ typedef struct XLogLongPageHeaderData uint32 xlp_xlog_blcksz; /* just as a cross-check */ } XLogLongPageHeaderData; -#define SizeOfXLogLongPHD MAXALIGN(sizeof(XLogLongPageHeaderData)) +#define SizeOfXLogLongPHD XLP_ALIGN(sizeof(XLogLongPageHeaderData)) typedef XLogLongPageHeaderData *XLogLongPageHeader; @@ -381,7 +389,7 @@ static inline Size XLogWriteLength(uint32 length, XLogSizeClass sizeClass, case caseSizeClass: \ if ((caseSizeClass) <= maxSizeClass) \ { \ - field_type typedLength = length; \ + field_type typedLength = (field_type) length; \ memcpy(output, &typedLength, sizeof(field_type)); \ written = sizeof(field_type); \ } \ @@ -394,6 +402,9 @@ static inline Size XLogWriteLength(uint32 length, XLogSizeClass sizeClass, WRITE_OP(XLS_UINT8, uint8); WRITE_OP(XLS_UINT16, uint16); WRITE_OP(XLS_UINT32, uint32); + default: + Assert(false); + pg_unreachable(); } #undef WRITE_OP @@ -436,12 +447,95 @@ static inline Size XLogReadLength(uint32 *length, XLogSizeClass sizeClass, READ_OP(XLS_UINT8, uint8); READ_OP(XLS_UINT16, uint16); READ_OP(XLS_UINT32, uint32); + default: + Assert(false); + pg_unreachable(); } #undef READ_OP return readSize; } + +inline static uint8 XLRHdrGetRmgrInfo(XLogRecHdr record) +{ + XLogSizeClass recSizeClass; + int offset; + + if (!(record->xl_info & XLR_HAS_RMGRINFO)) + return 0; + + recSizeClass = XLR_SIZECLASS(record->xl_info); + /* xl_rmgrinfo is located immediately behind the xl_payload_len field */ + offset = XLogSizeClassToByteLength(recSizeClass); + + return (uint8) record->xl_hdrdata[offset]; +} + + +/* Works on any partial record */ +inline static Size XLogRecordTotalLength(XLogRecHdr record) +{ + uint8 xl_info = record->xl_info; + XLogSizeClass sizeClass; + uint32 length = 0; + sizeClass = XLR_SIZECLASS(xl_info); + + XLogReadLength(&length, sizeClass, XLS_UINT32, + &record->xl_hdrdata[0], 6); + + return (Size) length + XLogRecordHdrLen(xl_info); +} + +inline static void XLogReadRecHdrInto(XLogRecHdr recdata, Size length, + XLogRecord *record) +{ + Size offset = 0; + Size hdr_size PG_USED_FOR_ASSERTS_ONLY = 0; + XLogSizeClass sizeClass; + + Assert(length >= XLogRecordMinHdrSize); + + record->xl_info = recdata->xl_info; + + hdr_size = XLogRecordHdrLen(record->xl_info); + Assert(length >= hdr_size); + + record->xl_rmid = recdata->xl_rmid; + + sizeClass = XLR_SIZECLASS(record->xl_info); + offset += XLogReadLength(&record->xl_payload_len, sizeClass, + XLS_UINT32, &recdata->xl_hdrdata[offset], length - offset); + + if (record->xl_info & XLR_HAS_RMGRINFO) + { + record->xl_rmgrinfo = recdata->xl_hdrdata[offset]; + offset += sizeof(uint8); + } + else + { + record->xl_rmgrinfo = 0; + } + + if (record->xl_info & XLR_HAS_XID) + { + memcpy(&record->xl_xid, &recdata->xl_hdrdata[offset], sizeof(TransactionId)); + offset += sizeof(TransactionId); + } + else + { + record->xl_xid = InvalidTransactionId; + } + + memcpy(&record->xl_prev, &recdata->xl_hdrdata[offset], sizeof(XLogRecPtr)); + offset += sizeof(XLogRecPtr); + + memcpy(&record->xl_crc, &recdata->xl_hdrdata[offset], sizeof(pg_crc32c)); + offset += sizeof(pg_crc32c); + + Assert(hdr_size - 2 == offset); +} + /* * Method table for resource managers. * diff --git a/src/include/access/xlogreader.h b/src/include/access/xlogreader.h index a1d0216404..bb64f89dba 100644 --- a/src/include/access/xlogreader.h +++ b/src/include/access/xlogreader.h @@ -397,7 +397,7 @@ extern bool WALRead(XLogReaderState *state, extern size_t DecodeXLogRecordRequiredSpace(size_t xl_tot_len); extern bool DecodeXLogRecord(XLogReaderState *state, DecodedXLogRecord *decoded, - XLogRecord *record, + XLogRecHdr record, XLogRecPtr lsn, char **errormsg); @@ -405,7 +405,7 @@ extern bool DecodeXLogRecord(XLogReaderState *state, * Macros that provide access to parts of the record most recently returned by * XLogReadRecord() or XLogNextRecord(). */ -#define XLogRecGetTotalLen(decoder) ((decoder)->record->header.xl_tot_len) +#define XLogRecGetTotalLen(decoder) ((decoder)->record->header.xl_payload_len) #define XLogRecGetPrev(decoder) ((decoder)->record->header.xl_prev) #define XLogRecGetInfo(decoder) ((decoder)->record->header.xl_info) #define XLogRecGetRmgrInfo(decoder) ((decoder)->record->header.xl_rmgrinfo) diff --git a/src/include/access/xlogrecord.h b/src/include/access/xlogrecord.h index 42b06f163e..af615e2866 100644 --- a/src/include/access/xlogrecord.h +++ b/src/include/access/xlogrecord.h @@ -38,9 +38,38 @@ typedef enum XLogSizeClass { XLS_UINT32 = 3 /* length <= UINT32_MAX; stored in uint32 (4B) */ } XLogSizeClass; +static inline int XLogSizeClassToByteLength(XLogSizeClass sz) +{ + switch (sz) { + case XLS_EMPTY: + return 0; + case XLS_UINT8: + return sizeof(uint8); + case XLS_UINT16: + return sizeof(uint16); + case XLS_UINT32: + return sizeof(uint32); + default: + pg_unreachable(); + } +} + +typedef struct XLogRecord +{ + uint8 xl_info; /* flag bits, see below */ + RmgrId xl_rmid; /* resource manager for this record */ + uint32 xl_payload_len; /* total len of entire record */ + uint8 xl_rmgrinfo; /* rmgr flag bits, see below */ + TransactionId xl_xid; /* xact id */ + XLogRecPtr xl_prev; /* ptr to previous record in log */ + pg_crc32c xl_crc; /* CRC for this record */ +} XLogRecord; + +#define SizeOfXLogRecord (offsetof(XLogRecord, xl_crc) + sizeof(pg_crc32c)) + /* * The overall layout of an XLOG record is: - * Fixed-size header (XLogRecord struct) + * Variable-size header (containing the data of the XLogRecord struct) * XLogRecordBlockHeader struct * XLogRecordBlockHeader struct * ... @@ -50,6 +79,15 @@ typedef enum XLogSizeClass { * ... * main data * + * Different xlog record headers need to store different header fields, so + * depending on flags in the xl_info field the layout may change. + * + * Note that for records with a payload larger than 0xFFFFFFF, the size field + * will be 4 bytes long, which can be larger than the minimum MAXALIGN quantum. + * To still be able to read the record correctly, we always align WAL to 8 + * bytes, so that the length bytes are always present in the first aligned + * quantum of data. + * * There can be zero or more XLogRecordBlockHeaders, and 0 or more bytes of * rmgr-specific data not associated with a block. XLogRecord structs * always start on MAXALIGN boundaries in the WAL files, but the rest of @@ -59,22 +97,25 @@ typedef enum XLogSizeClass { * XLogRecordDataHeaderLong structs all begin with a single 'id' byte. It's * used to distinguish between block references, and the main data structs. */ -typedef struct XLogRecord +typedef struct XLogRecHdrData { - uint32 xl_tot_len; /* total len of entire record */ - TransactionId xl_xid; /* xact id */ - XLogRecPtr xl_prev; /* ptr to previous record in log */ uint8 xl_info; /* flag bits, see below */ RmgrId xl_rmid; /* resource manager for this record */ - uint8 xl_rmgrinfo; /* rmgr flag bits, see below */ - /* 1 byte of padding here, initialize to zero */ - pg_crc32c xl_crc; /* CRC for this record */ - - /* XLogRecordBlockHeaders and XLogRecordDataHeader follow, no padding */ + char xl_hdrdata[FLEXIBLE_ARRAY_MEMBER]; /* variable length record data */ -} XLogRecord; + /* + * Payload is as follows, in order: + * XLogRecord data encoded + * xl_payload_len; 0, 1, 2 or 4 bytes + * xl_rmgrinfo; 0 or 1 byte + * xl_xid; 0 or 4 bytes + * xl_prev; 8 bytes + * xl_prev follows; 8 bytes + * record payload, if present. + */ +} XLogRecHdrData; -#define SizeOfXLogRecord (offsetof(XLogRecord, xl_crc) + sizeof(pg_crc32c)) +typedef XLogRecHdrData *XLogRecHdr; /* * XLogReader needs to allocate all the data of a WAL record in a single @@ -87,13 +128,28 @@ typedef struct XLogRecord */ #define XLogRecordMaxSize (1020 * 1024 * 1024) +#define XLR_SIZECLASS_MASK 0x03 +#define XLR_SIZECLASS(xl_info) ((XLogSizeClass) ((xl_info) & XLR_SIZECLASS_MASK)) + +/* + * There are several rmgrs which don't use info bits, so we omit that + * byte whenever possible. + */ +#define XLR_HAS_RMGRINFO 0x04 + +/* + * If a WAL record uses the current transaction ID, that will be included + * in the record header. In all other cases we omit the XID to save bytes. + */ +#define XLR_HAS_XID 0x08 + /* * If a WAL record modifies any relation files, in ways not covered by the * usual block references, this flag is set. This is not used for anything * by PostgreSQL itself, but it allows external tools that read WAL and keep * track of modified blocks to recognize such special record types. */ -#define XLR_SPECIAL_REL_UPDATE 0x01 +#define XLR_SPECIAL_REL_UPDATE 0x10 /* * Enforces consistency checks of replayed WAL at recovery. If enabled, @@ -102,7 +158,48 @@ typedef struct XLogRecord * of XLogInsert can use this value if necessary, but if * wal_consistency_checking is enabled for a rmgr this is set unconditionally. */ -#define XLR_CHECK_CONSISTENCY 0x02 +#define XLR_CHECK_CONSISTENCY (0x20) + +#define XLogRecordMaxHdrSize ( \ + sizeof(uint8) /* xl_info */ + \ + sizeof(uint8) /* xl_rmid */ + \ + sizeof(uint32) /* xl_payload_len */ + \ + sizeof(uint8) /* xl_rmgrinfo */ + \ + sizeof(TransactionId) /* xl_xid */ + \ + sizeof(XLogRecPtr) /* xl_prev */ + \ + sizeof(pg_crc32c) /* xl_crc */ \ +) + +#define XLogRecordMinHdrSize ( \ + sizeof(uint8) /* xl_info */ + \ + sizeof(uint8) /* xl_rmid */ + \ + 0 /* xl_payload_len */ + \ + 0 /* xl_rmgrinfo */ + \ + 0 /* xl_xid */ + \ + sizeof(XLogRecPtr) /* xl_prev */ + \ + sizeof(pg_crc32c) /* xl_crc */ \ +) + +static inline Size XLogRecordHdrLen(uint8 info) +{ + Size size = 0; + /* xl_info */ + size += sizeof(uint8); + /* xl_rmid */ + size += sizeof(uint8); + /* xl_payload_len */ + size += XLogSizeClassToByteLength(XLR_SIZECLASS(info)); + /* xl_rmgrinfo */ + size += ((info) & XLR_HAS_RMGRINFO) ? sizeof(uint8) : 0; + /* xl_xid */ + size += ((info) & XLR_HAS_XID) ? sizeof(TransactionId) : 0; + /* xl_prev */ + size += sizeof(XLogRecPtr); + /* xl_crc */ + size += sizeof(pg_crc32c); + + return size; +} /* * Header info for block data appended to an XLOG record. -- 2.40.1 ^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: XLog size reductions: Reduced XLog record header size for PG17 @ 2023-07-03 11:08 Matthias van de Meent <[email protected]> parent: Matthias van de Meent <[email protected]> 0 siblings, 1 reply; 35+ messages in thread From: Matthias van de Meent @ 2023-07-03 11:08 UTC (permalink / raw) To: pgsql-hackers; +Cc: Heikki Linnakangas <[email protected]>; Robert Haas <[email protected]>; Dilip Kumar <[email protected]>; Andres Freund <[email protected]> On Fri, 30 Jun 2023 at 17:36, Matthias van de Meent <[email protected]> wrote: > > Hi, > > The attached v2 patchset contains some small fixes for the failing > cfbot 32-bit tests - at least locally it does so. > > I'd overlooked one remaining use of MAXALIGN64() in xlog.c in the last > patch of the set, which has now been updated to XLP_ALIGN as well. > Additionally, XLP_ALIGN has been updated to use TYPEALIGN64 instead of > TYPEALIGN so that we don't lose bits of the aligned value in 32-bit > systems. Apparently there was some usage of MAXALIGN() in xlogreader that I'd missed, and which only shows up in TAP tests. In v3 I've fixed that, together with some improved early detection of invalid record headers. Kind regards, Matthias van de Meent Neon (https://neon.tech/) Attachments: [application/octet-stream] v3-0005-Stop-the-logging-of-XIDs-in-records-where-the-XID.patch (10.4K, ../../CAEze2Wh2e57adcGXWJLo4hRQY-__CmWwaaSbUf9PGGOX5ESrmQ@mail.gmail.com/2-v3-0005-Stop-the-logging-of-XIDs-in-records-where-the-XID.patch) download | inline diff: From 56fdf28a10d531984b1765f3bd6ae3fbee8c0006 Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Wed, 25 Jan 2023 15:40:30 +0100 Subject: [PATCH v3 5/6] Stop the logging of XIDs in records where the XID is unused. Many WAL record types do not need to contain an XID. This change prepares for the removal of the xid from the main xlog record header, by requiring the user of the XLog insertion API to explicitly request the inclusion of the current xid in the log record - in all other cases we'll log, assume and track InvalidXid instead. --- src/backend/access/heap/heapam.c | 16 +++++++++------- src/backend/access/heap/pruneheap.c | 1 + src/backend/access/heap/rewriteheap.c | 1 + src/backend/access/transam/generic_xlog.c | 3 +++ src/backend/access/transam/multixact.c | 1 + src/backend/access/transam/xact.c | 4 ++-- src/backend/access/transam/xloginsert.c | 12 ++++++++++-- src/backend/commands/tablecmds.c | 2 +- src/backend/replication/logical/message.c | 2 +- src/backend/utils/cache/inval.c | 1 + src/include/access/xlog.h | 1 + 11 files changed, 31 insertions(+), 13 deletions(-) diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 1512f22c76..30b7b05f54 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -1969,7 +1969,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid, heaptup->t_len - SizeofHeapTupleHeader); /* filtering by origin on a row level is much more efficient */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP_ID, info); @@ -2365,7 +2365,7 @@ heap_multi_insert(Relation relation, TupleTableSlot **slots, int ntuples, XLogRegisterBufData(0, tupledata, totaldatalen); /* filtering by origin on a row level is much more efficient */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP2_ID, info); @@ -2865,7 +2865,7 @@ l1: } /* filtering by origin on a row level is much more efficient */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP_ID, XLOG_HEAP_DELETE); @@ -5535,6 +5535,7 @@ l4: cleared_all_frozen ? XLH_LOCK_ALL_FROZEN_CLEARED : 0; XLogRegisterData((char *) &xlrec, SizeOfHeapLockUpdated); + XLogSetRecordFlags(XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP2_ID, XLOG_HEAP2_LOCK_UPDATED); @@ -5685,7 +5686,7 @@ heap_finish_speculative(Relation relation, ItemPointer tid) XLogBeginInsert(); /* We want the same filtering on this as on a plain insert */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); XLogRegisterData((char *) &xlrec, SizeOfHeapConfirm); XLogRegisterBuffer(0, buffer, REGBUF_STANDARD); @@ -5831,7 +5832,8 @@ heap_abort_speculative(Relation relation, ItemPointer tid) XLogRegisterData((char *) &xlrec, SizeOfHeapDelete); XLogRegisterBuffer(0, buffer, REGBUF_STANDARD); - /* No replica identity & replication origin logged */ + /* No replica identity & replication origin logged, but XID is required */ + XLogSetRecordFlags(XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP_ID, XLOG_HEAP_DELETE); @@ -8563,7 +8565,7 @@ log_heap_update(Relation reln, Buffer oldbuf, } /* filtering by origin on a row level is much more efficient */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP_ID, info); @@ -8638,7 +8640,7 @@ log_heap_new_cid(Relation relation, HeapTuple tup) XLogRegisterData((char *) &xlrec, SizeOfHeapNewCid); /* will be looked at irrespective of origin */ - + XLogSetRecordFlags(XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP2_ID, XLOG_HEAP2_NEW_CID); return recptr; diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index 47b9e20915..24e1bbe1ad 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -445,6 +445,7 @@ heap_page_prune(Relation relation, Buffer buffer, XLogRegisterBufData(0, (char *) prstate.nowunused, prstate.nunused * sizeof(OffsetNumber)); + XLogSetRecordFlags(XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP2_ID, XLOG_HEAP2_PRUNE); PageSetLSN(BufferGetPage(buffer), recptr); diff --git a/src/backend/access/heap/rewriteheap.c b/src/backend/access/heap/rewriteheap.c index 424958912c..9f77537335 100644 --- a/src/backend/access/heap/rewriteheap.c +++ b/src/backend/access/heap/rewriteheap.c @@ -921,6 +921,7 @@ logical_heap_rewrite_flush_mappings(RewriteState state) src->off += len; XLogBeginInsert(); + XLogSetRecordFlags(XLOG_INCLUDE_XID); XLogRegisterData((char *) (&xlrec), sizeof(xlrec)); XLogRegisterData(waldata_start, len); diff --git a/src/backend/access/transam/generic_xlog.c b/src/backend/access/transam/generic_xlog.c index 6c68191ca6..c8983b40ad 100644 --- a/src/backend/access/transam/generic_xlog.c +++ b/src/backend/access/transam/generic_xlog.c @@ -399,6 +399,9 @@ GenericXLogFinish(GenericXLogState *state) } } + /* Assume XID is required */ + XLogSetRecordFlags(XLOG_INCLUDE_XID); + /* Insert xlog record */ lsn = XLogInsert(RM_GENERIC_ID, 0); diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c index c70b6b7358..883da6a33b 100644 --- a/src/backend/access/transam/multixact.c +++ b/src/backend/access/transam/multixact.c @@ -834,6 +834,7 @@ MultiXactIdCreateFromMembers(int nmembers, MultiXactMember *members) * Not clear that it's worth the trouble though. */ XLogBeginInsert(); + XLogSetRecordFlags(XLOG_INCLUDE_XID); XLogRegisterData((char *) (&xlrec), SizeOfMultiXactCreate); XLogRegisterData((char *) members, nmembers * sizeof(MultiXactMember)); diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index da0ac6c8d3..57504594b0 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -5793,7 +5793,7 @@ XactLogCommitRecord(TimestampTz commit_time, XLogRegisterData((char *) (&xl_origin), sizeof(xl_xact_origin)); /* we allow filtering by xacts */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); return XLogInsertExtended(RM_XACT_ID, info, rmgr_info); } @@ -5940,7 +5940,7 @@ XactLogAbortRecord(TimestampTz abort_time, XLogRegisterData((char *) (&xl_origin), sizeof(xl_xact_origin)); /* Include the replication origin */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); return XLogInsertExtended(RM_XACT_ID, info, rmgr_info); } diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c index 9a3eb4fb4b..4816c5284b 100644 --- a/src/backend/access/transam/xloginsert.c +++ b/src/backend/access/transam/xloginsert.c @@ -456,6 +456,10 @@ XLogSetRecordFlags(uint8 flags) * (LSN is the XLOG point up to which the XLOG must be flushed to disk * before the data page can be written out. This implements the basic * WAL rule "write the log before the data".) + * + * Note: To include the current backend's TransactionID in the record, + * you have to set the XLOG_INCLUDE_XID flag using XLogRecordSetFlags + * before calling XLogInsert. */ XLogRecPtr XLogInsert(RmgrId rmid, uint8 rmgr_info) @@ -847,7 +851,7 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP } /* followed by toplevel XID, if not already included in previous record */ - if (IsSubxactTopXidLogPending()) + if (curinsert_flags & XLOG_INCLUDE_XID && IsSubxactTopXidLogPending()) { TransactionId xid = GetTopTransactionIdIfAny(); @@ -924,7 +928,11 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP * once we know where in the WAL the record will be inserted. The CRC does * not include the record header yet. */ - rechdr->xl_xid = GetCurrentTransactionIdIfAny(); + if (curinsert_flags & XLOG_INCLUDE_XID) + rechdr->xl_xid = GetCurrentTransactionIdIfAny(); + else + rechdr->xl_xid = InvalidTransactionId; + rechdr->xl_tot_len = (uint32) total_len; rechdr->xl_info = info; rechdr->xl_rmid = rmid; diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index d985278ac6..264a0399b4 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -2132,7 +2132,7 @@ ExecuteTruncateGuts(List *explicit_rels, XLogRegisterData((char *) &xlrec, SizeOfHeapTruncate); XLogRegisterData((char *) logrelids, list_length(relids_logged) * sizeof(Oid)); - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); (void) XLogInsert(RM_HEAP_ID, XLOG_HEAP_TRUNCATE); } diff --git a/src/backend/replication/logical/message.c b/src/backend/replication/logical/message.c index c31dc1203f..af86d18457 100644 --- a/src/backend/replication/logical/message.c +++ b/src/backend/replication/logical/message.c @@ -69,7 +69,7 @@ LogLogicalMessage(const char *prefix, const char *message, size_t size, XLogRegisterData(unconstify(char *, message), size); /* allow origin filtering */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); return XLogInsert(RM_LOGICALMSG_ID, XLOG_LOGICAL_MESSAGE); } diff --git a/src/backend/utils/cache/inval.c b/src/backend/utils/cache/inval.c index 0008826f67..b0afdd6f90 100644 --- a/src/backend/utils/cache/inval.c +++ b/src/backend/utils/cache/inval.c @@ -1632,6 +1632,7 @@ LogLogicalInvalidations(void) ProcessMessageSubGroupMulti(group, RelCacheMsgs, XLogRegisterData((char *) msgs, n * sizeof(SharedInvalidationMessage))); + XLogSetRecordFlags(XLOG_INCLUDE_XID); XLogInsert(RM_XACT_ID, XLOG_XACT_INVALIDATIONS); } } diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h index 9007f559f3..f4dfaeda29 100644 --- a/src/include/access/xlog.h +++ b/src/include/access/xlog.h @@ -150,6 +150,7 @@ extern PGDLLIMPORT bool XLOG_DEBUG; */ #define XLOG_INCLUDE_ORIGIN 0x01 /* include the replication origin */ #define XLOG_MARK_UNIMPORTANT 0x02 /* record not important for durability */ +#define XLOG_INCLUDE_XID 0x04 /* include the transaction ID */ /* Checkpoint statistics */ -- 2.40.1 [application/octet-stream] v3-0001-Reduce-overhead-of-small-block-data-in-xlog-recor.patch (10.9K, ../../CAEze2Wh2e57adcGXWJLo4hRQY-__CmWwaaSbUf9PGGOX5ESrmQ@mail.gmail.com/3-v3-0001-Reduce-overhead-of-small-block-data-in-xlog-recor.patch) download | inline diff: From 459dc36fde0dd0b3516d4d544aac6cd45a7a0458 Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Tue, 24 Jan 2023 12:37:09 +0100 Subject: [PATCH v3 1/6] Reduce overhead of small block data in xlog records We reserve 2 bits in the block ID header for how large the block's registered data is: 0, 1 or 2 bytes, for 0, 1-255 and 256-UINT16_MAX bytes to the block, respectively. This reduces the minimum size of any block-referencing xlog record from 48 bytes to 46 bytes, and reduces the size of any non-hot heap update record by at least 2 bytes. It includes more infrastructure for future value-dependent encoding of length fields, even though it is currently only used in one place. --- src/backend/access/transam/xloginsert.c | 13 ++- src/backend/access/transam/xlogreader.c | 34 +++++-- src/include/access/xlog_internal.h | 114 ++++++++++++++++++++++++ src/include/access/xlogrecord.h | 43 +++++++-- 4 files changed, 188 insertions(+), 16 deletions(-) diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c index 54247e1d81..0fce063ca4 100644 --- a/src/backend/access/transam/xloginsert.c +++ b/src/backend/access/transam/xloginsert.c @@ -581,6 +581,8 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, XLogRecordBlockHeader bkpb; XLogRecordBlockImageHeader bimg; XLogRecordBlockCompressHeader cbimg = {0}; + int data_length = 0; + XLogSizeClass data_sizeclass = XLS_EMPTY; bool samerel; bool is_compressed = false; bool include_image; @@ -622,7 +624,7 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, bkpb.id = block_id; bkpb.fork_flags = regbuf->forkno; - bkpb.data_length = 0; + data_length = 0; if ((regbuf->flags & REGBUF_WILL_INIT) == REGBUF_WILL_INIT) bkpb.fork_flags |= BKPBLOCK_WILL_INIT; @@ -785,13 +787,16 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, * overall list. */ bkpb.fork_flags |= BKPBLOCK_HAS_DATA; - bkpb.data_length = (uint16) regbuf->rdata_len; + data_length = (uint16) regbuf->rdata_len; + data_sizeclass = XLogLengthToSizeClass(data_length, XLS_UINT16); total_len += regbuf->rdata_len; rdt_datas_last->next = regbuf->rdata_head; rdt_datas_last = regbuf->rdata_tail; } + bkpb.id |= data_sizeclass << XLR_BLOCKID_SZCLASS_SHIFT; + if (prev_regbuf && RelFileLocatorEquals(regbuf->rlocator, prev_regbuf->rlocator)) { samerel = true; @@ -804,6 +809,10 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, /* Ok, copy the header to the scratch buffer */ memcpy(scratch, &bkpb, SizeOfXLogRecordBlockHeader); scratch += SizeOfXLogRecordBlockHeader; + + scratch += XLogWriteLength(data_length, data_sizeclass, + XLS_UINT16, scratch); + if (include_image) { memcpy(scratch, &bimg, SizeOfXLogRecordBlockImageHeader); diff --git a/src/backend/access/transam/xlogreader.c b/src/backend/access/transam/xlogreader.c index 2e7b1ba8e1..24c6c88b2f 100644 --- a/src/backend/access/transam/xlogreader.c +++ b/src/backend/access/transam/xlogreader.c @@ -1660,11 +1660,11 @@ DecodeXLogRecord(XLogReaderState *state, */ #define COPY_HEADER_FIELD(_dst, _size) \ do { \ - if (remaining < _size) \ + if (remaining < (_size)) \ goto shortdata_err; \ - memcpy(_dst, ptr, _size); \ - ptr += _size; \ - remaining -= _size; \ + memcpy((_dst), ptr, (_size)); \ + ptr += (_size); \ + remaining -= (_size); \ } while(0) char *ptr; @@ -1690,8 +1690,13 @@ DecodeXLogRecord(XLogReaderState *state, datatotal = 0; while (remaining > datatotal) { + XLogSizeClass sizeClass; COPY_HEADER_FIELD(&block_id, sizeof(uint8)); + sizeClass = (block_id & XLR_BLOCKID_SZCLASS_MASK) >> + XLR_BLOCKID_SZCLASS_SHIFT; + block_id &= XLR_BLOCK_ID_MASK; + if (block_id == XLR_BLOCK_ID_DATA_SHORT) { /* XLogRecordDataHeaderShort */ @@ -1755,7 +1760,26 @@ DecodeXLogRecord(XLogReaderState *state, blk->prefetch_buffer = InvalidBuffer; - COPY_HEADER_FIELD(&blk->data_len, sizeof(uint16)); + { + Size read; + uint32 length = 0; + read = XLogReadLength(&length, sizeClass, + XLS_UINT16, + ptr, remaining); + + if (read < 0) + { + report_invalid_record(state, + "Could not read length from record at %X/%X", + LSN_FORMAT_ARGS(state->ReadRecPtr)); + goto err; + } + + ptr += read; + remaining -= read; + blk->data_len = length; + } + /* cross-check that the HAS_DATA flag is set iff data_length > 0 */ if (blk->has_data && blk->data_len == 0) { diff --git a/src/include/access/xlog_internal.h b/src/include/access/xlog_internal.h index b0fd338a00..42b3c66547 100644 --- a/src/include/access/xlog_internal.h +++ b/src/include/access/xlog_internal.h @@ -328,6 +328,120 @@ typedef enum struct LogicalDecodingContext; struct XLogRecordBuffer; +/* + * XLogSizeClass interactions + */ + +/* + * XLogLengthToSizeClass + * Turns a length into a size class + * MaxSizeClass is used to allow the compiler to eliminate branches that + * can never be hit in the caller's code path; e.g. when processing uint16. + */ +static inline XLogSizeClass XLogLengthToSizeClass(uint32 length, + const XLogSizeClass maxSizeClass) +{ + XLogSizeClass sizeClass; + + if (length == 0) + sizeClass = XLS_EMPTY; + else if (length <= UINT8_MAX && maxSizeClass >= XLS_UINT8) + sizeClass = XLS_UINT8; + else if (length <= UINT16_MAX && maxSizeClass >= XLS_UINT16) + sizeClass = XLS_UINT16; + else + { + Assert(maxSizeClass == XLS_UINT32); + sizeClass = XLS_UINT32; + } + + Assert(sizeClass <= maxSizeClass); + + return sizeClass; +} + +/* + * XLogWriteLength + * + * Write a length with size determined by sizeClass into the output. + * Returns the Size of bytes written. The user is responsible for making + * sure that out-of-bounds write is impossible. + * Writes at most 4 bytes. + */ +static inline Size XLogWriteLength(uint32 length, XLogSizeClass sizeClass, + const XLogSizeClass maxSizeClass, + char *output) +{ + Size written = -1; + + Assert(sizeClass <= maxSizeClass && + XLogLengthToSizeClass(length, maxSizeClass) == sizeClass); + +#define WRITE_OP(caseSizeClass, field_type) \ + case caseSizeClass: \ + if ((caseSizeClass) <= maxSizeClass) \ + { \ + field_type typedLength = length; \ + memcpy(output, &typedLength, sizeof(field_type)); \ + written = sizeof(field_type); \ + } \ + break + + switch (sizeClass) { + case XLS_EMPTY: + written = 0; + break; + WRITE_OP(XLS_UINT8, uint8); + WRITE_OP(XLS_UINT16, uint16); + WRITE_OP(XLS_UINT32, uint32); + } + +#undef WRITE_OP + return written; +} + +/* + * XLogReadLength + * + * Read a length with size determined by sizeClass from the input into + * *length. Returns the Size of bytes read, or -1 if the input size was + * too small to read the relevant length field. + */ +static inline Size XLogReadLength(uint32 *length, XLogSizeClass sizeClass, + const XLogSizeClass maxSizeClass, + char *input, Size inSize) +{ + Size readSize = -1; + + Assert(sizeClass <= maxSizeClass); + +#define READ_OP(caseSizeClass, field_type) \ + case caseSizeClass: \ + if ((caseSizeClass) <= maxSizeClass) \ + { \ + field_type typedLength; \ + if (inSize < sizeof(field_type)) \ + return -1; \ + memcpy(&typedLength, input, sizeof(field_type)); \ + readSize = sizeof(field_type); \ + *length = typedLength; \ + } \ + break + + switch (sizeClass) { + case XLS_EMPTY: + readSize = 0; + *length = 0; + break; + READ_OP(XLS_UINT8, uint8); + READ_OP(XLS_UINT16, uint16); + READ_OP(XLS_UINT32, uint32); + } + +#undef READ_OP + return readSize; +} + /* * Method table for resource managers. * diff --git a/src/include/access/xlogrecord.h b/src/include/access/xlogrecord.h index f355e08e1d..b0aada1031 100644 --- a/src/include/access/xlogrecord.h +++ b/src/include/access/xlogrecord.h @@ -17,6 +17,27 @@ #include "storage/block.h" #include "storage/relfilelocator.h" +/* + * XLogSizeClass + * + * XLog data segments registered to e.g. blocks are often quite small. + * This XLogSizeClass infrastructure allows us to do variable length encoding + * on these fields, so that we may save some bytes in each record on length + * fields. + * + * Note that it is possible to encode smaller sizes in the larger size classes, + * but the reverse is not true. For maximum efficiency, it is important to only + * use + * + * Values > UINT32_MAX are not supported in this scheme. + */ +typedef enum XLogSizeClass { + XLS_EMPTY = 0, /* length == 0 */ + XLS_UINT8 = 1, /* length <= UINT8_MAX; stored in uint8 (1B) */ + XLS_UINT16 = 2, /* length <= UINT16_MAX; stored in uint16 (2B) */ + XLS_UINT32 = 3 /* length <= UINT32_MAX; stored in uint32 (4B) */ +} XLogSizeClass; + /* * The overall layout of an XLOG record is: * Fixed-size header (XLogRecord struct) @@ -104,15 +125,14 @@ typedef struct XLogRecordBlockHeader { uint8 id; /* block reference ID */ uint8 fork_flags; /* fork within the relation, and flags */ - uint16 data_length; /* number of payload bytes (not including page - * image) */ + /* Depending on XLR_BLOCKID_SZCLASS; 0, 1 or 2 bytes of data_length follow */ /* If BKPBLOCK_HAS_IMAGE, an XLogRecordBlockImageHeader struct follows */ /* If BKPBLOCK_SAME_REL is not set, a RelFileLocator follows */ /* BlockNumber follows */ } XLogRecordBlockHeader; -#define SizeOfXLogRecordBlockHeader (offsetof(XLogRecordBlockHeader, data_length) + sizeof(uint16)) +#define SizeOfXLogRecordBlockHeader (offsetof(XLogRecordBlockHeader, fork_flags) + sizeof(uint8)) /* * Additional header information when a full-page image is included @@ -226,6 +246,11 @@ typedef struct XLogRecordDataHeaderLong #define SizeOfXLogRecordDataHeaderLong (sizeof(uint8) + sizeof(uint32)) +#define XLR_BLOCK_ID_MASK 0x3F + +#define XLR_BLOCKID_SZCLASS_SHIFT 6 +#define XLR_BLOCKID_SZCLASS_MASK (3 << XLR_BLOCKID_SZCLASS_SHIFT) + /* * Block IDs used to distinguish different kinds of record fragments. Block * references are numbered from 0 to XLR_MAX_BLOCK_ID. A rmgr is free to use @@ -234,15 +259,15 @@ typedef struct XLogRecordDataHeaderLong * numbers are reserved to denote the "main" data portion of the record, * as well as replication-supporting transaction metadata. * - * The maximum is currently set at 32, quite arbitrarily. Most records only + * The maximum is currently set at 0x20 (32), quite arbitrarily. Most records only * need a handful of block references, but there are a few exceptions that * need more. */ -#define XLR_MAX_BLOCK_ID 32 +#define XLR_MAX_BLOCK_ID 0x20 -#define XLR_BLOCK_ID_DATA_SHORT 255 -#define XLR_BLOCK_ID_DATA_LONG 254 -#define XLR_BLOCK_ID_ORIGIN 253 -#define XLR_BLOCK_ID_TOPLEVEL_XID 252 +#define XLR_BLOCK_ID_DATA_SHORT 0x3F +#define XLR_BLOCK_ID_DATA_LONG 0x3E +#define XLR_BLOCK_ID_ORIGIN 0x3D +#define XLR_BLOCK_ID_TOPLEVEL_XID 0x3C #endif /* XLOGRECORD_H */ -- 2.40.1 [application/octet-stream] v3-0002-Replace-accesses-to-rmgr-s-XLogRecord-info-bit-ac.patch (44.3K, ../../CAEze2Wh2e57adcGXWJLo4hRQY-__CmWwaaSbUf9PGGOX5ESrmQ@mail.gmail.com/4-v3-0002-Replace-accesses-to-rmgr-s-XLogRecord-info-bit-ac.patch) download | inline diff: From c8b3e45614e9227572ae981ee0fb260c8f31005d Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Mon, 23 Jan 2023 16:24:01 +0100 Subject: [PATCH v3 2/6] Replace accesses to rmgr's XLogRecord info bit accesses with XLogRecGetRmgrInfo This allows refactoring of the info bits into its own struct field, instead of sharing the field with the records own info bits. --- contrib/pg_walinspect/pg_walinspect.c | 8 ++--- src/backend/access/brin/brin_xlog.c | 4 +-- src/backend/access/gin/ginxlog.c | 2 +- src/backend/access/gist/gistxlog.c | 2 +- src/backend/access/hash/hash_xlog.c | 2 +- src/backend/access/heap/heapam.c | 10 +++---- src/backend/access/nbtree/nbtxlog.c | 2 +- src/backend/access/rmgrdesc/brindesc.c | 2 +- src/backend/access/rmgrdesc/clogdesc.c | 2 +- src/backend/access/rmgrdesc/committsdesc.c | 2 +- src/backend/access/rmgrdesc/dbasedesc.c | 2 +- src/backend/access/rmgrdesc/gindesc.c | 2 +- src/backend/access/rmgrdesc/gistdesc.c | 2 +- src/backend/access/rmgrdesc/hashdesc.c | 2 +- src/backend/access/rmgrdesc/heapdesc.c | 6 ++-- src/backend/access/rmgrdesc/logicalmsgdesc.c | 2 +- src/backend/access/rmgrdesc/mxactdesc.c | 2 +- src/backend/access/rmgrdesc/nbtdesc.c | 2 +- src/backend/access/rmgrdesc/relmapdesc.c | 2 +- src/backend/access/rmgrdesc/replorigindesc.c | 2 +- src/backend/access/rmgrdesc/seqdesc.c | 2 +- src/backend/access/rmgrdesc/smgrdesc.c | 2 +- src/backend/access/rmgrdesc/spgdesc.c | 2 +- src/backend/access/rmgrdesc/standbydesc.c | 2 +- src/backend/access/rmgrdesc/tblspcdesc.c | 2 +- src/backend/access/rmgrdesc/xactdesc.c | 30 +++++++++---------- src/backend/access/rmgrdesc/xlogdesc.c | 2 +- src/backend/access/spgist/spgxlog.c | 2 +- src/backend/access/transam/clog.c | 2 +- src/backend/access/transam/commit_ts.c | 2 +- src/backend/access/transam/multixact.c | 2 +- src/backend/access/transam/twophase.c | 2 +- src/backend/access/transam/xact.c | 10 +++---- src/backend/access/transam/xlog.c | 2 +- src/backend/access/transam/xlogrecovery.c | 22 +++++++------- src/backend/access/transam/xlogstats.c | 2 +- src/backend/catalog/storage.c | 2 +- src/backend/commands/dbcommands.c | 2 +- src/backend/commands/sequence.c | 2 +- src/backend/commands/tablespace.c | 2 +- src/backend/replication/logical/decode.c | 20 +++++++------ src/backend/replication/logical/message.c | 2 +- src/backend/replication/logical/origin.c | 2 +- src/backend/storage/ipc/standby.c | 2 +- src/backend/utils/cache/relmapper.c | 2 +- src/bin/pg_rewind/parsexlog.c | 6 ++-- src/bin/pg_waldump/pg_waldump.c | 4 +-- src/include/access/xact.h | 6 ++-- src/include/access/xlogreader.h | 3 +- .../test_custom_rmgrs/test_custom_rmgrs.c | 4 +-- 50 files changed, 105 insertions(+), 102 deletions(-) diff --git a/contrib/pg_walinspect/pg_walinspect.c b/contrib/pg_walinspect/pg_walinspect.c index 796a74f322..1b6f4cb178 100644 --- a/contrib/pg_walinspect/pg_walinspect.c +++ b/contrib/pg_walinspect/pg_walinspect.c @@ -196,10 +196,10 @@ GetWALRecordInfo(XLogReaderState *record, Datum *values, int i = 0; desc = GetRmgr(XLogRecGetRmid(record)); - record_type = desc.rm_identify(XLogRecGetInfo(record)); + record_type = desc.rm_identify(XLogRecGetRmgrInfo(record)); if (record_type == NULL) - record_type = psprintf("UNKNOWN (%x)", XLogRecGetInfo(record) & ~XLR_INFO_MASK); + record_type = psprintf("UNKNOWN (%x)", XLogRecGetRmgrInfo(record)); initStringInfo(&rec_desc); desc.rm_desc(&rec_desc, record); @@ -258,11 +258,11 @@ GetWALBlockInfo(FunctionCallInfo fcinfo, XLogReaderState *record, Assert(XLogRecHasAnyBlockRefs(record)); desc = GetRmgr(XLogRecGetRmid(record)); - record_type = desc.rm_identify(XLogRecGetInfo(record)); + record_type = desc.rm_identify(XLogRecGetRmgrInfo(record)); if (record_type == NULL) record_type = psprintf("UNKNOWN (%x)", - XLogRecGetInfo(record) & ~XLR_INFO_MASK); + XLogRecGetRmgrInfo(record)); initStringInfo(&rec_desc); desc.rm_desc(&rec_desc, record); diff --git a/src/backend/access/brin/brin_xlog.c b/src/backend/access/brin/brin_xlog.c index 89145b68f6..00dd85bdc5 100644 --- a/src/backend/access/brin/brin_xlog.c +++ b/src/backend/access/brin/brin_xlog.c @@ -56,7 +56,7 @@ brin_xlog_insert_update(XLogReaderState *record, * If we inserted the first and only tuple on the page, re-initialize the * page from scratch. */ - if (XLogRecGetInfo(record) & XLOG_BRIN_INIT_PAGE) + if (XLogRecGetRmgrInfo(record) & XLOG_BRIN_INIT_PAGE) { buffer = XLogInitBufferForRedo(record, 0); page = BufferGetPage(buffer); @@ -308,7 +308,7 @@ brin_xlog_desummarize_page(XLogReaderState *record) void brin_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info & XLOG_BRIN_OPMASK) { diff --git a/src/backend/access/gin/ginxlog.c b/src/backend/access/gin/ginxlog.c index f7c84beef8..feb675ed19 100644 --- a/src/backend/access/gin/ginxlog.c +++ b/src/backend/access/gin/ginxlog.c @@ -725,7 +725,7 @@ ginRedoDeleteListPages(XLogReaderState *record) void gin_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); MemoryContext oldCtx; /* diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index 15249aa921..a49210b64f 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -398,7 +398,7 @@ gistRedoPageReuse(XLogReaderState *record) void gist_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); MemoryContext oldCxt; /* diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index e8e06c62a9..3a45b7ee21 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -1048,7 +1048,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) void hash_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 7ed72abe59..1512f22c76 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -9255,7 +9255,7 @@ heap_xlog_insert(XLogReaderState *record) * If we inserted the first and only tuple on the page, re-initialize the * page from scratch. */ - if (XLogRecGetInfo(record) & XLOG_HEAP_INIT_PAGE) + if (XLogRecGetRmgrInfo(record) & XLOG_HEAP_INIT_PAGE) { buffer = XLogInitBufferForRedo(record, 0); page = BufferGetPage(buffer); @@ -9349,7 +9349,7 @@ heap_xlog_multi_insert(XLogReaderState *record) uint32 newlen; Size freespace = 0; int i; - bool isinit = (XLogRecGetInfo(record) & XLOG_HEAP_INIT_PAGE) != 0; + bool isinit = (XLogRecGetRmgrInfo(record) & XLOG_HEAP_INIT_PAGE) != 0; XLogRedoAction action; /* @@ -9597,7 +9597,7 @@ heap_xlog_update(XLogReaderState *record, bool hot_update) nbuffer = obuffer; newaction = oldaction; } - else if (XLogRecGetInfo(record) & XLOG_HEAP_INIT_PAGE) + else if (XLogRecGetRmgrInfo(record) & XLOG_HEAP_INIT_PAGE) { nbuffer = XLogInitBufferForRedo(record, 0); page = (Page) BufferGetPage(nbuffer); @@ -9961,7 +9961,7 @@ heap_xlog_inplace(XLogReaderState *record) void heap_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* * These operations don't overwrite MVCC data so no conflict processing is @@ -10007,7 +10007,7 @@ heap_redo(XLogReaderState *record) void heap2_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info & XLOG_HEAP_OPMASK) { diff --git a/src/backend/access/nbtree/nbtxlog.c b/src/backend/access/nbtree/nbtxlog.c index c87e46ed66..97d8d068c6 100644 --- a/src/backend/access/nbtree/nbtxlog.c +++ b/src/backend/access/nbtree/nbtxlog.c @@ -1015,7 +1015,7 @@ btree_xlog_reuse_page(XLogReaderState *record) void btree_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); MemoryContext oldCtx; oldCtx = MemoryContextSwitchTo(opCtx); diff --git a/src/backend/access/rmgrdesc/brindesc.c b/src/backend/access/rmgrdesc/brindesc.c index 1466a31bbb..1d3e5ee03e 100644 --- a/src/backend/access/rmgrdesc/brindesc.c +++ b/src/backend/access/rmgrdesc/brindesc.c @@ -20,7 +20,7 @@ void brin_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); info &= XLOG_BRIN_OPMASK; if (info == XLOG_BRIN_CREATE_INDEX) diff --git a/src/backend/access/rmgrdesc/clogdesc.c b/src/backend/access/rmgrdesc/clogdesc.c index e60b76f9da..915f9945d0 100644 --- a/src/backend/access/rmgrdesc/clogdesc.c +++ b/src/backend/access/rmgrdesc/clogdesc.c @@ -21,7 +21,7 @@ void clog_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == CLOG_ZEROPAGE) { diff --git a/src/backend/access/rmgrdesc/committsdesc.c b/src/backend/access/rmgrdesc/committsdesc.c index e7155cd507..3d673c2302 100644 --- a/src/backend/access/rmgrdesc/committsdesc.c +++ b/src/backend/access/rmgrdesc/committsdesc.c @@ -22,7 +22,7 @@ void commit_ts_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == COMMIT_TS_ZEROPAGE) { diff --git a/src/backend/access/rmgrdesc/dbasedesc.c b/src/backend/access/rmgrdesc/dbasedesc.c index 3922120d64..a9529bde05 100644 --- a/src/backend/access/rmgrdesc/dbasedesc.c +++ b/src/backend/access/rmgrdesc/dbasedesc.c @@ -22,7 +22,7 @@ void dbase_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_DBASE_CREATE_FILE_COPY) { diff --git a/src/backend/access/rmgrdesc/gindesc.c b/src/backend/access/rmgrdesc/gindesc.c index 246a6a6b85..08fc7ba49b 100644 --- a/src/backend/access/rmgrdesc/gindesc.c +++ b/src/backend/access/rmgrdesc/gindesc.c @@ -74,7 +74,7 @@ void gin_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/access/rmgrdesc/gistdesc.c b/src/backend/access/rmgrdesc/gistdesc.c index 5dc6e1dcee..6473069c1a 100644 --- a/src/backend/access/rmgrdesc/gistdesc.c +++ b/src/backend/access/rmgrdesc/gistdesc.c @@ -60,7 +60,7 @@ void gist_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/access/rmgrdesc/hashdesc.c b/src/backend/access/rmgrdesc/hashdesc.c index b6810a9320..bcb2013048 100644 --- a/src/backend/access/rmgrdesc/hashdesc.c +++ b/src/backend/access/rmgrdesc/hashdesc.c @@ -20,7 +20,7 @@ void hash_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/access/rmgrdesc/heapdesc.c b/src/backend/access/rmgrdesc/heapdesc.c index d73248abdd..15f943051a 100644 --- a/src/backend/access/rmgrdesc/heapdesc.c +++ b/src/backend/access/rmgrdesc/heapdesc.c @@ -95,7 +95,7 @@ void heap_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); info &= XLOG_HEAP_OPMASK; if (info == XLOG_HEAP_INSERT) @@ -172,7 +172,7 @@ void heap2_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); info &= XLOG_HEAP_OPMASK; if (info == XLOG_HEAP2_PRUNE) @@ -265,7 +265,7 @@ heap2_desc(StringInfo buf, XLogReaderState *record) else if (info == XLOG_HEAP2_MULTI_INSERT) { xl_heap_multi_insert *xlrec = (xl_heap_multi_insert *) rec; - bool isinit = (XLogRecGetInfo(record) & XLOG_HEAP_INIT_PAGE) != 0; + bool isinit = (XLogRecGetRmgrInfo(record) & XLOG_HEAP_INIT_PAGE) != 0; appendStringInfo(buf, "ntuples: %d, flags: 0x%02X", xlrec->ntuples, xlrec->flags); diff --git a/src/backend/access/rmgrdesc/logicalmsgdesc.c b/src/backend/access/rmgrdesc/logicalmsgdesc.c index 0578ec5535..2bb6cb06c3 100644 --- a/src/backend/access/rmgrdesc/logicalmsgdesc.c +++ b/src/backend/access/rmgrdesc/logicalmsgdesc.c @@ -19,7 +19,7 @@ void logicalmsg_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_LOGICAL_MESSAGE) { diff --git a/src/backend/access/rmgrdesc/mxactdesc.c b/src/backend/access/rmgrdesc/mxactdesc.c index a2fa1eca18..cdcf8a4f45 100644 --- a/src/backend/access/rmgrdesc/mxactdesc.c +++ b/src/backend/access/rmgrdesc/mxactdesc.c @@ -50,7 +50,7 @@ void multixact_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_MULTIXACT_ZERO_OFF_PAGE || info == XLOG_MULTIXACT_ZERO_MEM_PAGE) diff --git a/src/backend/access/rmgrdesc/nbtdesc.c b/src/backend/access/rmgrdesc/nbtdesc.c index e4fbaa4d5d..ede8df72c0 100644 --- a/src/backend/access/rmgrdesc/nbtdesc.c +++ b/src/backend/access/rmgrdesc/nbtdesc.c @@ -24,7 +24,7 @@ void btree_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/access/rmgrdesc/relmapdesc.c b/src/backend/access/rmgrdesc/relmapdesc.c index 82ad51eb9a..f4f0e5ce8b 100644 --- a/src/backend/access/rmgrdesc/relmapdesc.c +++ b/src/backend/access/rmgrdesc/relmapdesc.c @@ -20,7 +20,7 @@ void relmap_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_RELMAP_UPDATE) { diff --git a/src/backend/access/rmgrdesc/replorigindesc.c b/src/backend/access/rmgrdesc/replorigindesc.c index 8ce4588fda..539dc57939 100644 --- a/src/backend/access/rmgrdesc/replorigindesc.c +++ b/src/backend/access/rmgrdesc/replorigindesc.c @@ -19,7 +19,7 @@ void replorigin_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/access/rmgrdesc/seqdesc.c b/src/backend/access/rmgrdesc/seqdesc.c index ba60544085..6def7b653b 100644 --- a/src/backend/access/rmgrdesc/seqdesc.c +++ b/src/backend/access/rmgrdesc/seqdesc.c @@ -21,7 +21,7 @@ void seq_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); xl_seq_rec *xlrec = (xl_seq_rec *) rec; if (info == XLOG_SEQ_LOG) diff --git a/src/backend/access/rmgrdesc/smgrdesc.c b/src/backend/access/rmgrdesc/smgrdesc.c index bd841b96e8..8604bee0c0 100644 --- a/src/backend/access/rmgrdesc/smgrdesc.c +++ b/src/backend/access/rmgrdesc/smgrdesc.c @@ -21,7 +21,7 @@ void smgr_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_SMGR_CREATE) { diff --git a/src/backend/access/rmgrdesc/spgdesc.c b/src/backend/access/rmgrdesc/spgdesc.c index 87f62f0fb4..9e1e48240b 100644 --- a/src/backend/access/rmgrdesc/spgdesc.c +++ b/src/backend/access/rmgrdesc/spgdesc.c @@ -20,7 +20,7 @@ void spg_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/access/rmgrdesc/standbydesc.c b/src/backend/access/rmgrdesc/standbydesc.c index f2bce9a37a..9a08e10fe1 100644 --- a/src/backend/access/rmgrdesc/standbydesc.c +++ b/src/backend/access/rmgrdesc/standbydesc.c @@ -47,7 +47,7 @@ void standby_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_STANDBY_LOCK) { diff --git a/src/backend/access/rmgrdesc/tblspcdesc.c b/src/backend/access/rmgrdesc/tblspcdesc.c index b8c89f8c54..c1feb4fa05 100644 --- a/src/backend/access/rmgrdesc/tblspcdesc.c +++ b/src/backend/access/rmgrdesc/tblspcdesc.c @@ -21,7 +21,7 @@ void tblspc_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_TBLSPC_CREATE) { diff --git a/src/backend/access/rmgrdesc/xactdesc.c b/src/backend/access/rmgrdesc/xactdesc.c index 01610c5ddc..8903bb32de 100644 --- a/src/backend/access/rmgrdesc/xactdesc.c +++ b/src/backend/access/rmgrdesc/xactdesc.c @@ -32,7 +32,7 @@ */ void -ParseCommitRecord(uint8 info, xl_xact_commit *xlrec, xl_xact_parsed_commit *parsed) +ParseCommitRecord(uint8 rmgrinfo, xl_xact_commit *xlrec, xl_xact_parsed_commit *parsed) { char *data = ((char *) xlrec) + MinSizeOfXactCommit; @@ -43,7 +43,7 @@ ParseCommitRecord(uint8 info, xl_xact_commit *xlrec, xl_xact_parsed_commit *pars parsed->xact_time = xlrec->xact_time; - if (info & XLOG_XACT_HAS_INFO) + if (rmgrinfo & XLOG_XACT_HAS_INFO) { xl_xact_xinfo *xl_xinfo = (xl_xact_xinfo *) data; @@ -138,7 +138,7 @@ ParseCommitRecord(uint8 info, xl_xact_commit *xlrec, xl_xact_parsed_commit *pars } void -ParseAbortRecord(uint8 info, xl_xact_abort *xlrec, xl_xact_parsed_abort *parsed) +ParseAbortRecord(uint8 rmgrinfo, xl_xact_abort *xlrec, xl_xact_parsed_abort *parsed) { char *data = ((char *) xlrec) + MinSizeOfXactAbort; @@ -149,7 +149,7 @@ ParseAbortRecord(uint8 info, xl_xact_abort *xlrec, xl_xact_parsed_abort *parsed) parsed->xact_time = xlrec->xact_time; - if (info & XLOG_XACT_HAS_INFO) + if (rmgrinfo & XLOG_XACT_HAS_INFO) { xl_xact_xinfo *xl_xinfo = (xl_xact_xinfo *) data; @@ -236,7 +236,7 @@ ParseAbortRecord(uint8 info, xl_xact_abort *xlrec, xl_xact_parsed_abort *parsed) * ParsePrepareRecord */ void -ParsePrepareRecord(uint8 info, xl_xact_prepare *xlrec, xl_xact_parsed_prepare *parsed) +ParsePrepareRecord(uint8 rmgrinfo, xl_xact_prepare *xlrec, xl_xact_parsed_prepare *parsed) { char *bufptr; @@ -328,11 +328,11 @@ xact_desc_stats(StringInfo buf, const char *label, } static void -xact_desc_commit(StringInfo buf, uint8 info, xl_xact_commit *xlrec, RepOriginId origin_id) +xact_desc_commit(StringInfo buf, uint8 rmgrinfo, xl_xact_commit *xlrec, RepOriginId origin_id) { xl_xact_parsed_commit parsed; - ParseCommitRecord(info, xlrec, &parsed); + ParseCommitRecord(rmgrinfo, xlrec, &parsed); /* If this is a prepared xact, show the xid of the original xact */ if (TransactionIdIsValid(parsed.twophase_xid)) @@ -364,11 +364,11 @@ xact_desc_commit(StringInfo buf, uint8 info, xl_xact_commit *xlrec, RepOriginId } static void -xact_desc_abort(StringInfo buf, uint8 info, xl_xact_abort *xlrec, RepOriginId origin_id) +xact_desc_abort(StringInfo buf, uint8 rmgrinfo, xl_xact_abort *xlrec, RepOriginId origin_id) { xl_xact_parsed_abort parsed; - ParseAbortRecord(info, xlrec, &parsed); + ParseAbortRecord(rmgrinfo, xlrec, &parsed); /* If this is a prepared xact, show the xid of the original xact */ if (TransactionIdIsValid(parsed.twophase_xid)) @@ -391,11 +391,11 @@ xact_desc_abort(StringInfo buf, uint8 info, xl_xact_abort *xlrec, RepOriginId or } static void -xact_desc_prepare(StringInfo buf, uint8 info, xl_xact_prepare *xlrec, RepOriginId origin_id) +xact_desc_prepare(StringInfo buf, uint8 rmgrinfo, xl_xact_prepare *xlrec, RepOriginId origin_id) { xl_xact_parsed_prepare parsed; - ParsePrepareRecord(info, xlrec, &parsed); + ParsePrepareRecord(rmgrinfo, xlrec, &parsed); appendStringInfo(buf, "gid %s: ", parsed.twophase_gid); appendStringInfoString(buf, timestamptz_to_str(parsed.xact_time)); @@ -436,27 +436,27 @@ void xact_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & XLOG_XACT_OPMASK; + uint8 info = XLogRecGetRmgrInfo(record) & XLOG_XACT_OPMASK; if (info == XLOG_XACT_COMMIT || info == XLOG_XACT_COMMIT_PREPARED) { xl_xact_commit *xlrec = (xl_xact_commit *) rec; - xact_desc_commit(buf, XLogRecGetInfo(record), xlrec, + xact_desc_commit(buf, XLogRecGetRmgrInfo(record), xlrec, XLogRecGetOrigin(record)); } else if (info == XLOG_XACT_ABORT || info == XLOG_XACT_ABORT_PREPARED) { xl_xact_abort *xlrec = (xl_xact_abort *) rec; - xact_desc_abort(buf, XLogRecGetInfo(record), xlrec, + xact_desc_abort(buf, XLogRecGetRmgrInfo(record), xlrec, XLogRecGetOrigin(record)); } else if (info == XLOG_XACT_PREPARE) { xl_xact_prepare *xlrec = (xl_xact_prepare *) rec; - xact_desc_prepare(buf, XLogRecGetInfo(record), xlrec, + xact_desc_prepare(buf, XLogRecGetRmgrInfo(record), xlrec, XLogRecGetOrigin(record)); } else if (info == XLOG_XACT_ASSIGNMENT) diff --git a/src/backend/access/rmgrdesc/xlogdesc.c b/src/backend/access/rmgrdesc/xlogdesc.c index f390c177e4..3692dfb96a 100644 --- a/src/backend/access/rmgrdesc/xlogdesc.c +++ b/src/backend/access/rmgrdesc/xlogdesc.c @@ -37,7 +37,7 @@ void xlog_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_CHECKPOINT_SHUTDOWN || info == XLOG_CHECKPOINT_ONLINE) diff --git a/src/backend/access/spgist/spgxlog.c b/src/backend/access/spgist/spgxlog.c index 459ac929ba..53190a67f9 100644 --- a/src/backend/access/spgist/spgxlog.c +++ b/src/backend/access/spgist/spgxlog.c @@ -936,7 +936,7 @@ spgRedoVacuumRedirect(XLogReaderState *record) void spg_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); MemoryContext oldCxt; oldCxt = MemoryContextSwitchTo(opCtx); diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c index 4a431d5876..11c10b936d 100644 --- a/src/backend/access/transam/clog.c +++ b/src/backend/access/transam/clog.c @@ -984,7 +984,7 @@ WriteTruncateXlogRec(int pageno, TransactionId oldestXact, Oid oldestXactDb) void clog_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in clog records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/backend/access/transam/commit_ts.c b/src/backend/access/transam/commit_ts.c index b897fabc70..a351ee84ce 100644 --- a/src/backend/access/transam/commit_ts.c +++ b/src/backend/access/transam/commit_ts.c @@ -966,7 +966,7 @@ WriteTruncateXlogRec(int pageno, TransactionId oldestXid) void commit_ts_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in commit_ts records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c index abb022e067..c70b6b7358 100644 --- a/src/backend/access/transam/multixact.c +++ b/src/backend/access/transam/multixact.c @@ -3232,7 +3232,7 @@ WriteMTruncateXlogRec(Oid oldestMultiDB, void multixact_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in multixact records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c index 068e59bec0..f9ab5a9c7f 100644 --- a/src/backend/access/transam/twophase.c +++ b/src/backend/access/transam/twophase.c @@ -1429,7 +1429,7 @@ XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len) } if (XLogRecGetRmid(xlogreader) != RM_XACT_ID || - (XLogRecGetInfo(xlogreader) & XLOG_XACT_OPMASK) != XLOG_XACT_PREPARE) + (XLogRecGetRmgrInfo(xlogreader) & XLOG_XACT_OPMASK) != XLOG_XACT_PREPARE) ereport(ERROR, (errcode_for_file_access(), errmsg("expected two-phase state data is not present in WAL at %X/%X", diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index 8daaa535ed..c560f07390 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -6183,7 +6183,7 @@ xact_redo_abort(xl_xact_parsed_abort *parsed, TransactionId xid, void xact_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & XLOG_XACT_OPMASK; + uint8 info = XLogRecGetRmgrInfo(record) & XLOG_XACT_OPMASK; /* Backup blocks are not used in xact records */ Assert(!XLogRecHasAnyBlockRefs(record)); @@ -6193,7 +6193,7 @@ xact_redo(XLogReaderState *record) xl_xact_commit *xlrec = (xl_xact_commit *) XLogRecGetData(record); xl_xact_parsed_commit parsed; - ParseCommitRecord(XLogRecGetInfo(record), xlrec, &parsed); + ParseCommitRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); xact_redo_commit(&parsed, XLogRecGetXid(record), record->EndRecPtr, XLogRecGetOrigin(record)); } @@ -6202,7 +6202,7 @@ xact_redo(XLogReaderState *record) xl_xact_commit *xlrec = (xl_xact_commit *) XLogRecGetData(record); xl_xact_parsed_commit parsed; - ParseCommitRecord(XLogRecGetInfo(record), xlrec, &parsed); + ParseCommitRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); xact_redo_commit(&parsed, parsed.twophase_xid, record->EndRecPtr, XLogRecGetOrigin(record)); @@ -6216,7 +6216,7 @@ xact_redo(XLogReaderState *record) xl_xact_abort *xlrec = (xl_xact_abort *) XLogRecGetData(record); xl_xact_parsed_abort parsed; - ParseAbortRecord(XLogRecGetInfo(record), xlrec, &parsed); + ParseAbortRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); xact_redo_abort(&parsed, XLogRecGetXid(record), record->EndRecPtr, XLogRecGetOrigin(record)); } @@ -6225,7 +6225,7 @@ xact_redo(XLogReaderState *record) xl_xact_abort *xlrec = (xl_xact_abort *) XLogRecGetData(record); xl_xact_parsed_abort parsed; - ParseAbortRecord(XLogRecGetInfo(record), xlrec, &parsed); + ParseAbortRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); xact_redo_abort(&parsed, parsed.twophase_xid, record->EndRecPtr, XLogRecGetOrigin(record)); diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 8b0710abe6..b01cd6cc23 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -7719,7 +7719,7 @@ UpdateFullPageWrites(void) void xlog_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); XLogRecPtr lsn = record->EndRecPtr; /* diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index becc2bda62..6508246999 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -1995,7 +1995,7 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl static void xlogrecovery_redo(XLogReaderState *record, TimeLineID replayTLI) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); XLogRecPtr lsn = record->EndRecPtr; Assert(XLogRecGetRmid(record) == RM_XLOG_ID); @@ -2213,7 +2213,7 @@ void xlog_outdesc(StringInfo buf, XLogReaderState *record) { RmgrData rmgr = GetRmgr(XLogRecGetRmid(record)); - uint8 info = XLogRecGetInfo(record); + uint8 info = XLogRecGetRmgrInfo(record); const char *id; appendStringInfoString(buf, rmgr.rm_name); @@ -2221,7 +2221,7 @@ xlog_outdesc(StringInfo buf, XLogReaderState *record) id = rmgr.rm_identify(info); if (id == NULL) - appendStringInfo(buf, "UNKNOWN (%X): ", info & ~XLR_INFO_MASK); + appendStringInfo(buf, "UNKNOWN (%X): ", info); else appendStringInfo(buf, "%s: ", id); @@ -2341,7 +2341,7 @@ checkTimeLineSwitch(XLogRecPtr lsn, TimeLineID newTLI, TimeLineID prevTLI, static bool getRecordTimestamp(XLogReaderState *record, TimestampTz *recordXtime) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); uint8 xact_info = info & XLOG_XACT_OPMASK; uint8 rmid = XLogRecGetRmid(record); @@ -2535,7 +2535,7 @@ recoveryStopsBefore(XLogReaderState *record) if (XLogRecGetRmid(record) != RM_XACT_ID) return false; - xact_info = XLogRecGetInfo(record) & XLOG_XACT_OPMASK; + xact_info = XLogRecGetRmgrInfo(record); if (xact_info == XLOG_XACT_COMMIT) { @@ -2548,7 +2548,7 @@ recoveryStopsBefore(XLogReaderState *record) xl_xact_parsed_commit parsed; isCommit = true; - ParseCommitRecord(XLogRecGetInfo(record), + ParseCommitRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); recordXid = parsed.twophase_xid; @@ -2564,7 +2564,7 @@ recoveryStopsBefore(XLogReaderState *record) xl_xact_parsed_abort parsed; isCommit = false; - ParseAbortRecord(XLogRecGetInfo(record), + ParseAbortRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); recordXid = parsed.twophase_xid; @@ -2653,7 +2653,7 @@ recoveryStopsAfter(XLogReaderState *record) if (!ArchiveRecoveryRequested) return false; - info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + info = XLogRecGetRmgrInfo(record); rmid = XLogRecGetRmid(record); /* @@ -2721,7 +2721,7 @@ recoveryStopsAfter(XLogReaderState *record) xl_xact_commit *xlrec = (xl_xact_commit *) XLogRecGetData(record); xl_xact_parsed_commit parsed; - ParseCommitRecord(XLogRecGetInfo(record), + ParseCommitRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); recordXid = parsed.twophase_xid; @@ -2731,7 +2731,7 @@ recoveryStopsAfter(XLogReaderState *record) xl_xact_abort *xlrec = (xl_xact_abort *) XLogRecGetData(record); xl_xact_parsed_abort parsed; - ParseAbortRecord(XLogRecGetInfo(record), + ParseAbortRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); recordXid = parsed.twophase_xid; @@ -2925,7 +2925,7 @@ recoveryApplyDelay(XLogReaderState *record) if (XLogRecGetRmid(record) != RM_XACT_ID) return false; - xact_info = XLogRecGetInfo(record) & XLOG_XACT_OPMASK; + xact_info = XLogRecGetRmgrInfo(record) & XLOG_XACT_OPMASK; if (xact_info != XLOG_XACT_COMMIT && xact_info != XLOG_XACT_COMMIT_PREPARED) diff --git a/src/backend/access/transam/xlogstats.c b/src/backend/access/transam/xlogstats.c index 2d315df67a..267b108943 100644 --- a/src/backend/access/transam/xlogstats.c +++ b/src/backend/access/transam/xlogstats.c @@ -79,7 +79,7 @@ XLogRecStoreStats(XLogStats *stats, XLogReaderState *record) * RmgrId). */ - recid = XLogRecGetInfo(record) >> 4; + recid = XLogRecGetRmgrInfo(record) >> 4; /* * XACT records need to be handled differently. Those records use the diff --git a/src/backend/catalog/storage.c b/src/backend/catalog/storage.c index 2add053489..4d73023a65 100644 --- a/src/backend/catalog/storage.c +++ b/src/backend/catalog/storage.c @@ -958,7 +958,7 @@ void smgr_redo(XLogReaderState *record) { XLogRecPtr lsn = record->EndRecPtr; - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in smgr records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c index 09f1ab41ad..dffb683e74 100644 --- a/src/backend/commands/dbcommands.c +++ b/src/backend/commands/dbcommands.c @@ -3109,7 +3109,7 @@ recovery_create_dbdir(char *path, bool only_tblspc) void dbase_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in dbase records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c index ef01449678..eae2cb09b1 100644 --- a/src/backend/commands/sequence.c +++ b/src/backend/commands/sequence.c @@ -1829,7 +1829,7 @@ void seq_redo(XLogReaderState *record) { XLogRecPtr lsn = record->EndRecPtr; - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); Buffer buffer; Page page; Page localpage; diff --git a/src/backend/commands/tablespace.c b/src/backend/commands/tablespace.c index 13b0dee146..1de6803df7 100644 --- a/src/backend/commands/tablespace.c +++ b/src/backend/commands/tablespace.c @@ -1516,7 +1516,7 @@ get_tablespace_name(Oid spc_oid) void tblspc_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in tblspc records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c index d91055a440..419ab527a8 100644 --- a/src/backend/replication/logical/decode.c +++ b/src/backend/replication/logical/decode.c @@ -132,7 +132,7 @@ void xlog_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) { SnapBuild *builder = ctx->snapshot_builder; - uint8 info = XLogRecGetInfo(buf->record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(buf->record); ReorderBufferProcessXid(ctx->reorder, XLogRecGetXid(buf->record), buf->origptr); @@ -205,7 +205,7 @@ xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) SnapBuild *builder = ctx->snapshot_builder; ReorderBuffer *reorder = ctx->reorder; XLogReaderState *r = buf->record; - uint8 info = XLogRecGetInfo(r) & XLOG_XACT_OPMASK; + uint8 info = XLogRecGetRmgrInfo(r) & XLOG_XACT_OPMASK; /* * If the snapshot isn't yet fully built, we cannot decode anything, so @@ -225,7 +225,9 @@ xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) bool two_phase = false; xlrec = (xl_xact_commit *) XLogRecGetData(r); - ParseCommitRecord(XLogRecGetInfo(buf->record), xlrec, &parsed); + ParseCommitRecord(XLogRecGetRmgrInfo(buf->record), + xlrec, + &parsed); if (!TransactionIdIsValid(parsed.twophase_xid)) xid = XLogRecGetXid(r); @@ -253,7 +255,7 @@ xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) bool two_phase = false; xlrec = (xl_xact_abort *) XLogRecGetData(r); - ParseAbortRecord(XLogRecGetInfo(buf->record), xlrec, &parsed); + ParseAbortRecord(XLogRecGetRmgrInfo(buf->record), xlrec, &parsed); if (!TransactionIdIsValid(parsed.twophase_xid)) xid = XLogRecGetXid(r); @@ -316,7 +318,7 @@ xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) /* ok, parse it */ xlrec = (xl_xact_prepare *) XLogRecGetData(r); - ParsePrepareRecord(XLogRecGetInfo(buf->record), + ParsePrepareRecord(XLogRecGetRmgrInfo(buf->record), xlrec, &parsed); /* @@ -361,7 +363,7 @@ standby_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) { SnapBuild *builder = ctx->snapshot_builder; XLogReaderState *r = buf->record; - uint8 info = XLogRecGetInfo(r) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(r); ReorderBufferProcessXid(ctx->reorder, XLogRecGetXid(r), buf->origptr); @@ -405,7 +407,7 @@ standby_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) void heap2_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) { - uint8 info = XLogRecGetInfo(buf->record) & XLOG_HEAP_OPMASK; + uint8 info = XLogRecGetRmgrInfo(buf->record) & XLOG_HEAP_OPMASK; TransactionId xid = XLogRecGetXid(buf->record); SnapBuild *builder = ctx->snapshot_builder; @@ -465,7 +467,7 @@ heap2_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) void heap_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) { - uint8 info = XLogRecGetInfo(buf->record) & XLOG_HEAP_OPMASK; + uint8 info = XLogRecGetRmgrInfo(buf->record) & XLOG_HEAP_OPMASK; TransactionId xid = XLogRecGetXid(buf->record); SnapBuild *builder = ctx->snapshot_builder; @@ -590,7 +592,7 @@ logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) SnapBuild *builder = ctx->snapshot_builder; XLogReaderState *r = buf->record; TransactionId xid = XLogRecGetXid(r); - uint8 info = XLogRecGetInfo(r) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(r); RepOriginId origin_id = XLogRecGetOrigin(r); Snapshot snapshot = NULL; xl_logical_message *message; diff --git a/src/backend/replication/logical/message.c b/src/backend/replication/logical/message.c index c5de14afc6..c31dc1203f 100644 --- a/src/backend/replication/logical/message.c +++ b/src/backend/replication/logical/message.c @@ -80,7 +80,7 @@ LogLogicalMessage(const char *prefix, const char *message, size_t size, void logicalmsg_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info != XLOG_LOGICAL_MESSAGE) elog(PANIC, "logicalmsg_redo: unknown op code %u", info); diff --git a/src/backend/replication/logical/origin.c b/src/backend/replication/logical/origin.c index b0255ffd25..4977d2eb9a 100644 --- a/src/backend/replication/logical/origin.c +++ b/src/backend/replication/logical/origin.c @@ -826,7 +826,7 @@ StartupReplicationOrigin(void) void replorigin_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c index 4c06741a69..3fa1ca12a3 100644 --- a/src/backend/storage/ipc/standby.c +++ b/src/backend/storage/ipc/standby.c @@ -1160,7 +1160,7 @@ StandbyReleaseOldLocks(TransactionId oldxid) void standby_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in standby records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/backend/utils/cache/relmapper.c b/src/backend/utils/cache/relmapper.c index 26575cae6c..b825dda353 100644 --- a/src/backend/utils/cache/relmapper.c +++ b/src/backend/utils/cache/relmapper.c @@ -1084,7 +1084,7 @@ perform_relmap_update(bool shared, const RelMapFile *updates) void relmap_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in relmap records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/bin/pg_rewind/parsexlog.c b/src/bin/pg_rewind/parsexlog.c index 27782237d0..2b6b46f0e3 100644 --- a/src/bin/pg_rewind/parsexlog.c +++ b/src/bin/pg_rewind/parsexlog.c @@ -222,7 +222,7 @@ findLastCheckpoint(const char *datadir, XLogRecPtr forkptr, int tliIndex, * be the latest checkpoint before WAL forked and not the checkpoint * where the primary has been stopped to be rewound. */ - info = XLogRecGetInfo(xlogreader) & ~XLR_INFO_MASK; + info = XLogRecGetRmgrInfo(xlogreader); if (searchptr < forkptr && XLogRecGetRmid(xlogreader) == RM_XLOG_ID && (info == XLOG_CHECKPOINT_SHUTDOWN || @@ -370,7 +370,7 @@ extractPageInfo(XLogReaderState *record) int block_id; RmgrId rmid = XLogRecGetRmid(record); uint8 info = XLogRecGetInfo(record); - uint8 rminfo = info & ~XLR_INFO_MASK; + uint8 rminfo = XLogRecGetRmgrInfo(record); /* Is this a special record type that I recognize? */ @@ -440,7 +440,7 @@ extractPageInfo(XLogReaderState *record) pg_fatal("WAL record modifies a relation, but record type is not recognized: " "lsn: %X/%X, rmid: %d, rmgr: %s, info: %02X", LSN_FORMAT_ARGS(record->ReadRecPtr), - rmid, RmgrName(rmid), info); + rmid, RmgrName(rmid), info | rminfo); } for (block_id = 0; block_id <= XLogRecMaxBlockId(record); block_id++) diff --git a/src/bin/pg_waldump/pg_waldump.c b/src/bin/pg_waldump/pg_waldump.c index 96845e1a1a..85694b3d5c 100644 --- a/src/bin/pg_waldump/pg_waldump.c +++ b/src/bin/pg_waldump/pg_waldump.c @@ -545,7 +545,7 @@ XLogDumpDisplayRecord(XLogDumpConfig *config, XLogReaderState *record) const RmgrDescData *desc = GetRmgrDesc(XLogRecGetRmid(record)); uint32 rec_len; uint32 fpi_len; - uint8 info = XLogRecGetInfo(record); + uint8 info = XLogRecGetRmgrInfo(record); XLogRecPtr xl_prev = XLogRecGetPrev(record); StringInfoData s; @@ -560,7 +560,7 @@ XLogDumpDisplayRecord(XLogDumpConfig *config, XLogReaderState *record) id = desc->rm_identify(info); if (id == NULL) - printf("desc: UNKNOWN (%x) ", info & ~XLR_INFO_MASK); + printf("desc: UNKNOWN (%x) ", info); else printf("desc: %s ", id); diff --git a/src/include/access/xact.h b/src/include/access/xact.h index 7d3b9446e6..b94b264402 100644 --- a/src/include/access/xact.h +++ b/src/include/access/xact.h @@ -519,9 +519,9 @@ extern void xact_desc(StringInfo buf, XLogReaderState *record); extern const char *xact_identify(uint8 info); /* also in xactdesc.c, so they can be shared between front/backend code */ -extern void ParseCommitRecord(uint8 info, xl_xact_commit *xlrec, xl_xact_parsed_commit *parsed); -extern void ParseAbortRecord(uint8 info, xl_xact_abort *xlrec, xl_xact_parsed_abort *parsed); -extern void ParsePrepareRecord(uint8 info, xl_xact_prepare *xlrec, xl_xact_parsed_prepare *parsed); +extern void ParseCommitRecord(uint8 rmgrinfo, xl_xact_commit *xlrec, xl_xact_parsed_commit *parsed); +extern void ParseAbortRecord(uint8 rmgrinfo, xl_xact_abort *xlrec, xl_xact_parsed_abort *parsed); +extern void ParsePrepareRecord(uint8 rmgrinfo, xl_xact_prepare *xlrec, xl_xact_parsed_prepare *parsed); extern void EnterParallelMode(void); extern void ExitParallelMode(void); diff --git a/src/include/access/xlogreader.h b/src/include/access/xlogreader.h index da32c7db77..2b976dcc03 100644 --- a/src/include/access/xlogreader.h +++ b/src/include/access/xlogreader.h @@ -407,7 +407,8 @@ extern bool DecodeXLogRecord(XLogReaderState *state, */ #define XLogRecGetTotalLen(decoder) ((decoder)->record->header.xl_tot_len) #define XLogRecGetPrev(decoder) ((decoder)->record->header.xl_prev) -#define XLogRecGetInfo(decoder) ((decoder)->record->header.xl_info) +#define XLogRecGetInfo(decoder) ((decoder)->record->header.xl_info & XLR_INFO_MASK) +#define XLogRecGetRmgrInfo(decoder) (((decoder)->record->header.xl_info) & XLR_RMGR_INFO_MASK) #define XLogRecGetRmid(decoder) ((decoder)->record->header.xl_rmid) #define XLogRecGetXid(decoder) ((decoder)->record->header.xl_xid) #define XLogRecGetOrigin(decoder) ((decoder)->record->record_origin) diff --git a/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c b/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c index a304ba54bb..b5537a5f5b 100644 --- a/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c +++ b/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c @@ -81,7 +81,7 @@ _PG_init(void) void testcustomrmgrs_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info != XLOG_TEST_CUSTOM_RMGRS_MESSAGE) elog(PANIC, "testcustomrmgrs_redo: unknown op code %u", info); @@ -91,7 +91,7 @@ void testcustomrmgrs_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_TEST_CUSTOM_RMGRS_MESSAGE) { -- 2.40.1 [application/octet-stream] v3-0003-Rename-rmgr_identify-info-bits-argument-to-rmgrin.patch (21.6K, ../../CAEze2Wh2e57adcGXWJLo4hRQY-__CmWwaaSbUf9PGGOX5ESrmQ@mail.gmail.com/5-v3-0003-Rename-rmgr_identify-info-bits-argument-to-rmgrin.patch) download | inline diff: From 09bfc689208d7aea8d875f4638bdccd6d77c1293 Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Mon, 23 Jan 2023 17:31:18 +0100 Subject: [PATCH v3 3/6] Rename rmgr_identify 'info'-bits argument to rmgrinfo This indicates the provided bits are fully owned by the rmgr, and none of the xlog record's owned bits are shared with the rmgr anymore. --- src/backend/access/rmgrdesc/brindesc.c | 4 ++-- src/backend/access/rmgrdesc/clogdesc.c | 4 ++-- src/backend/access/rmgrdesc/dbasedesc.c | 4 ++-- src/backend/access/rmgrdesc/gindesc.c | 4 ++-- src/backend/access/rmgrdesc/gistdesc.c | 4 ++-- src/backend/access/rmgrdesc/hashdesc.c | 4 ++-- src/backend/access/rmgrdesc/heapdesc.c | 8 ++++---- src/backend/access/rmgrdesc/logicalmsgdesc.c | 4 ++-- src/backend/access/rmgrdesc/mxactdesc.c | 4 ++-- src/backend/access/rmgrdesc/nbtdesc.c | 4 ++-- src/backend/access/rmgrdesc/relmapdesc.c | 4 ++-- src/backend/access/rmgrdesc/seqdesc.c | 4 ++-- src/backend/access/rmgrdesc/smgrdesc.c | 4 ++-- src/backend/access/rmgrdesc/spgdesc.c | 4 ++-- src/backend/access/rmgrdesc/standbydesc.c | 4 ++-- src/backend/access/rmgrdesc/tblspcdesc.c | 4 ++-- src/backend/access/rmgrdesc/xlogdesc.c | 4 ++-- src/include/access/brin_xlog.h | 2 +- src/include/access/clog.h | 2 +- src/include/access/ginxlog.h | 2 +- src/include/access/gistxlog.h | 2 +- src/include/access/hash_xlog.h | 2 +- src/include/access/heapam_xlog.h | 4 ++-- src/include/access/multixact.h | 2 +- src/include/access/nbtxlog.h | 2 +- src/include/access/spgxlog.h | 2 +- src/include/access/xlog.h | 2 +- src/include/catalog/storage_xlog.h | 2 +- src/include/commands/dbcommands_xlog.h | 2 +- src/include/commands/sequence.h | 2 +- src/include/commands/tablespace.h | 2 +- src/include/replication/message.h | 2 +- src/include/storage/standbydefs.h | 2 +- src/include/utils/relmapper.h | 2 +- src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c | 4 ++-- 35 files changed, 56 insertions(+), 56 deletions(-) diff --git a/src/backend/access/rmgrdesc/brindesc.c b/src/backend/access/rmgrdesc/brindesc.c index 1d3e5ee03e..fa272f0720 100644 --- a/src/backend/access/rmgrdesc/brindesc.c +++ b/src/backend/access/rmgrdesc/brindesc.c @@ -71,11 +71,11 @@ brin_desc(StringInfo buf, XLogReaderState *record) } const char * -brin_identify(uint8 info) +brin_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_BRIN_CREATE_INDEX: id = "CREATE_INDEX"; diff --git a/src/backend/access/rmgrdesc/clogdesc.c b/src/backend/access/rmgrdesc/clogdesc.c index 915f9945d0..b368af595e 100644 --- a/src/backend/access/rmgrdesc/clogdesc.c +++ b/src/backend/access/rmgrdesc/clogdesc.c @@ -41,11 +41,11 @@ clog_desc(StringInfo buf, XLogReaderState *record) } const char * -clog_identify(uint8 info) +clog_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case CLOG_ZEROPAGE: id = "ZEROPAGE"; diff --git a/src/backend/access/rmgrdesc/dbasedesc.c b/src/backend/access/rmgrdesc/dbasedesc.c index a9529bde05..a405c0738a 100644 --- a/src/backend/access/rmgrdesc/dbasedesc.c +++ b/src/backend/access/rmgrdesc/dbasedesc.c @@ -54,11 +54,11 @@ dbase_desc(StringInfo buf, XLogReaderState *record) } const char * -dbase_identify(uint8 info) +dbase_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_DBASE_CREATE_FILE_COPY: id = "CREATE_FILE_COPY"; diff --git a/src/backend/access/rmgrdesc/gindesc.c b/src/backend/access/rmgrdesc/gindesc.c index 08fc7ba49b..e7ed7a6094 100644 --- a/src/backend/access/rmgrdesc/gindesc.c +++ b/src/backend/access/rmgrdesc/gindesc.c @@ -179,11 +179,11 @@ gin_desc(StringInfo buf, XLogReaderState *record) } const char * -gin_identify(uint8 info) +gin_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_GIN_CREATE_PTREE: id = "CREATE_PTREE"; diff --git a/src/backend/access/rmgrdesc/gistdesc.c b/src/backend/access/rmgrdesc/gistdesc.c index 6473069c1a..899e12bcc3 100644 --- a/src/backend/access/rmgrdesc/gistdesc.c +++ b/src/backend/access/rmgrdesc/gistdesc.c @@ -86,11 +86,11 @@ gist_desc(StringInfo buf, XLogReaderState *record) } const char * -gist_identify(uint8 info) +gist_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_GIST_PAGE_UPDATE: id = "PAGE_UPDATE"; diff --git a/src/backend/access/rmgrdesc/hashdesc.c b/src/backend/access/rmgrdesc/hashdesc.c index bcb2013048..451251caca 100644 --- a/src/backend/access/rmgrdesc/hashdesc.c +++ b/src/backend/access/rmgrdesc/hashdesc.c @@ -122,11 +122,11 @@ hash_desc(StringInfo buf, XLogReaderState *record) } const char * -hash_identify(uint8 info) +hash_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_HASH_INIT_META_PAGE: id = "INIT_META_PAGE"; diff --git a/src/backend/access/rmgrdesc/heapdesc.c b/src/backend/access/rmgrdesc/heapdesc.c index 15f943051a..5224869af1 100644 --- a/src/backend/access/rmgrdesc/heapdesc.c +++ b/src/backend/access/rmgrdesc/heapdesc.c @@ -302,11 +302,11 @@ heap2_desc(StringInfo buf, XLogReaderState *record) } const char * -heap_identify(uint8 info) +heap_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_HEAP_INSERT: id = "INSERT"; @@ -347,11 +347,11 @@ heap_identify(uint8 info) } const char * -heap2_identify(uint8 info) +heap2_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_HEAP2_PRUNE: id = "PRUNE"; diff --git a/src/backend/access/rmgrdesc/logicalmsgdesc.c b/src/backend/access/rmgrdesc/logicalmsgdesc.c index 2bb6cb06c3..7566630961 100644 --- a/src/backend/access/rmgrdesc/logicalmsgdesc.c +++ b/src/backend/access/rmgrdesc/logicalmsgdesc.c @@ -43,9 +43,9 @@ logicalmsg_desc(StringInfo buf, XLogReaderState *record) } const char * -logicalmsg_identify(uint8 info) +logicalmsg_identify(uint8 rmgrinfo) { - if ((info & ~XLR_INFO_MASK) == XLOG_LOGICAL_MESSAGE) + if (rmgrinfo == XLOG_LOGICAL_MESSAGE) return "MESSAGE"; return NULL; diff --git a/src/backend/access/rmgrdesc/mxactdesc.c b/src/backend/access/rmgrdesc/mxactdesc.c index cdcf8a4f45..7676778239 100644 --- a/src/backend/access/rmgrdesc/mxactdesc.c +++ b/src/backend/access/rmgrdesc/mxactdesc.c @@ -81,11 +81,11 @@ multixact_desc(StringInfo buf, XLogReaderState *record) } const char * -multixact_identify(uint8 info) +multixact_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_MULTIXACT_ZERO_OFF_PAGE: id = "ZERO_OFF_PAGE"; diff --git a/src/backend/access/rmgrdesc/nbtdesc.c b/src/backend/access/rmgrdesc/nbtdesc.c index ede8df72c0..ff9e28ca55 100644 --- a/src/backend/access/rmgrdesc/nbtdesc.c +++ b/src/backend/access/rmgrdesc/nbtdesc.c @@ -134,11 +134,11 @@ btree_desc(StringInfo buf, XLogReaderState *record) } const char * -btree_identify(uint8 info) +btree_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_BTREE_INSERT_LEAF: id = "INSERT_LEAF"; diff --git a/src/backend/access/rmgrdesc/relmapdesc.c b/src/backend/access/rmgrdesc/relmapdesc.c index f4f0e5ce8b..9c9f191700 100644 --- a/src/backend/access/rmgrdesc/relmapdesc.c +++ b/src/backend/access/rmgrdesc/relmapdesc.c @@ -32,11 +32,11 @@ relmap_desc(StringInfo buf, XLogReaderState *record) } const char * -relmap_identify(uint8 info) +relmap_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_RELMAP_UPDATE: id = "UPDATE"; diff --git a/src/backend/access/rmgrdesc/seqdesc.c b/src/backend/access/rmgrdesc/seqdesc.c index 6def7b653b..eed631b5d8 100644 --- a/src/backend/access/rmgrdesc/seqdesc.c +++ b/src/backend/access/rmgrdesc/seqdesc.c @@ -31,11 +31,11 @@ seq_desc(StringInfo buf, XLogReaderState *record) } const char * -seq_identify(uint8 info) +seq_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_SEQ_LOG: id = "LOG"; diff --git a/src/backend/access/rmgrdesc/smgrdesc.c b/src/backend/access/rmgrdesc/smgrdesc.c index 8604bee0c0..653caf58e4 100644 --- a/src/backend/access/rmgrdesc/smgrdesc.c +++ b/src/backend/access/rmgrdesc/smgrdesc.c @@ -43,11 +43,11 @@ smgr_desc(StringInfo buf, XLogReaderState *record) } const char * -smgr_identify(uint8 info) +smgr_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_SMGR_CREATE: id = "CREATE"; diff --git a/src/backend/access/rmgrdesc/spgdesc.c b/src/backend/access/rmgrdesc/spgdesc.c index 9e1e48240b..0d1af9a5b6 100644 --- a/src/backend/access/rmgrdesc/spgdesc.c +++ b/src/backend/access/rmgrdesc/spgdesc.c @@ -128,11 +128,11 @@ spg_desc(StringInfo buf, XLogReaderState *record) } const char * -spg_identify(uint8 info) +spg_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_SPGIST_ADD_LEAF: id = "ADD_LEAF"; diff --git a/src/backend/access/rmgrdesc/standbydesc.c b/src/backend/access/rmgrdesc/standbydesc.c index 9a08e10fe1..56acea1407 100644 --- a/src/backend/access/rmgrdesc/standbydesc.c +++ b/src/backend/access/rmgrdesc/standbydesc.c @@ -76,11 +76,11 @@ standby_desc(StringInfo buf, XLogReaderState *record) } const char * -standby_identify(uint8 info) +standby_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_STANDBY_LOCK: id = "LOCK"; diff --git a/src/backend/access/rmgrdesc/tblspcdesc.c b/src/backend/access/rmgrdesc/tblspcdesc.c index c1feb4fa05..c5849ccf7c 100644 --- a/src/backend/access/rmgrdesc/tblspcdesc.c +++ b/src/backend/access/rmgrdesc/tblspcdesc.c @@ -38,11 +38,11 @@ tblspc_desc(StringInfo buf, XLogReaderState *record) } const char * -tblspc_identify(uint8 info) +tblspc_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_TBLSPC_CREATE: id = "CREATE"; diff --git a/src/backend/access/rmgrdesc/xlogdesc.c b/src/backend/access/rmgrdesc/xlogdesc.c index 3692dfb96a..4db4f3331f 100644 --- a/src/backend/access/rmgrdesc/xlogdesc.c +++ b/src/backend/access/rmgrdesc/xlogdesc.c @@ -151,11 +151,11 @@ xlog_desc(StringInfo buf, XLogReaderState *record) } const char * -xlog_identify(uint8 info) +xlog_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_CHECKPOINT_SHUTDOWN: id = "CHECKPOINT_SHUTDOWN"; diff --git a/src/include/access/brin_xlog.h b/src/include/access/brin_xlog.h index ae263a371f..45a688d09b 100644 --- a/src/include/access/brin_xlog.h +++ b/src/include/access/brin_xlog.h @@ -145,7 +145,7 @@ typedef struct xl_brin_desummarize extern void brin_redo(XLogReaderState *record); extern void brin_desc(StringInfo buf, XLogReaderState *record); -extern const char *brin_identify(uint8 info); +extern const char *brin_identify(uint8 rmgrinfo); extern void brin_mask(char *pagedata, BlockNumber blkno); #endif /* BRIN_XLOG_H */ diff --git a/src/include/access/clog.h b/src/include/access/clog.h index d99444f073..224cad4415 100644 --- a/src/include/access/clog.h +++ b/src/include/access/clog.h @@ -58,6 +58,6 @@ extern int clogsyncfiletag(const FileTag *ftag, char *path); extern void clog_redo(XLogReaderState *record); extern void clog_desc(StringInfo buf, XLogReaderState *record); -extern const char *clog_identify(uint8 info); +extern const char *clog_identify(uint8 rmgrinfo); #endif /* CLOG_H */ diff --git a/src/include/access/ginxlog.h b/src/include/access/ginxlog.h index 37095e5f17..adf1f6b6d8 100644 --- a/src/include/access/ginxlog.h +++ b/src/include/access/ginxlog.h @@ -208,7 +208,7 @@ typedef struct ginxlogDeleteListPages extern void gin_redo(XLogReaderState *record); extern void gin_desc(StringInfo buf, XLogReaderState *record); -extern const char *gin_identify(uint8 info); +extern const char *gin_identify(uint8 rmgrinfo); extern void gin_xlog_startup(void); extern void gin_xlog_cleanup(void); extern void gin_mask(char *pagedata, BlockNumber blkno); diff --git a/src/include/access/gistxlog.h b/src/include/access/gistxlog.h index aff2ffbdcc..0ca37f3a07 100644 --- a/src/include/access/gistxlog.h +++ b/src/include/access/gistxlog.h @@ -109,7 +109,7 @@ typedef struct gistxlogPageReuse extern void gist_redo(XLogReaderState *record); extern void gist_desc(StringInfo buf, XLogReaderState *record); -extern const char *gist_identify(uint8 info); +extern const char *gist_identify(uint8 rmgrinfo); extern void gist_xlog_startup(void); extern void gist_xlog_cleanup(void); extern void gist_mask(char *pagedata, BlockNumber blkno); diff --git a/src/include/access/hash_xlog.h b/src/include/access/hash_xlog.h index b93619d1a8..74859a2a47 100644 --- a/src/include/access/hash_xlog.h +++ b/src/include/access/hash_xlog.h @@ -263,7 +263,7 @@ typedef struct xl_hash_vacuum_one_page extern void hash_redo(XLogReaderState *record); extern void hash_desc(StringInfo buf, XLogReaderState *record); -extern const char *hash_identify(uint8 info); +extern const char *hash_identify(uint8 rmgrinfo); extern void hash_mask(char *pagedata, BlockNumber blkno); #endif /* HASH_XLOG_H */ diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h index a038450787..70e9acd350 100644 --- a/src/include/access/heapam_xlog.h +++ b/src/include/access/heapam_xlog.h @@ -406,11 +406,11 @@ extern void HeapTupleHeaderAdvanceConflictHorizon(HeapTupleHeader tuple, extern void heap_redo(XLogReaderState *record); extern void heap_desc(StringInfo buf, XLogReaderState *record); -extern const char *heap_identify(uint8 info); +extern const char *heap_identify(uint8 rmgrinfo); extern void heap_mask(char *pagedata, BlockNumber blkno); extern void heap2_redo(XLogReaderState *record); extern void heap2_desc(StringInfo buf, XLogReaderState *record); -extern const char *heap2_identify(uint8 info); +extern const char *heap2_identify(uint8 rmgrinfo); extern void heap_xlog_logical_rewrite(XLogReaderState *r); extern XLogRecPtr log_heap_visible(Relation rel, Buffer heap_buffer, diff --git a/src/include/access/multixact.h b/src/include/access/multixact.h index 246f757f6a..d52e69aa05 100644 --- a/src/include/access/multixact.h +++ b/src/include/access/multixact.h @@ -158,7 +158,7 @@ extern void multixact_twophase_postabort(TransactionId xid, uint16 info, extern void multixact_redo(XLogReaderState *record); extern void multixact_desc(StringInfo buf, XLogReaderState *record); -extern const char *multixact_identify(uint8 info); +extern const char *multixact_identify(uint8 rmgrinfo); extern char *mxid_to_string(MultiXactId multi, int nmembers, MultiXactMember *members); diff --git a/src/include/access/nbtxlog.h b/src/include/access/nbtxlog.h index 7dd9fd0206..422ac13d56 100644 --- a/src/include/access/nbtxlog.h +++ b/src/include/access/nbtxlog.h @@ -362,6 +362,6 @@ extern void btree_mask(char *pagedata, BlockNumber blkno); * prototypes for functions in nbtdesc.c */ extern void btree_desc(StringInfo buf, XLogReaderState *record); -extern const char *btree_identify(uint8 info); +extern const char *btree_identify(uint8 rmgrinfo); #endif /* NBTXLOG_H */ diff --git a/src/include/access/spgxlog.h b/src/include/access/spgxlog.h index a7f10bf2d3..3533894baf 100644 --- a/src/include/access/spgxlog.h +++ b/src/include/access/spgxlog.h @@ -251,7 +251,7 @@ typedef struct spgxlogVacuumRedirect extern void spg_redo(XLogReaderState *record); extern void spg_desc(StringInfo buf, XLogReaderState *record); -extern const char *spg_identify(uint8 info); +extern const char *spg_identify(uint8 rmgrinfo); extern void spg_xlog_startup(void); extern void spg_xlog_cleanup(void); extern void spg_mask(char *pagedata, BlockNumber blkno); diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h index 48ca852381..9007f559f3 100644 --- a/src/include/access/xlog.h +++ b/src/include/access/xlog.h @@ -211,7 +211,7 @@ extern void XLogSetReplicationSlotMinimumLSN(XLogRecPtr lsn); extern void xlog_redo(struct XLogReaderState *record); extern void xlog_desc(StringInfo buf, struct XLogReaderState *record); -extern const char *xlog_identify(uint8 info); +extern const char *xlog_identify(uint8 rmgrinfo); extern void issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli); diff --git a/src/include/catalog/storage_xlog.h b/src/include/catalog/storage_xlog.h index 6b0a7aa3df..27514163be 100644 --- a/src/include/catalog/storage_xlog.h +++ b/src/include/catalog/storage_xlog.h @@ -54,6 +54,6 @@ extern void log_smgrcreate(const RelFileLocator *rlocator, ForkNumber forkNum); extern void smgr_redo(XLogReaderState *record); extern void smgr_desc(StringInfo buf, XLogReaderState *record); -extern const char *smgr_identify(uint8 info); +extern const char *smgr_identify(uint8 rmgrinfo); #endif /* STORAGE_XLOG_H */ diff --git a/src/include/commands/dbcommands_xlog.h b/src/include/commands/dbcommands_xlog.h index a1433275f5..f3c11060b6 100644 --- a/src/include/commands/dbcommands_xlog.h +++ b/src/include/commands/dbcommands_xlog.h @@ -55,6 +55,6 @@ typedef struct xl_dbase_drop_rec extern void dbase_redo(XLogReaderState *record); extern void dbase_desc(StringInfo buf, XLogReaderState *record); -extern const char *dbase_identify(uint8 info); +extern const char *dbase_identify(uint8 rmgrinfo); #endif /* DBCOMMANDS_XLOG_H */ diff --git a/src/include/commands/sequence.h b/src/include/commands/sequence.h index 7db7b3da7b..fb7059b8f6 100644 --- a/src/include/commands/sequence.h +++ b/src/include/commands/sequence.h @@ -64,7 +64,7 @@ extern void ResetSequenceCaches(void); extern void seq_redo(XLogReaderState *record); extern void seq_desc(StringInfo buf, XLogReaderState *record); -extern const char *seq_identify(uint8 info); +extern const char *seq_identify(uint8 rmgrinfo); extern void seq_mask(char *page, BlockNumber blkno); #endif /* SEQUENCE_H */ diff --git a/src/include/commands/tablespace.h b/src/include/commands/tablespace.h index f1961c1813..db21eb59ee 100644 --- a/src/include/commands/tablespace.h +++ b/src/include/commands/tablespace.h @@ -64,6 +64,6 @@ extern void remove_tablespace_symlink(const char *linkloc); extern void tblspc_redo(XLogReaderState *record); extern void tblspc_desc(StringInfo buf, XLogReaderState *record); -extern const char *tblspc_identify(uint8 info); +extern const char *tblspc_identify(uint8 rmgrinfo); #endif /* TABLESPACE_H */ diff --git a/src/include/replication/message.h b/src/include/replication/message.h index 6ce7f2038b..2fe985a6fe 100644 --- a/src/include/replication/message.h +++ b/src/include/replication/message.h @@ -36,6 +36,6 @@ extern XLogRecPtr LogLogicalMessage(const char *prefix, const char *message, #define XLOG_LOGICAL_MESSAGE 0x00 extern void logicalmsg_redo(XLogReaderState *record); extern void logicalmsg_desc(StringInfo buf, XLogReaderState *record); -extern const char *logicalmsg_identify(uint8 info); +extern const char *logicalmsg_identify(uint8 rmgrinfo); #endif /* PG_LOGICAL_MESSAGE_H */ diff --git a/src/include/storage/standbydefs.h b/src/include/storage/standbydefs.h index 188e348618..84e1f03c8f 100644 --- a/src/include/storage/standbydefs.h +++ b/src/include/storage/standbydefs.h @@ -22,7 +22,7 @@ /* Recovery handlers for the Standby Rmgr (RM_STANDBY_ID) */ extern void standby_redo(XLogReaderState *record); extern void standby_desc(StringInfo buf, XLogReaderState *record); -extern const char *standby_identify(uint8 info); +extern const char *standby_identify(uint8 rmgrinfo); extern void standby_desc_invalidations(StringInfo buf, int nmsgs, SharedInvalidationMessage *msgs, Oid dbId, Oid tsId, diff --git a/src/include/utils/relmapper.h b/src/include/utils/relmapper.h index 5c173bdbb3..e3c41ff093 100644 --- a/src/include/utils/relmapper.h +++ b/src/include/utils/relmapper.h @@ -68,6 +68,6 @@ extern void RestoreRelationMap(char *startAddress); extern void relmap_redo(XLogReaderState *record); extern void relmap_desc(StringInfo buf, XLogReaderState *record); -extern const char *relmap_identify(uint8 info); +extern const char *relmap_identify(uint8 rmgrinfo); #endif /* RELMAPPER_H */ diff --git a/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c b/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c index b5537a5f5b..21514f8e33 100644 --- a/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c +++ b/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c @@ -103,9 +103,9 @@ testcustomrmgrs_desc(StringInfo buf, XLogReaderState *record) } const char * -testcustomrmgrs_identify(uint8 info) +testcustomrmgrs_identify(uint8 rmgrinfo) { - if ((info & ~XLR_INFO_MASK) == XLOG_TEST_CUSTOM_RMGRS_MESSAGE) + if (rmgrinfo == XLOG_TEST_CUSTOM_RMGRS_MESSAGE) return "TEST_CUSTOM_RMGRS_MESSAGE"; return NULL; -- 2.40.1 [application/octet-stream] v3-0004-Move-rmgr-info-bits-into-their-own-field-in-the-x.patch (26.1K, ../../CAEze2Wh2e57adcGXWJLo4hRQY-__CmWwaaSbUf9PGGOX5ESrmQ@mail.gmail.com/6-v3-0004-Move-rmgr-info-bits-into-their-own-field-in-the-x.patch) download | inline diff: From 8956da9e1a6bc55a6256272ee4438e8693a87f18 Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Mon, 23 Jan 2023 17:21:12 +0100 Subject: [PATCH v3 4/6] Move rmgr info bits into their own field in the xlog record header. This takes one byte of the 2 previously lost to alignment, and allows each rmgr to use more info bits (useful for e.g. merging the heap rmgrs back into one). --- contrib/pg_walinspect/pg_walinspect.c | 7 +++--- src/backend/access/transam/xact.c | 22 +++++++++-------- src/backend/access/transam/xlog.c | 7 +++--- src/backend/access/transam/xloginsert.c | 27 ++++++++++++--------- src/backend/access/transam/xlogprefetcher.c | 2 +- src/backend/access/transam/xlogreader.c | 2 +- src/backend/access/transam/xlogrecovery.c | 10 ++++---- src/backend/access/transam/xlogstats.c | 12 ++++----- src/backend/catalog/storage.c | 7 +++--- src/backend/commands/dbcommands.c | 20 +++++++++------ src/backend/utils/probes.d | 2 +- src/bin/pg_resetwal/pg_resetwal.c | 3 ++- src/bin/pg_rewind/parsexlog.c | 4 +-- src/bin/pg_waldump/pg_waldump.c | 7 +++--- src/include/access/brin_xlog.h | 3 +-- src/include/access/heapam_xlog.h | 4 +-- src/include/access/nbtxlog.h | 3 +-- src/include/access/xact.h | 6 ++--- src/include/access/xlog_internal.h | 2 +- src/include/access/xloginsert.h | 3 ++- src/include/access/xlogreader.h | 4 +-- src/include/access/xlogrecord.h | 11 ++------- src/include/access/xlogstats.h | 5 ++-- 23 files changed, 89 insertions(+), 84 deletions(-) diff --git a/contrib/pg_walinspect/pg_walinspect.c b/contrib/pg_walinspect/pg_walinspect.c index 1b6f4cb178..909700f9a1 100644 --- a/contrib/pg_walinspect/pg_walinspect.c +++ b/contrib/pg_walinspect/pg_walinspect.c @@ -697,7 +697,7 @@ GetXLogSummaryStats(XLogStats *stats, ReturnSetInfo *rsinfo, { int rj; - for (rj = 0; rj < MAX_XLINFO_TYPES; rj++) + for (rj = 0; rj < MAX_XLRMGRINFO_TYPES; rj++) { const char *id; @@ -712,10 +712,9 @@ GetXLogSummaryStats(XLogStats *stats, ReturnSetInfo *rsinfo, old_cxt = MemoryContextSwitchTo(tmp_cxt); - /* the upper four bits in xl_info are the rmgr's */ - id = desc.rm_identify(rj << 4); + id = desc.rm_identify(rj); if (id == NULL) - id = psprintf("UNKNOWN (%x)", rj << 4); + id = psprintf("UNKNOWN (%x)", rj); FillXLogStatsRow(psprintf("%s/%s", desc.rm_name, id), count, total_count, rec_len, total_rec_len, fpi_len, diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c560f07390..da0ac6c8d3 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -5650,7 +5650,8 @@ XactLogCommitRecord(TimestampTz commit_time, xl_xact_invals xl_invals; xl_xact_twophase xl_twophase; xl_xact_origin xl_origin; - uint8 info; + uint8 info = 0; + uint8 rmgr_info; Assert(CritSectionCount > 0); @@ -5658,9 +5659,9 @@ XactLogCommitRecord(TimestampTz commit_time, /* decide between a plain and 2pc commit */ if (!TransactionIdIsValid(twophase_xid)) - info = XLOG_XACT_COMMIT; + rmgr_info = XLOG_XACT_COMMIT; else - info = XLOG_XACT_COMMIT_PREPARED; + rmgr_info = XLOG_XACT_COMMIT_PREPARED; /* First figure out and collect all the information needed */ @@ -5736,7 +5737,7 @@ XactLogCommitRecord(TimestampTz commit_time, } if (xl_xinfo.xinfo != 0) - info |= XLOG_XACT_HAS_INFO; + rmgr_info |= XLOG_XACT_HAS_INFO; /* Then include all the collected data into the commit record. */ @@ -5794,7 +5795,7 @@ XactLogCommitRecord(TimestampTz commit_time, /* we allow filtering by xacts */ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); - return XLogInsert(RM_XACT_ID, info); + return XLogInsertExtended(RM_XACT_ID, info, rmgr_info); } /* @@ -5820,7 +5821,8 @@ XactLogAbortRecord(TimestampTz abort_time, xl_xact_dbinfo xl_dbinfo; xl_xact_origin xl_origin; - uint8 info; + uint8 info = 0; + uint8 rmgr_info; Assert(CritSectionCount > 0); @@ -5828,9 +5830,9 @@ XactLogAbortRecord(TimestampTz abort_time, /* decide between a plain and 2pc abort */ if (!TransactionIdIsValid(twophase_xid)) - info = XLOG_XACT_ABORT; + rmgr_info = XLOG_XACT_ABORT; else - info = XLOG_XACT_ABORT_PREPARED; + rmgr_info = XLOG_XACT_ABORT_PREPARED; /* First figure out and collect all the information needed */ @@ -5889,7 +5891,7 @@ XactLogAbortRecord(TimestampTz abort_time, } if (xl_xinfo.xinfo != 0) - info |= XLOG_XACT_HAS_INFO; + rmgr_info |= XLOG_XACT_HAS_INFO; /* Then include all the collected data into the abort record. */ @@ -5940,7 +5942,7 @@ XactLogAbortRecord(TimestampTz abort_time, /* Include the replication origin */ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); - return XLogInsert(RM_XACT_ID, info); + return XLogInsertExtended(RM_XACT_ID, info, rmgr_info); } /* diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index b01cd6cc23..4b1ff0d1aa 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -738,9 +738,9 @@ XLogInsertRecord(XLogRecData *rdata, pg_crc32c rdata_crc; bool inserted; XLogRecord *rechdr = (XLogRecord *) rdata->data; - uint8 info = rechdr->xl_info & ~XLR_INFO_MASK; + uint8 rmgrinfo = rechdr->xl_rmgrinfo; bool isLogSwitch = (rechdr->xl_rmid == RM_XLOG_ID && - info == XLOG_SWITCH); + rmgrinfo == XLOG_SWITCH); XLogRecPtr StartPos; XLogRecPtr EndPos; bool prevDoPageWrites = doPageWrites; @@ -4730,7 +4730,8 @@ BootStrapXLOG(void) record->xl_prev = 0; record->xl_xid = InvalidTransactionId; record->xl_tot_len = SizeOfXLogRecord + SizeOfXLogRecordDataHeaderShort + sizeof(checkPoint); - record->xl_info = XLOG_CHECKPOINT_SHUTDOWN; + record->xl_info = 0; + record->xl_rmgrinfo = XLOG_CHECKPOINT_SHUTDOWN; record->xl_rmid = RM_XLOG_ID; recptr += SizeOfXLogRecord; /* fill the XLogRecordDataHeaderShort struct */ diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c index 0fce063ca4..9a3eb4fb4b 100644 --- a/src/backend/access/transam/xloginsert.c +++ b/src/backend/access/transam/xloginsert.c @@ -135,7 +135,7 @@ static bool begininsert_called = false; /* Memory context to hold the registered buffer and data references. */ static MemoryContext xloginsert_cxt; -static XLogRecData *XLogRecordAssemble(RmgrId rmid, uint8 info, +static XLogRecData *XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecPtr, bool doPageWrites, XLogRecPtr *fpw_lsn, int *num_fpi, bool *topxid_included); @@ -447,9 +447,9 @@ XLogSetRecordFlags(uint8 flags) } /* - * Insert an XLOG record having the specified RMID and info bytes, with the - * body of the record being the data and buffer references registered earlier - * with XLogRegister* calls. + * Insert an XLOG record having the specified RMID and rmgr_info bytes, with + * the body of the record being the data and buffer references registered + * earlier with XLogRegister* calls. * * Returns XLOG pointer to end of record (beginning of next record). * This can be used as LSN for data pages affected by the logged action. @@ -458,7 +458,13 @@ XLogSetRecordFlags(uint8 flags) * WAL rule "write the log before the data".) */ XLogRecPtr -XLogInsert(RmgrId rmid, uint8 info) +XLogInsert(RmgrId rmid, uint8 rmgr_info) +{ + return XLogInsertExtended(rmid, 0, rmgr_info); +} + +XLogRecPtr +XLogInsertExtended(RmgrId rmid, uint8 info, uint8 rmgr_info) { XLogRecPtr EndPos; @@ -470,12 +476,11 @@ XLogInsert(RmgrId rmid, uint8 info) * The caller can set rmgr bits, XLR_SPECIAL_REL_UPDATE and * XLR_CHECK_CONSISTENCY; the rest are reserved for use by me. */ - if ((info & ~(XLR_RMGR_INFO_MASK | - XLR_SPECIAL_REL_UPDATE | + if ((info & ~(XLR_SPECIAL_REL_UPDATE | XLR_CHECK_CONSISTENCY)) != 0) elog(PANIC, "invalid xlog info mask %02X", info); - TRACE_POSTGRESQL_WAL_INSERT(rmid, info); + TRACE_POSTGRESQL_WAL_INSERT(rmid, info, rmgr_info); /* * In bootstrap mode, we don't actually log anything but XLOG resources; @@ -504,7 +509,7 @@ XLogInsert(RmgrId rmid, uint8 info) */ GetFullPageWriteInfo(&RedoRecPtr, &doPageWrites); - rdt = XLogRecordAssemble(rmid, info, RedoRecPtr, doPageWrites, + rdt = XLogRecordAssemble(rmid, info, rmgr_info, RedoRecPtr, doPageWrites, &fpw_lsn, &num_fpi, &topxid_included); EndPos = XLogInsertRecord(rdt, fpw_lsn, curinsert_flags, num_fpi, @@ -532,8 +537,7 @@ XLogInsert(RmgrId rmid, uint8 info) * current subtransaction. */ static XLogRecData * -XLogRecordAssemble(RmgrId rmid, uint8 info, - XLogRecPtr RedoRecPtr, bool doPageWrites, +XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecPtr, bool doPageWrites, XLogRecPtr *fpw_lsn, int *num_fpi, bool *topxid_included) { XLogRecData *rdt; @@ -924,6 +928,7 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, rechdr->xl_tot_len = (uint32) total_len; rechdr->xl_info = info; rechdr->xl_rmid = rmid; + rechdr->xl_rmgrinfo = rmgr_info; rechdr->xl_prev = InvalidXLogRecPtr; rechdr->xl_crc = rdata_crc; diff --git a/src/backend/access/transam/xlogprefetcher.c b/src/backend/access/transam/xlogprefetcher.c index 539928cb85..dd4ca866a4 100644 --- a/src/backend/access/transam/xlogprefetcher.c +++ b/src/backend/access/transam/xlogprefetcher.c @@ -536,7 +536,7 @@ XLogPrefetcherNextBlock(uintptr_t pgsr_private, XLogRecPtr *lsn) if (replaying_lsn < record->lsn) { uint8 rmid = record->header.xl_rmid; - uint8 record_type = record->header.xl_info & ~XLR_INFO_MASK; + uint8 record_type = record->header.xl_rmgrinfo; if (rmid == RM_XLOG_ID) { diff --git a/src/backend/access/transam/xlogreader.c b/src/backend/access/transam/xlogreader.c index 24c6c88b2f..733de8a8dc 100644 --- a/src/backend/access/transam/xlogreader.c +++ b/src/backend/access/transam/xlogreader.c @@ -860,7 +860,7 @@ restart: * Special processing if it's an XLOG SWITCH record */ if (record->xl_rmid == RM_XLOG_ID && - (record->xl_info & ~XLR_INFO_MASK) == XLOG_SWITCH) + record->xl_rmgrinfo == XLOG_SWITCH) { /* Pretend it extends to end of segment */ state->NextRecPtr += state->segcxt.ws_segsize - 1; diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index 6508246999..56fa2f74a2 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -632,7 +632,7 @@ InitWalRecovery(ControlFileData *ControlFile, bool *wasShutdown_ptr, if (record != NULL) { memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint)); - wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN); + wasShutdown = (record->xl_rmgrinfo == XLOG_CHECKPOINT_SHUTDOWN); ereport(DEBUG1, (errmsg_internal("checkpoint record is at %X/%X", LSN_FORMAT_ARGS(CheckPointLoc)))); @@ -786,7 +786,7 @@ InitWalRecovery(ControlFileData *ControlFile, bool *wasShutdown_ptr, (errmsg("could not locate a valid checkpoint record"))); } memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint)); - wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN); + wasShutdown = (record->xl_rmgrinfo == XLOG_CHECKPOINT_SHUTDOWN); } /* @@ -1857,7 +1857,7 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl { TimeLineID newReplayTLI = *replayTLI; TimeLineID prevReplayTLI = *replayTLI; - uint8 info = record->xl_info & ~XLR_INFO_MASK; + uint8 info = record->xl_rmgrinfo; if (info == XLOG_CHECKPOINT_SHUTDOWN) { @@ -3992,12 +3992,12 @@ ReadCheckpointRecord(XLogPrefetcher *xlogprefetcher, XLogRecPtr RecPtr, (errmsg("invalid resource manager ID in checkpoint record"))); return NULL; } - info = record->xl_info & ~XLR_INFO_MASK; + info = record->xl_rmgrinfo; if (info != XLOG_CHECKPOINT_SHUTDOWN && info != XLOG_CHECKPOINT_ONLINE) { ereport(LOG, - (errmsg("invalid xl_info in checkpoint record"))); + (errmsg("invalid xl_rmgrinfo in checkpoint record"))); return NULL; } if (record->xl_tot_len != SizeOfXLogRecord + SizeOfXLogRecordDataHeaderShort + sizeof(CheckPoint)) diff --git a/src/backend/access/transam/xlogstats.c b/src/backend/access/transam/xlogstats.c index 267b108943..28d6d971ac 100644 --- a/src/backend/access/transam/xlogstats.c +++ b/src/backend/access/transam/xlogstats.c @@ -74,21 +74,21 @@ XLogRecStoreStats(XLogStats *stats, XLogReaderState *record) /* * Update per-record statistics, where the record is identified by a - * combination of the RmgrId and the four bits of the xl_info field that - * are the rmgr's domain (resulting in sixteen possible entries per + * combination of the RmgrId and the eight bits of the xl_rmgrinfo field + * that are the rmgr's domain (resulting in 256 possible entries per * RmgrId). */ - recid = XLogRecGetRmgrInfo(record) >> 4; + recid = XLogRecGetRmgrInfo(record); /* * XACT records need to be handled differently. Those records use the * first bit of those four bits for an optional flag variable and the - * following three bits for the opcode. We filter opcode out of xl_info - * and use it as the identifier of the record. + * following three bits for the opcode. We filter opcode out of + * xl_rmgrinfo and use it as the identifier of the record. */ if (rmid == RM_XACT_ID) - recid &= 0x07; + recid &= 0x70; stats->record_stats[rmid][recid].count++; stats->record_stats[rmid][recid].rec_len += rec_len; diff --git a/src/backend/catalog/storage.c b/src/backend/catalog/storage.c index 4d73023a65..5b48b24e7b 100644 --- a/src/backend/catalog/storage.c +++ b/src/backend/catalog/storage.c @@ -194,7 +194,7 @@ log_smgrcreate(const RelFileLocator *rlocator, ForkNumber forkNum) XLogBeginInsert(); XLogRegisterData((char *) &xlrec, sizeof(xlrec)); - XLogInsert(RM_SMGR_ID, XLOG_SMGR_CREATE | XLR_SPECIAL_REL_UPDATE); + XLogInsertExtended(RM_SMGR_ID, XLR_SPECIAL_REL_UPDATE, XLOG_SMGR_CREATE); } /* @@ -375,8 +375,9 @@ RelationTruncate(Relation rel, BlockNumber nblocks) XLogBeginInsert(); XLogRegisterData((char *) &xlrec, sizeof(xlrec)); - lsn = XLogInsert(RM_SMGR_ID, - XLOG_SMGR_TRUNCATE | XLR_SPECIAL_REL_UPDATE); + lsn = XLogInsertExtended(RM_SMGR_ID, + XLR_SPECIAL_REL_UPDATE, + XLOG_SMGR_TRUNCATE); /* * Flush, because otherwise the truncation of the main relation might diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c index dffb683e74..fe89540aed 100644 --- a/src/backend/commands/dbcommands.c +++ b/src/backend/commands/dbcommands.c @@ -624,8 +624,9 @@ CreateDatabaseUsingFileCopy(Oid src_dboid, Oid dst_dboid, Oid src_tsid, XLogRegisterData((char *) &xlrec, sizeof(xl_dbase_create_file_copy_rec)); - (void) XLogInsert(RM_DBASE_ID, - XLOG_DBASE_CREATE_FILE_COPY | XLR_SPECIAL_REL_UPDATE); + (void) XLogInsertExtended(RM_DBASE_ID, + XLR_SPECIAL_REL_UPDATE, + XLOG_DBASE_CREATE_FILE_COPY); } pfree(srcpath); pfree(dstpath); @@ -2086,8 +2087,9 @@ movedb(const char *dbname, const char *tblspcname) XLogRegisterData((char *) &xlrec, sizeof(xl_dbase_create_file_copy_rec)); - (void) XLogInsert(RM_DBASE_ID, - XLOG_DBASE_CREATE_FILE_COPY | XLR_SPECIAL_REL_UPDATE); + (void) XLogInsertExtended(RM_DBASE_ID, + XLR_SPECIAL_REL_UPDATE, + XLOG_DBASE_CREATE_FILE_COPY); } /* @@ -2180,8 +2182,9 @@ movedb(const char *dbname, const char *tblspcname) XLogRegisterData((char *) &xlrec, sizeof(xl_dbase_drop_rec)); XLogRegisterData((char *) &src_tblspcoid, sizeof(Oid)); - (void) XLogInsert(RM_DBASE_ID, - XLOG_DBASE_DROP | XLR_SPECIAL_REL_UPDATE); + (void) XLogInsertExtended(RM_DBASE_ID, + XLR_SPECIAL_REL_UPDATE, + XLOG_DBASE_DROP); } /* Now it's safe to release the database lock */ @@ -2903,8 +2906,9 @@ remove_dbtablespaces(Oid db_id) XLogRegisterData((char *) &xlrec, MinSizeOfDbaseDropRec); XLogRegisterData((char *) tablespace_ids, ntblspc * sizeof(Oid)); - (void) XLogInsert(RM_DBASE_ID, - XLOG_DBASE_DROP | XLR_SPECIAL_REL_UPDATE); + (void) XLogInsertExtended(RM_DBASE_ID, + XLR_SPECIAL_REL_UPDATE, + XLOG_DBASE_DROP); } list_free(ltblspc); diff --git a/src/backend/utils/probes.d b/src/backend/utils/probes.d index 0af275587b..131067b5ac 100644 --- a/src/backend/utils/probes.d +++ b/src/backend/utils/probes.d @@ -87,7 +87,7 @@ provider postgresql { probe smgr__md__write__start(ForkNumber, BlockNumber, Oid, Oid, Oid, int); probe smgr__md__write__done(ForkNumber, BlockNumber, Oid, Oid, Oid, int, int, int); - probe wal__insert(unsigned char, unsigned char); + probe wal__insert(unsigned char, unsigned char, unsigned char); probe wal__switch(); probe wal__buffer__write__dirty__start(); probe wal__buffer__write__dirty__done(); diff --git a/src/bin/pg_resetwal/pg_resetwal.c b/src/bin/pg_resetwal/pg_resetwal.c index e7ef2b8bd0..80cb9a5bc4 100644 --- a/src/bin/pg_resetwal/pg_resetwal.c +++ b/src/bin/pg_resetwal/pg_resetwal.c @@ -1066,7 +1066,8 @@ WriteEmptyXLOG(void) record->xl_prev = 0; record->xl_xid = InvalidTransactionId; record->xl_tot_len = SizeOfXLogRecord + SizeOfXLogRecordDataHeaderShort + sizeof(CheckPoint); - record->xl_info = XLOG_CHECKPOINT_SHUTDOWN; + record->xl_info = 0; + record->xl_rmgrinfo = XLOG_CHECKPOINT_SHUTDOWN; record->xl_rmid = RM_XLOG_ID; recptr += SizeOfXLogRecord; diff --git a/src/bin/pg_rewind/parsexlog.c b/src/bin/pg_rewind/parsexlog.c index 2b6b46f0e3..aa03fe8031 100644 --- a/src/bin/pg_rewind/parsexlog.c +++ b/src/bin/pg_rewind/parsexlog.c @@ -438,9 +438,9 @@ extractPageInfo(XLogReaderState *record) * track that change. */ pg_fatal("WAL record modifies a relation, but record type is not recognized: " - "lsn: %X/%X, rmid: %d, rmgr: %s, info: %02X", + "lsn: %X/%X, rmid: %d, rmgr: %s, info: %02X, rmgrinfo: %02X", LSN_FORMAT_ARGS(record->ReadRecPtr), - rmid, RmgrName(rmid), info | rminfo); + rmid, RmgrName(rmid), info, rminfo); } for (block_id = 0; block_id <= XLogRecMaxBlockId(record); block_id++) diff --git a/src/bin/pg_waldump/pg_waldump.c b/src/bin/pg_waldump/pg_waldump.c index 85694b3d5c..713c526385 100644 --- a/src/bin/pg_waldump/pg_waldump.c +++ b/src/bin/pg_waldump/pg_waldump.c @@ -694,7 +694,7 @@ XLogDumpDisplayStats(XLogDumpConfig *config, XLogStats *stats) } else { - for (rj = 0; rj < MAX_XLINFO_TYPES; rj++) + for (rj = 0; rj < MAX_XLRMGRINFO_TYPES; rj++) { const char *id; @@ -707,10 +707,9 @@ XLogDumpDisplayStats(XLogDumpConfig *config, XLogStats *stats) if (count == 0) continue; - /* the upper four bits in xl_info are the rmgr's */ - id = desc->rm_identify(rj << 4); + id = desc->rm_identify(rj); if (id == NULL) - id = psprintf("UNKNOWN (%x)", rj << 4); + id = psprintf("UNKNOWN (%x)", rj); XLogDumpStatsRow(psprintf("%s/%s", desc->rm_name, id), count, total_count, rec_len, total_rec_len, diff --git a/src/include/access/brin_xlog.h b/src/include/access/brin_xlog.h index 45a688d09b..92190688c9 100644 --- a/src/include/access/brin_xlog.h +++ b/src/include/access/brin_xlog.h @@ -25,8 +25,7 @@ /* * WAL record definitions for BRIN's WAL operations * - * XLOG allows to store some information in high 4 bits of log - * record xl_info field. + * XLOG allows to store some information in the 8-bit xl_rmgrinfo field. */ #define XLOG_BRIN_CREATE_INDEX 0x00 #define XLOG_BRIN_INSERT 0x10 diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h index 70e9acd350..6705c8a316 100644 --- a/src/include/access/heapam_xlog.h +++ b/src/include/access/heapam_xlog.h @@ -26,8 +26,8 @@ /* * WAL record definitions for heapam.c's WAL operations * - * XLOG allows to store some information in high 4 bits of log - * record xl_info field. We use 3 for opcode and one for init bit. + * XLOG allows to store some information in the 8-bit xl_rmgrinfo field. + * We use 3 for opcode and one for init bit. */ #define XLOG_HEAP_INSERT 0x00 #define XLOG_HEAP_DELETE 0x10 diff --git a/src/include/access/nbtxlog.h b/src/include/access/nbtxlog.h index 422ac13d56..6d1231a01e 100644 --- a/src/include/access/nbtxlog.h +++ b/src/include/access/nbtxlog.h @@ -21,8 +21,7 @@ /* * XLOG records for btree operations * - * XLOG allows to store some information in high 4 bits of log - * record xl_info field + * XLOG allows to store some information in the 8-bit xl_rmgrinfo field. */ #define XLOG_BTREE_INSERT_LEAF 0x00 /* add index tuple without split */ #define XLOG_BTREE_INSERT_UPPER 0x10 /* same, on a non-leaf page */ diff --git a/src/include/access/xact.h b/src/include/access/xact.h index b94b264402..2a9a708279 100644 --- a/src/include/access/xact.h +++ b/src/include/access/xact.h @@ -163,8 +163,8 @@ typedef struct SavedTransactionCharacteristics */ /* - * XLOG allows to store some information in high 4 bits of log record xl_info - * field. We use 3 for the opcode, and one about an optional flag variable. + * XLOG allows to store some information in the 8-bit xl_rmgrinfo field. + * We use 3 for the opcode, and one about an optional flag variable. */ #define XLOG_XACT_COMMIT 0x00 #define XLOG_XACT_PREPARE 0x10 @@ -232,7 +232,7 @@ typedef struct xl_xact_assignment * A minimal commit/abort record only consists of a xl_xact_commit/abort * struct. The presence of additional information is indicated by bits set in * 'xl_xact_xinfo->xinfo'. The presence of the xinfo field itself is signaled - * by a set XLOG_XACT_HAS_INFO bit in the xl_info field. + * by a set XLOG_XACT_HAS_INFO bit in the xl_rmgrinfo field. * * NB: All the individual data chunks should be sized to multiples of * sizeof(int) and only require int32 alignment. If they require bigger diff --git a/src/include/access/xlog_internal.h b/src/include/access/xlog_internal.h index 42b3c66547..876e2790f9 100644 --- a/src/include/access/xlog_internal.h +++ b/src/include/access/xlog_internal.h @@ -448,7 +448,7 @@ static inline Size XLogReadLength(uint32 *length, XLogSizeClass sizeClass, * This struct must be kept in sync with the PG_RMGR definition in * rmgr.c. * - * rm_identify must return a name for the record based on xl_info (without + * rm_identify must return a name for the record based on xl_rmgrinfo (without * reference to the rmid). For example, XLOG_BTREE_VACUUM would be named * "VACUUM". rm_desc can then be called to obtain additional detail for the * record, if available (e.g. the last block). diff --git a/src/include/access/xloginsert.h b/src/include/access/xloginsert.h index 31785dc578..c6c46e5409 100644 --- a/src/include/access/xloginsert.h +++ b/src/include/access/xloginsert.h @@ -41,7 +41,8 @@ /* prototypes for public functions in xloginsert.c: */ extern void XLogBeginInsert(void); extern void XLogSetRecordFlags(uint8 flags); -extern XLogRecPtr XLogInsert(RmgrId rmid, uint8 info); +extern XLogRecPtr XLogInsertExtended(RmgrId rmid, uint8 info, uint8 rmgr_info); +extern XLogRecPtr XLogInsert(RmgrId rmid, uint8 rmgr_info); extern void XLogEnsureRecordSpace(int max_block_id, int ndatas); extern void XLogRegisterData(char *data, uint32 len); extern void XLogRegisterBuffer(uint8 block_id, Buffer buffer, uint8 flags); diff --git a/src/include/access/xlogreader.h b/src/include/access/xlogreader.h index 2b976dcc03..a1d0216404 100644 --- a/src/include/access/xlogreader.h +++ b/src/include/access/xlogreader.h @@ -407,8 +407,8 @@ extern bool DecodeXLogRecord(XLogReaderState *state, */ #define XLogRecGetTotalLen(decoder) ((decoder)->record->header.xl_tot_len) #define XLogRecGetPrev(decoder) ((decoder)->record->header.xl_prev) -#define XLogRecGetInfo(decoder) ((decoder)->record->header.xl_info & XLR_INFO_MASK) -#define XLogRecGetRmgrInfo(decoder) (((decoder)->record->header.xl_info) & XLR_RMGR_INFO_MASK) +#define XLogRecGetInfo(decoder) ((decoder)->record->header.xl_info) +#define XLogRecGetRmgrInfo(decoder) ((decoder)->record->header.xl_rmgrinfo) #define XLogRecGetRmid(decoder) ((decoder)->record->header.xl_rmid) #define XLogRecGetXid(decoder) ((decoder)->record->header.xl_xid) #define XLogRecGetOrigin(decoder) ((decoder)->record->record_origin) diff --git a/src/include/access/xlogrecord.h b/src/include/access/xlogrecord.h index b0aada1031..42b06f163e 100644 --- a/src/include/access/xlogrecord.h +++ b/src/include/access/xlogrecord.h @@ -66,7 +66,8 @@ typedef struct XLogRecord XLogRecPtr xl_prev; /* ptr to previous record in log */ uint8 xl_info; /* flag bits, see below */ RmgrId xl_rmid; /* resource manager for this record */ - /* 2 bytes of padding here, initialize to zero */ + uint8 xl_rmgrinfo; /* rmgr flag bits, see below */ + /* 1 byte of padding here, initialize to zero */ pg_crc32c xl_crc; /* CRC for this record */ /* XLogRecordBlockHeaders and XLogRecordDataHeader follow, no padding */ @@ -75,14 +76,6 @@ typedef struct XLogRecord #define SizeOfXLogRecord (offsetof(XLogRecord, xl_crc) + sizeof(pg_crc32c)) -/* - * The high 4 bits in xl_info may be used freely by rmgr. The - * XLR_SPECIAL_REL_UPDATE and XLR_CHECK_CONSISTENCY bits can be passed by - * XLogInsert caller. The rest are set internally by XLogInsert. - */ -#define XLR_INFO_MASK 0x0F -#define XLR_RMGR_INFO_MASK 0xF0 - /* * XLogReader needs to allocate all the data of a WAL record in a single * chunk. This means that a single XLogRecord cannot exceed MaxAllocSize diff --git a/src/include/access/xlogstats.h b/src/include/access/xlogstats.h index 89410ce92b..30974f5ba7 100644 --- a/src/include/access/xlogstats.h +++ b/src/include/access/xlogstats.h @@ -16,7 +16,8 @@ #include "access/rmgr.h" #include "access/xlogreader.h" -#define MAX_XLINFO_TYPES 16 +/* all 8 bits of xl_rmgrinfo are available */ +#define MAX_XLRMGRINFO_TYPES 256 typedef struct XLogRecStats { @@ -33,7 +34,7 @@ typedef struct XLogStats XLogRecPtr endptr; #endif XLogRecStats rmgr_stats[RM_MAX_ID + 1]; - XLogRecStats record_stats[RM_MAX_ID + 1][MAX_XLINFO_TYPES]; + XLogRecStats record_stats[RM_MAX_ID + 1][MAX_XLRMGRINFO_TYPES]; } XLogStats; extern void XLogRecGetLen(XLogReaderState *record, uint32 *rec_len, -- 2.40.1 [application/octet-stream] v3-0006-Reformat-the-XLog-record-header.patch (44.2K, ../../CAEze2Wh2e57adcGXWJLo4hRQY-__CmWwaaSbUf9PGGOX5ESrmQ@mail.gmail.com/7-v3-0006-Reformat-the-XLog-record-header.patch) download | inline diff: From ca47bb00fad1acac29d86cb76806b1e9310e6168 Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Mon, 19 Jun 2023 17:46:45 +0200 Subject: [PATCH v3 6/6] Reformat the XLog record header Many WAL records don't utilize all bytes of the XLog header, resulting in a lot of wasted bytes. This new XLog header format saves bytes in several ways: - Records that don't need the XID won't include it This potentially saves 4 bytes /record - Records that are small enough don't need 4 bytes to describe their length By varint encoding the length, we save up to 3 bytes /record (up to 4 bytes /record if dataless records are considered) - Only include xl_rmgrinfo if it is non-0 Several RMGRs have a record that they use most often. We save 1 byte/record if that record id is 0. - Lose some alignment losses We save 1 byte/record by removing one byte of alignment losses --- src/backend/access/transam/xlog.c | 133 ++++++++++++++------- src/backend/access/transam/xloginsert.c | 135 ++++++++++++++++------ src/backend/access/transam/xlogreader.c | 118 +++++++++++-------- src/backend/access/transam/xlogrecovery.c | 4 +- src/bin/pg_resetwal/pg_resetwal.c | 38 +++--- src/include/access/rmgr.h | 2 +- src/include/access/rmgrlist.h | 1 + src/include/access/xlog_internal.h | 102 +++++++++++++++- src/include/access/xlogreader.h | 4 +- src/include/access/xlogrecord.h | 125 +++++++++++++++++--- 10 files changed, 502 insertions(+), 160 deletions(-) diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 4b1ff0d1aa..5937b43f50 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -737,17 +737,18 @@ XLogInsertRecord(XLogRecData *rdata, XLogCtlInsert *Insert = &XLogCtl->Insert; pg_crc32c rdata_crc; bool inserted; - XLogRecord *rechdr = (XLogRecord *) rdata->data; - uint8 rmgrinfo = rechdr->xl_rmgrinfo; - bool isLogSwitch = (rechdr->xl_rmid == RM_XLOG_ID && - rmgrinfo == XLOG_SWITCH); + XLogRecHdr rechdr; + bool isLogSwitch; XLogRecPtr StartPos; XLogRecPtr EndPos; + XLogRecPtr xl_prev; bool prevDoPageWrites = doPageWrites; TimeLineID insertTLI; + int rec_payload_len, + rec_hdr_len; - /* we assume that all of the record header is in the first chunk */ - Assert(rdata->len >= SizeOfXLogRecord); + /* we assume that all data of the record header is in the first chunk */ + Assert(rdata->len >= XLogRecordMinHdrSize); /* cross-check on whether we should be here or not */ if (!XLogInsertAllowed()) @@ -758,6 +759,17 @@ XLogInsertRecord(XLogRecData *rdata, * change, so we can read it without a lock. */ insertTLI = XLogCtl->InsertTimeLineID; + rechdr = (XLogRecHdr) rdata->data; + + { + XLogRecord record = {0}; + XLogReadRecHdrInto(rechdr, (Size) rdata->len, &record); + + isLogSwitch = (record.xl_rmid == RM_XLOG_ID && + record.xl_rmgrinfo == XLOG_SWITCH); + rec_hdr_len = (int) XLogRecordHdrLen(rechdr->xl_info); + rec_payload_len = (int) record.xl_payload_len; + } /*---------- * @@ -834,34 +846,44 @@ XLogInsertRecord(XLogRecData *rdata, } /* - * Reserve space for the record in the WAL. This also sets the xl_prev - * pointer. + * Reserve space for the record in the WAL. */ if (isLogSwitch) - inserted = ReserveXLogSwitch(&StartPos, &EndPos, &rechdr->xl_prev); + inserted = ReserveXLogSwitch(&StartPos, &EndPos, &xl_prev); else { - ReserveXLogInsertLocation(rechdr->xl_tot_len, &StartPos, &EndPos, - &rechdr->xl_prev); + ReserveXLogInsertLocation(rec_hdr_len + rec_payload_len, + &StartPos, &EndPos, &xl_prev); inserted = true; } if (inserted) { + char *rec = (char *) rechdr; + + /* fill in xl_prev */ + memcpy(rec + rec_hdr_len - sizeof(pg_crc32c) - sizeof(XLogRecPtr), + &xl_prev, sizeof(XLogRecPtr)); + /* * Now that xl_prev has been filled in, calculate CRC of the record * header. */ - rdata_crc = rechdr->xl_crc; - COMP_CRC32C(rdata_crc, rechdr, offsetof(XLogRecord, xl_crc)); + + memcpy(&rdata_crc, rec + rec_hdr_len - sizeof(pg_crc32c), + sizeof(pg_crc32c)); + + COMP_CRC32C(rdata_crc, rec, rec_hdr_len - sizeof(pg_crc32c)); FIN_CRC32C(rdata_crc); - rechdr->xl_crc = rdata_crc; + + memcpy(rec + rec_hdr_len - sizeof(pg_crc32c), + &rdata_crc, sizeof(pg_crc32c)); /* * All the record data, including the header, is now ready to be * inserted. Copy the record in the space reserved. */ - CopyXLogRecordToWAL(rechdr->xl_tot_len, isLogSwitch, rdata, + CopyXLogRecordToWAL(rec_hdr_len + rec_payload_len, isLogSwitch, rdata, StartPos, EndPos, insertTLI); /* @@ -932,7 +954,8 @@ XLogInsertRecord(XLogRecData *rdata, */ if (inserted) { - EndPos = StartPos + SizeOfXLogRecord; + /* xlog switch is minimal record header, plus a byte for rmgrinfo */ + EndPos = StartPos + XLogRecordMinHdrSize + sizeof(uint8); if (StartPos / XLOG_BLCKSZ != EndPos / XLOG_BLCKSZ) { uint64 offset = XLogSegmentOffset(EndPos, wal_segment_size); @@ -949,7 +972,7 @@ XLogInsertRecord(XLogRecData *rdata, if (XLOG_DEBUG) { static XLogReaderState *debug_reader = NULL; - XLogRecord *record; + XLogRecHdr record; DecodedXLogRecord *decoded; StringInfoData buf; StringInfoData recordBuf; @@ -971,9 +994,9 @@ XLogInsertRecord(XLogRecData *rdata, appendBinaryStringInfo(&recordBuf, rdata->data, rdata->len); /* We also need temporary space to decode the record. */ - record = (XLogRecord *) recordBuf.data; + record = (XLogRecHdr) recordBuf.data; decoded = (DecodedXLogRecord *) - palloc(DecodeXLogRecordRequiredSpace(record->xl_tot_len)); + palloc(DecodeXLogRecordRequiredSpace(recordBuf.len)); if (!debug_reader) debug_reader = XLogReaderAllocate(wal_segment_size, NULL, @@ -1018,7 +1041,7 @@ XLogInsertRecord(XLogRecData *rdata, /* Report WAL traffic to the instrumentation. */ if (inserted) { - pgWalUsage.wal_bytes += rechdr->xl_tot_len; + pgWalUsage.wal_bytes += rec_hdr_len + rec_payload_len; pgWalUsage.wal_records++; pgWalUsage.wal_fpi += num_fpi; } @@ -1049,10 +1072,10 @@ ReserveXLogInsertLocation(int size, XLogRecPtr *StartPos, XLogRecPtr *EndPos, uint64 endbytepos; uint64 prevbytepos; - size = MAXALIGN(size); + size = XLP_ALIGN(size); /* All (non xlog-switch) records should contain data. */ - Assert(size > SizeOfXLogRecord); + Assert(size > XLogRecordMinHdrSize); /* * The duration the spinlock needs to be held is minimized by minimizing @@ -1103,7 +1126,7 @@ ReserveXLogSwitch(XLogRecPtr *StartPos, XLogRecPtr *EndPos, XLogRecPtr *PrevPtr) uint64 startbytepos; uint64 endbytepos; uint64 prevbytepos; - uint32 size = MAXALIGN(SizeOfXLogRecord); + uint32 size = XLP_ALIGN(XLogRecordMinHdrSize + sizeof(uint8)); XLogRecPtr ptr; uint32 segleft; @@ -1176,10 +1199,10 @@ CopyXLogRecordToWAL(int write_len, bool isLogSwitch, XLogRecData *rdata, freespace = INSERT_FREESPACE(CurrPos); /* - * there should be enough space for at least the first field (xl_tot_len) + * there should be enough space for at least the first XL_ALIGN quantum * on this page. */ - Assert(freespace >= sizeof(uint32)); + Assert(freespace >= XLP_ALIGN(1)); /* Copy record data */ written = 0; @@ -1247,7 +1270,7 @@ CopyXLogRecordToWAL(int write_len, bool isLogSwitch, XLogRecData *rdata, if (isLogSwitch && XLogSegmentOffset(CurrPos, wal_segment_size) != 0) { /* An xlog-switch record doesn't contain any data besides the header */ - Assert(write_len == SizeOfXLogRecord); + Assert(write_len == XLogRecordMinHdrSize + sizeof(uint8)); /* Assert that we did reserve the right amount of space */ Assert(XLogSegmentOffset(EndPos, wal_segment_size) == 0); @@ -1291,7 +1314,7 @@ CopyXLogRecordToWAL(int write_len, bool isLogSwitch, XLogRecData *rdata, else { /* Align the end position, so that the next record starts aligned */ - CurrPos = MAXALIGN64(CurrPos); + CurrPos = XLP_ALIGN(CurrPos); } if (CurrPos != EndPos) @@ -4649,11 +4672,15 @@ BootStrapXLOG(void) char *buffer; XLogPageHeader page; XLogLongPageHeader longpage; - XLogRecord *record; + XLogRecHdr rec_hdr; + char *baserecptr; char *recptr; uint64 sysidentifier; struct timeval tv; pg_crc32c crc; + const int rec_hdr_len = offsetof(XLogRecHdrData, xl_hdrdata) + + sizeof(uint8) + sizeof(XLogRecPtr) + sizeof(pg_crc32c); + const int rec_payload_len = sizeof(uint8) * 2 + sizeof(CheckPoint); /* allow ordinary WAL segment creation, like StartupXLOG() would */ SetInstallXLogFileSegmentActive(); @@ -4724,28 +4751,52 @@ BootStrapXLOG(void) longpage->xlp_seg_size = wal_segment_size; longpage->xlp_xlog_blcksz = XLOG_BLCKSZ; + /* if this changes, we need to update the code below */ + Assert(XLogLengthToSizeClass(SizeOfXLogRecordDataHeaderShort + sizeof(checkPoint), XLS_UINT32) == XLS_UINT8); + /* if this changes, we need to add XLR_HAS_RMGRINFO */ + Assert(XLOG_CHECKPOINT_SHUTDOWN == 0); + /* Insert the initial checkpoint record */ - recptr = ((char *) page + SizeOfXLogLongPHD); - record = (XLogRecord *) recptr; - record->xl_prev = 0; - record->xl_xid = InvalidTransactionId; - record->xl_tot_len = SizeOfXLogRecord + SizeOfXLogRecordDataHeaderShort + sizeof(checkPoint); - record->xl_info = 0; - record->xl_rmgrinfo = XLOG_CHECKPOINT_SHUTDOWN; - record->xl_rmid = RM_XLOG_ID; - recptr += SizeOfXLogRecord; + baserecptr = recptr = (((char *) page) + SizeOfXLogLongPHD); + + rec_hdr = (XLogRecHdr) baserecptr; + rec_hdr->xl_info = (char) XLS_UINT8; + rec_hdr->xl_rmid = (char) RM_XLOG_ID; + recptr += offsetof(XLogRecHdrData, xl_hdrdata); + + recptr += XLogWriteLength(SizeOfXLogRecordDataHeaderShort + sizeof(checkPoint), + XLS_UINT8, XLS_UINT8, recptr); + + /* include prevptr */ + { + XLogRecPtr prevptr = 0; + memcpy(recptr, &prevptr, sizeof(XLogRecPtr)); + recptr += sizeof(XLogRecPtr); + } + + /* reserve location of crc */ + recptr += sizeof(pg_crc32c); + + Assert(recptr - baserecptr == XLogRecordHdrLen(rec_hdr->xl_info)); + Assert(recptr - baserecptr == rec_hdr_len); + /* fill the XLogRecordDataHeaderShort struct */ *(recptr++) = (char) XLR_BLOCK_ID_DATA_SHORT; *(recptr++) = sizeof(checkPoint); memcpy(recptr, &checkPoint, sizeof(checkPoint)); recptr += sizeof(checkPoint); - Assert(recptr - (char *) record == record->xl_tot_len); + + /* Assert length of record matches expectations */ + Assert(recptr - baserecptr == XLogRecordTotalLength(rec_hdr)); + Assert((rec_hdr)->xl_info == XLS_UINT8); INIT_CRC32C(crc); - COMP_CRC32C(crc, ((char *) record) + SizeOfXLogRecord, record->xl_tot_len - SizeOfXLogRecord); - COMP_CRC32C(crc, (char *) record, offsetof(XLogRecord, xl_crc)); + COMP_CRC32C(crc, baserecptr + rec_hdr_len, rec_payload_len); + COMP_CRC32C(crc, baserecptr, rec_hdr_len - sizeof(pg_crc32c)); FIN_CRC32C(crc); - record->xl_crc = crc; + + memcpy(baserecptr + rec_hdr_len - sizeof(pg_crc32c), + &crc, sizeof(pg_crc32c)); /* Create first XLOG segment file */ openLogTLI = BootstrapTimeLineID; diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c index 4816c5284b..194196f596 100644 --- a/src/backend/access/transam/xloginsert.c +++ b/src/backend/access/transam/xloginsert.c @@ -545,12 +545,16 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP XLogRecPtr *fpw_lsn, int *num_fpi, bool *topxid_included) { XLogRecData *rdt; - uint64 total_len = 0; + uint64 payload_len = 0; + XLogSizeClass payload_sizeclass = XLS_EMPTY; int block_id; pg_crc32c rdata_crc; registered_buffer *prev_regbuf = NULL; XLogRecData *rdt_datas_last; - XLogRecord *rechdr; + TransactionId xid; + XLogRecHdr rechdr; + uint8 xlr_flags = 0; + uint32 rec_hdr_len; char *scratch = hdr_scratch; /* @@ -558,9 +562,10 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP * All the modifications we do to the rdata chains below must handle that. */ - /* The record begins with the fixed-size header */ - rechdr = (XLogRecord *) scratch; - scratch += SizeOfXLogRecord; + /* The record begins with the variable-size header */ + rechdr = (XLogRecHdr) scratch; + + scratch += XLogRecordMaxHdrSize; hdr_rdt.next = NULL; rdt_datas_last = &hdr_rdt; @@ -779,7 +784,7 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP } } - total_len += bimg.length; + payload_len += bimg.length; } if (needs_data) @@ -797,7 +802,7 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP bkpb.fork_flags |= BKPBLOCK_HAS_DATA; data_length = (uint16) regbuf->rdata_len; data_sizeclass = XLogLengthToSizeClass(data_length, XLS_UINT16); - total_len += regbuf->rdata_len; + payload_len += regbuf->rdata_len; rdt_datas_last->next = regbuf->rdata_head; rdt_datas_last = regbuf->rdata_tail; @@ -851,16 +856,33 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP } /* followed by toplevel XID, if not already included in previous record */ - if (curinsert_flags & XLOG_INCLUDE_XID && IsSubxactTopXidLogPending()) + if (curinsert_flags & XLOG_INCLUDE_XID) { - TransactionId xid = GetTopTransactionIdIfAny(); + xid = GetCurrentTransactionIdIfAny(); + + if (IsSubxactTopXidLogPending()) + { + TransactionId txid = GetTopTransactionIdIfAny(); + + xlr_flags |= XLR_HAS_XID; + Assert(TransactionIdIsValid(xid)); - /* Set the flag that the top xid is included in the WAL */ - *topxid_included = true; + /* Set the flag that the top xid is included in the WAL */ + *topxid_included = true; - *(scratch++) = (char) XLR_BLOCK_ID_TOPLEVEL_XID; - memcpy(scratch, &xid, sizeof(TransactionId)); - scratch += sizeof(TransactionId); + *(scratch++) = (char) XLR_BLOCK_ID_TOPLEVEL_XID; + memcpy(scratch, &txid, sizeof(TransactionId)); + scratch += sizeof(TransactionId); + } + else if (TransactionIdIsValid(xid)) + { + xlr_flags |= XLR_HAS_XID; + } + } + else + { + xid = InvalidTransactionId; + Assert((xlr_flags & XLR_HAS_XID) == 0); } /* followed by main data, if any */ @@ -889,12 +911,64 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP } rdt_datas_last->next = mainrdata_head; rdt_datas_last = mainrdata_last; - total_len += mainrdata_len; + payload_len += mainrdata_len; } rdt_datas_last->next = NULL; - hdr_rdt.len = (scratch - hdr_scratch); - total_len += hdr_rdt.len; + /* Add the block headers section's length to the payload */ + payload_len += scratch - (hdr_scratch + XLogRecordMaxHdrSize); + + /* + * Fill in the fields in the record header. Prev-link is filled in later, + * once we know where in the WAL the record will be inserted. The CRC does + * not include the record header yet. + */ + payload_sizeclass = XLogLengthToSizeClass(payload_len, XLS_UINT32); + + xlr_flags |= payload_sizeclass; + + if (rmgr_info != 0) + xlr_flags |= XLR_HAS_RMGRINFO; + + /* Set up the xlog header. and xl_rmgr */ + rechdr->xl_info = xlr_flags; + rechdr->xl_rmid = rmid; + + rec_hdr_len = 0; + + /* next, xl_payload_len */ + rec_hdr_len += XLogWriteLength(payload_len, payload_sizeclass, + XLS_UINT32, + &rechdr->xl_hdrdata[rec_hdr_len]); + + if (xlr_flags & XLR_HAS_RMGRINFO) + rechdr->xl_hdrdata[rec_hdr_len++] = (char) rmgr_info; + + if (xlr_flags & XLR_HAS_XID) + { + Assert(curinsert_flags & XLOG_INCLUDE_XID); + Assert(TransactionIdIsValid(xid)); + + memcpy(&rechdr->xl_hdrdata[rec_hdr_len], &xid, sizeof(TransactionId)); + rec_hdr_len += sizeof(TransactionId); + } + + /* reserve space for XLogRecPtr and checksum */ + rec_hdr_len += sizeof(XLogRecPtr); + rec_hdr_len += sizeof(pg_crc32c); + /* Add static header length */ + rec_hdr_len += offsetof(XLogRecHdrData, xl_hdrdata); + + Assert(rec_hdr_len == XLogRecordHdrLen(rechdr->xl_info)); + + /* move the record to be placed the rest of the payload */ + memmove(hdr_scratch + XLogRecordMaxHdrSize - rec_hdr_len, + hdr_scratch, rec_hdr_len); + + rechdr = (XLogRecHdr) (hdr_scratch + XLogRecordMaxHdrSize - rec_hdr_len); + + hdr_rdt.data = (char *) rechdr; + hdr_rdt.len = (scratch - hdr_rdt.data); /* * Calculate CRC of the data @@ -905,10 +979,14 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP * header. */ INIT_CRC32C(rdata_crc); - COMP_CRC32C(rdata_crc, hdr_scratch + SizeOfXLogRecord, hdr_rdt.len - SizeOfXLogRecord); + COMP_CRC32C(rdata_crc, hdr_scratch + XLogRecordMaxHdrSize, + hdr_rdt.len - rec_hdr_len); for (rdt = hdr_rdt.next; rdt != NULL; rdt = rdt->next) COMP_CRC32C(rdata_crc, rdt->data, rdt->len); + memcpy(hdr_rdt.data + rec_hdr_len - sizeof(pg_crc32c), + &rdata_crc, sizeof(pg_crc32c)); + /* * Ensure that the XLogRecord is not too large. * @@ -917,28 +995,11 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP * not emit records larger than the sizes advertised to be supported. This * cap is based on DecodeXLogRecordRequiredSpace(). */ - if (total_len >= XLogRecordMaxSize) + if (payload_len + rec_hdr_len >= XLogRecordMaxSize) ereport(ERROR, (errmsg_internal("oversized WAL record"), errdetail_internal("WAL record would be %llu bytes (of maximum %u bytes); rmid %u flags %u.", - (unsigned long long) total_len, XLogRecordMaxSize, rmid, info))); - - /* - * Fill in the fields in the record header. Prev-link is filled in later, - * once we know where in the WAL the record will be inserted. The CRC does - * not include the record header yet. - */ - if (curinsert_flags & XLOG_INCLUDE_XID) - rechdr->xl_xid = GetCurrentTransactionIdIfAny(); - else - rechdr->xl_xid = InvalidTransactionId; - - rechdr->xl_tot_len = (uint32) total_len; - rechdr->xl_info = info; - rechdr->xl_rmid = rmid; - rechdr->xl_rmgrinfo = rmgr_info; - rechdr->xl_prev = InvalidXLogRecPtr; - rechdr->xl_crc = rdata_crc; + (unsigned long long) payload_len, XLogRecordMaxSize, rmid, info))); return &hdr_rdt; } diff --git a/src/backend/access/transam/xlogreader.c b/src/backend/access/transam/xlogreader.c index 733de8a8dc..181dcdac9b 100644 --- a/src/backend/access/transam/xlogreader.c +++ b/src/backend/access/transam/xlogreader.c @@ -49,8 +49,9 @@ static int ReadPageInternal(XLogReaderState *state, XLogRecPtr pageptr, static void XLogReaderInvalReadState(XLogReaderState *state); static XLogPageReadResult XLogDecodeNextRecord(XLogReaderState *state, bool nonblocking); static bool ValidXLogRecordHeader(XLogReaderState *state, XLogRecPtr RecPtr, - XLogRecPtr PrevRecPtr, XLogRecord *record, bool randAccess); -static bool ValidXLogRecord(XLogReaderState *state, XLogRecord *record, + XLogRecPtr PrevRecPtr, XLogRecHdr rechdr, + bool randAccess); +static bool ValidXLogRecord(XLogReaderState *state, XLogRecHdr record, XLogRecPtr recptr); static void ResetDecoder(XLogReaderState *state); static void WALOpenSegmentInit(WALOpenSegment *seg, WALSegmentContext *segcxt, @@ -535,7 +536,7 @@ static XLogPageReadResult XLogDecodeNextRecord(XLogReaderState *state, bool nonblocking) { XLogRecPtr RecPtr; - XLogRecord *record; + XLogRecHdr record; XLogRecPtr targetPagePtr; bool randAccess; uint32 len, @@ -602,7 +603,7 @@ restart: * fits on the same page. */ readOff = ReadPageInternal(state, targetPagePtr, - Min(targetRecOff + SizeOfXLogRecord, XLOG_BLCKSZ)); + Min(targetRecOff + XLogRecordMaxHdrSize, XLOG_BLCKSZ)); if (readOff == XLREAD_WOULDBLOCK) return XLREAD_WOULDBLOCK; else if (readOff < 0) @@ -644,13 +645,13 @@ restart: * Read the record length. * * NB: Even though we use an XLogRecord pointer here, the whole record - * header might not fit on this page. xl_tot_len is the first field of the - * struct, so it must be on this page (the records are MAXALIGNed), but we - * cannot access any other fields until we've verified that we got the - * whole header. + * header might not fit on this page. xl_payload_len is the third field of + * the data and starts at the third byte, so it must be on this page (the + * records are aligned to 8 bytes), but we cannot access most other fields + * until we've verified that we got the whole header. */ - record = (XLogRecord *) (state->readBuf + RecPtr % XLOG_BLCKSZ); - total_len = record->xl_tot_len; + record = (XLogRecHdr) (state->readBuf + (RecPtr % XLOG_BLCKSZ)); + total_len = XLogRecordTotalLength(record); /* * If the whole record header is on this page, validate it immediately. @@ -660,7 +661,7 @@ restart: * record" code path below; otherwise we might fail to apply * ValidXLogRecordHeader at all. */ - if (targetRecOff <= XLOG_BLCKSZ - SizeOfXLogRecord) + if (targetRecOff <= XLOG_BLCKSZ - XLogRecordHdrLen(record->xl_info)) { if (!ValidXLogRecordHeader(state, RecPtr, state->DecodeRecPtr, record, randAccess)) @@ -670,12 +671,25 @@ restart: else { /* XXX: more validation should be done here */ - if (total_len < SizeOfXLogRecord) + + /* + * RM_INVALID_ID can be an indication of end-of-WAL, or otherwise is + * corruption + */ + if (record->xl_rmid == RM_INVALID_ID) + { + report_invalid_record(state, + "invalid resource manager id %u at %X/%X", + (uint32) record->xl_rmid, + LSN_FORMAT_ARGS(RecPtr)); + goto err; + } + if (total_len < XLogRecordMinHdrSize) { report_invalid_record(state, "invalid record length at %X/%X: expected at least %u, got %u", LSN_FORMAT_ARGS(RecPtr), - (uint32) SizeOfXLogRecord, total_len); + (uint32) XLogRecordMinHdrSize, total_len); goto err; } gotheader = false; @@ -818,7 +832,7 @@ restart: /* If we just reassembled the record header, validate it. */ if (!gotheader) { - record = (XLogRecord *) state->readRecordBuf; + record = (XLogRecHdr) state->readRecordBuf; if (!ValidXLogRecordHeader(state, RecPtr, state->DecodeRecPtr, record, randAccess)) goto err; @@ -828,14 +842,14 @@ restart: Assert(gotheader); - record = (XLogRecord *) state->readRecordBuf; + record = (XLogRecHdr) state->readRecordBuf; if (!ValidXLogRecord(state, record, RecPtr)) goto err; pageHeaderSize = XLogPageHeaderSize((XLogPageHeader) state->readBuf); state->DecodeRecPtr = RecPtr; state->NextRecPtr = targetPagePtr + pageHeaderSize - + MAXALIGN(pageHeader->xlp_rem_len); + + XLP_ALIGN(pageHeader->xlp_rem_len); } else { @@ -851,7 +865,7 @@ restart: if (!ValidXLogRecord(state, record, RecPtr)) goto err; - state->NextRecPtr = RecPtr + MAXALIGN(total_len); + state->NextRecPtr = RecPtr + XLP_ALIGN(total_len); state->DecodeRecPtr = RecPtr; } @@ -860,7 +874,7 @@ restart: * Special processing if it's an XLOG SWITCH record */ if (record->xl_rmid == RM_XLOG_ID && - record->xl_rmgrinfo == XLOG_SWITCH) + XLRHdrGetRmgrInfo(record) == XLOG_SWITCH) { /* Pretend it extends to end of segment */ state->NextRecPtr += state->segcxt.ws_segsize - 1; @@ -879,7 +893,7 @@ restart: if (!decoded->oversized) { /* The new decode buffer head must be MAXALIGNed. */ - Assert(decoded->size == MAXALIGN(decoded->size)); + Assert(decoded->size == XLP_ALIGN(decoded->size)); if ((char *) decoded == state->decode_buffer) state->decode_buffer_tail = state->decode_buffer + decoded->size; else @@ -1114,22 +1128,25 @@ XLogReaderInvalReadState(XLogReaderState *state) */ static bool ValidXLogRecordHeader(XLogReaderState *state, XLogRecPtr RecPtr, - XLogRecPtr PrevRecPtr, XLogRecord *record, + XLogRecPtr PrevRecPtr, XLogRecHdr rechdr, bool randAccess) { - if (record->xl_tot_len < SizeOfXLogRecord) + XLogRecord record = {0}; + XLogReadRecHdrInto(rechdr, XLogRecordHdrLen(rechdr->xl_info), &record); + + if (XLogRecordTotalLength(rechdr) < XLogRecordMinHdrSize) { report_invalid_record(state, "invalid record length at %X/%X: expected at least %u, got %u", LSN_FORMAT_ARGS(RecPtr), - (uint32) SizeOfXLogRecord, record->xl_tot_len); + (uint32) XLogRecordMinHdrSize, record.xl_payload_len); return false; } - if (!RmgrIdIsValid(record->xl_rmid)) + if (!RmgrIdIsValid(rechdr->xl_rmid)) { report_invalid_record(state, "invalid resource manager ID %u at %X/%X", - record->xl_rmid, LSN_FORMAT_ARGS(RecPtr)); + rechdr->xl_rmid, LSN_FORMAT_ARGS(RecPtr)); return false; } if (randAccess) @@ -1138,12 +1155,14 @@ ValidXLogRecordHeader(XLogReaderState *state, XLogRecPtr RecPtr, * We can't exactly verify the prev-link, but surely it should be less * than the record's own address. */ - if (!(record->xl_prev < RecPtr)) + if (!(record.xl_prev < RecPtr)) { report_invalid_record(state, - "record with incorrect prev-link %X/%X at %X/%X", - LSN_FORMAT_ARGS(record->xl_prev), - LSN_FORMAT_ARGS(RecPtr)); + "record with incorrect prev-link %X/%X at %X/%X, info %u, rmgr %u", + LSN_FORMAT_ARGS(record.xl_prev), + LSN_FORMAT_ARGS(RecPtr), + (uint32) record.xl_info, + (uint32) record.xl_rmid); return false; } } @@ -1154,12 +1173,13 @@ ValidXLogRecordHeader(XLogReaderState *state, XLogRecPtr RecPtr, * check guards against torn WAL pages where a stale but valid-looking * WAL record starts on a sector boundary. */ - if (record->xl_prev != PrevRecPtr) + if (record.xl_prev != PrevRecPtr) { report_invalid_record(state, - "record with incorrect prev-link %X/%X at %X/%X", - LSN_FORMAT_ARGS(record->xl_prev), - LSN_FORMAT_ARGS(RecPtr)); + "record with incorrect prev-link %X/%X at %X/%X, expected %X/%X", + LSN_FORMAT_ARGS(record.xl_prev), + LSN_FORMAT_ARGS(RecPtr), + LSN_FORMAT_ARGS(PrevRecPtr)); return false; } } @@ -1179,19 +1199,26 @@ ValidXLogRecordHeader(XLogReaderState *state, XLogRecPtr RecPtr, * SizeOfXLogRecord. */ static bool -ValidXLogRecord(XLogReaderState *state, XLogRecord *record, XLogRecPtr recptr) +ValidXLogRecord(XLogReaderState *state, XLogRecHdr record, XLogRecPtr recptr) { - pg_crc32c crc; + pg_crc32c crc, + hdr_crc; + Size hdr_len = XLogRecordHdrLen(record->xl_info); + Size rec_length = XLogRecordTotalLength(record); /* Calculate the CRC */ INIT_CRC32C(crc); - COMP_CRC32C(crc, ((char *) record) + SizeOfXLogRecord, record->xl_tot_len - SizeOfXLogRecord); + COMP_CRC32C(crc, ((char *) record) + hdr_len, rec_length - hdr_len); /* include the record header last */ - COMP_CRC32C(crc, (char *) record, offsetof(XLogRecord, xl_crc)); + COMP_CRC32C(crc, (char *) record, hdr_len - sizeof(pg_crc32c)); FIN_CRC32C(crc); - if (!EQ_CRC32C(record->xl_crc, crc)) + memcpy(&hdr_crc, ((char *) record) + hdr_len - sizeof(pg_crc32c), + sizeof(pg_crc32c)); + + if (!EQ_CRC32C(hdr_crc, crc)) { + Assert(false); report_invalid_record(state, "incorrect resource manager data checksum in record at %X/%X", LSN_FORMAT_ARGS(recptr)); @@ -1431,7 +1458,7 @@ XLogFindNextRecord(XLogReaderState *state, XLogRecPtr RecPtr) * * Note that record headers are MAXALIGN'ed */ - if (MAXALIGN(header->xlp_rem_len) >= (XLOG_BLCKSZ - pageHeaderSize)) + if (XLP_ALIGN(header->xlp_rem_len) >= (XLOG_BLCKSZ - pageHeaderSize)) tmpRecPtr = targetPagePtr + XLOG_BLCKSZ; else { @@ -1440,7 +1467,7 @@ XLogFindNextRecord(XLogReaderState *state, XLogRecPtr RecPtr) * tmpRecPtr to point to the first valid record */ tmpRecPtr = targetPagePtr + pageHeaderSize - + MAXALIGN(header->xlp_rem_len); + + XLP_ALIGN(header->xlp_rem_len); break; } } @@ -1651,7 +1678,7 @@ DecodeXLogRecordRequiredSpace(size_t xl_tot_len) bool DecodeXLogRecord(XLogReaderState *state, DecodedXLogRecord *decoded, - XLogRecord *record, + XLogRecHdr record, XLogRecPtr lsn, char **errormsg) { @@ -1674,7 +1701,7 @@ DecodeXLogRecord(XLogReaderState *state, RelFileLocator *rlocator = NULL; uint8 block_id; - decoded->header = *record; + XLogReadRecHdrInto(record, XLogRecordHdrLen(record->xl_info), &decoded->header); decoded->lsn = lsn; decoded->next = NULL; decoded->record_origin = InvalidRepOriginId; @@ -1683,8 +1710,8 @@ DecodeXLogRecord(XLogReaderState *state, decoded->main_data_len = 0; decoded->max_block_id = -1; ptr = (char *) record; - ptr += SizeOfXLogRecord; - remaining = record->xl_tot_len - SizeOfXLogRecord; + ptr += XLogRecordHdrLen(record->xl_info); + remaining = decoded->header.xl_payload_len; /* Decode the headers */ datatotal = 0; @@ -1959,8 +1986,8 @@ DecodeXLogRecord(XLogReaderState *state, } /* Report the actual size we used. */ - decoded->size = MAXALIGN(out - (char *) decoded); - Assert(DecodeXLogRecordRequiredSpace(record->xl_tot_len) >= + decoded->size = XLP_ALIGN(out - (char *) decoded); + Assert(DecodeXLogRecordRequiredSpace(decoded->header.xl_payload_len) >= decoded->size); return true; @@ -1989,6 +2016,7 @@ XLogRecGetBlockTag(XLogReaderState *record, uint8 block_id, if (!XLogRecGetBlockTagExtended(record, block_id, rlocator, forknum, blknum, NULL)) { + Assert(false); #ifndef FRONTEND elog(ERROR, "could not locate backup block with ID %d in WAL record", block_id); diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index 56fa2f74a2..0362ba747f 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -4000,10 +4000,10 @@ ReadCheckpointRecord(XLogPrefetcher *xlogprefetcher, XLogRecPtr RecPtr, (errmsg("invalid xl_rmgrinfo in checkpoint record"))); return NULL; } - if (record->xl_tot_len != SizeOfXLogRecord + SizeOfXLogRecordDataHeaderShort + sizeof(CheckPoint)) + if (record->xl_payload_len != SizeOfXLogRecordDataHeaderShort + sizeof(CheckPoint)) { ereport(LOG, - (errmsg("invalid length of checkpoint record"))); + (errmsg("invalid payload length of checkpoint record"))); return NULL; } return record; diff --git a/src/bin/pg_resetwal/pg_resetwal.c b/src/bin/pg_resetwal/pg_resetwal.c index 80cb9a5bc4..4e171eed1f 100644 --- a/src/bin/pg_resetwal/pg_resetwal.c +++ b/src/bin/pg_resetwal/pg_resetwal.c @@ -1040,12 +1040,15 @@ WriteEmptyXLOG(void) PGAlignedXLogBlock buffer; XLogPageHeader page; XLogLongPageHeader longpage; - XLogRecord *record; pg_crc32c crc; char path[MAXPGPATH]; int fd; int nbytes; char *recptr; + char *baserecptr; + const int rec_hdr_len = offsetof(XLogRecHdrData, xl_hdrdata) + + sizeof(uint8) + sizeof(XLogRecPtr) + sizeof(pg_crc32c); + const int rec_payload_len = sizeof(uint8) * 2 + sizeof(CheckPoint); memset(buffer.data, 0, XLOG_BLCKSZ); @@ -1061,26 +1064,33 @@ WriteEmptyXLOG(void) longpage->xlp_xlog_blcksz = XLOG_BLCKSZ; /* Insert the initial checkpoint record */ - recptr = (char *) page + SizeOfXLogLongPHD; - record = (XLogRecord *) recptr; - record->xl_prev = 0; - record->xl_xid = InvalidTransactionId; - record->xl_tot_len = SizeOfXLogRecord + SizeOfXLogRecordDataHeaderShort + sizeof(CheckPoint); - record->xl_info = 0; - record->xl_rmgrinfo = XLOG_CHECKPOINT_SHUTDOWN; - record->xl_rmid = RM_XLOG_ID; - - recptr += SizeOfXLogRecord; + baserecptr = recptr = (char *) page + SizeOfXLogLongPHD; + *(recptr++) = (char) XLS_UINT8; + *(recptr++) = (char) RM_XLOG_ID; + recptr += XLogWriteLength(SizeOfXLogRecordDataHeaderShort + sizeof(CheckPoint), + XLS_UINT8, XLS_UINT8, recptr); + + /* include prevptr */ + { + XLogRecPtr prevptr = 0; + memcpy(recptr, &prevptr, sizeof(XLogRecPtr)); + recptr += sizeof(XLogRecPtr); + } + /* reserve location of crc */ + recptr += sizeof(pg_crc32c); + *(recptr++) = (char) XLR_BLOCK_ID_DATA_SHORT; *(recptr++) = sizeof(CheckPoint); memcpy(recptr, &ControlFile.checkPointCopy, sizeof(CheckPoint)); INIT_CRC32C(crc); - COMP_CRC32C(crc, ((char *) record) + SizeOfXLogRecord, record->xl_tot_len - SizeOfXLogRecord); - COMP_CRC32C(crc, (char *) record, offsetof(XLogRecord, xl_crc)); + COMP_CRC32C(crc, baserecptr + rec_hdr_len, rec_payload_len); + COMP_CRC32C(crc, baserecptr, rec_hdr_len - sizeof(pg_crc32c)); FIN_CRC32C(crc); - record->xl_crc = crc; + + memcpy(baserecptr + rec_hdr_len - sizeof(pg_crc32c), + &crc, sizeof(pg_crc32c)); /* Write the first page */ XLogFilePath(path, ControlFile.checkPointCopy.ThisTimeLineID, diff --git a/src/include/access/rmgr.h b/src/include/access/rmgr.h index 3b6a497e1b..7038ba0833 100644 --- a/src/include/access/rmgr.h +++ b/src/include/access/rmgr.h @@ -41,7 +41,7 @@ typedef enum RmgrIds static inline bool RmgrIdIsBuiltin(int rmid) { - return rmid <= RM_MAX_BUILTIN_ID; + return rmid <= RM_MAX_BUILTIN_ID && rmid != RM_INVALID_ID; } static inline bool diff --git a/src/include/access/rmgrlist.h b/src/include/access/rmgrlist.h index 463bcb67c5..b53d731f56 100644 --- a/src/include/access/rmgrlist.h +++ b/src/include/access/rmgrlist.h @@ -25,6 +25,7 @@ */ /* symbol name, textual name, redo, desc, identify, startup, cleanup, mask, decode */ +PG_RMGR(RM_INVALID_ID, "invalid", NULL, NULL, NULL, NULL, NULL, NULL, NULL) PG_RMGR(RM_XLOG_ID, "XLOG", xlog_redo, xlog_desc, xlog_identify, NULL, NULL, NULL, xlog_decode) PG_RMGR(RM_XACT_ID, "Transaction", xact_redo, xact_desc, xact_identify, NULL, NULL, NULL, xact_decode) PG_RMGR(RM_SMGR_ID, "Storage", smgr_redo, smgr_desc, smgr_identify, NULL, NULL, NULL, NULL) diff --git a/src/include/access/xlog_internal.h b/src/include/access/xlog_internal.h index 876e2790f9..ea32a9ba1b 100644 --- a/src/include/access/xlog_internal.h +++ b/src/include/access/xlog_internal.h @@ -26,12 +26,20 @@ #include "pgtime.h" #include "storage/block.h" #include "storage/relfilelocator.h" +#include "transam.h" +/* + * WAL records (and other XLogPage page-level content) are aligned to 8 bytes. + * + * Note that the contents of the records is not aligned (!) + */ +#define ALIGNOF_XLP_CONTENT 8 +#define XLP_ALIGN(LEN) TYPEALIGN64(ALIGNOF_XLP_CONTENT, LEN) /* * Each page of XLOG file has a header like this: */ -#define XLOG_PAGE_MAGIC 0xD113 /* can be used as WAL version indicator */ +#define XLOG_PAGE_MAGIC 0xD114 /* can be used as WAL version indicator */ typedef struct XLogPageHeaderData { @@ -49,7 +57,7 @@ typedef struct XLogPageHeaderData uint32 xlp_rem_len; /* total len of remaining data for record */ } XLogPageHeaderData; -#define SizeOfXLogShortPHD MAXALIGN(sizeof(XLogPageHeaderData)) +#define SizeOfXLogShortPHD XLP_ALIGN(sizeof(XLogPageHeaderData)) typedef XLogPageHeaderData *XLogPageHeader; @@ -66,7 +74,7 @@ typedef struct XLogLongPageHeaderData uint32 xlp_xlog_blcksz; /* just as a cross-check */ } XLogLongPageHeaderData; -#define SizeOfXLogLongPHD MAXALIGN(sizeof(XLogLongPageHeaderData)) +#define SizeOfXLogLongPHD XLP_ALIGN(sizeof(XLogLongPageHeaderData)) typedef XLogLongPageHeaderData *XLogLongPageHeader; @@ -381,7 +389,7 @@ static inline Size XLogWriteLength(uint32 length, XLogSizeClass sizeClass, case caseSizeClass: \ if ((caseSizeClass) <= maxSizeClass) \ { \ - field_type typedLength = length; \ + field_type typedLength = (field_type) length; \ memcpy(output, &typedLength, sizeof(field_type)); \ written = sizeof(field_type); \ } \ @@ -394,6 +402,9 @@ static inline Size XLogWriteLength(uint32 length, XLogSizeClass sizeClass, WRITE_OP(XLS_UINT8, uint8); WRITE_OP(XLS_UINT16, uint16); WRITE_OP(XLS_UINT32, uint32); + default: + Assert(false); + pg_unreachable(); } #undef WRITE_OP @@ -436,12 +447,95 @@ static inline Size XLogReadLength(uint32 *length, XLogSizeClass sizeClass, READ_OP(XLS_UINT8, uint8); READ_OP(XLS_UINT16, uint16); READ_OP(XLS_UINT32, uint32); + default: + Assert(false); + pg_unreachable(); } #undef READ_OP return readSize; } + +inline static uint8 XLRHdrGetRmgrInfo(XLogRecHdr record) +{ + XLogSizeClass recSizeClass; + int offset; + + if (!(record->xl_info & XLR_HAS_RMGRINFO)) + return 0; + + recSizeClass = XLR_SIZECLASS(record->xl_info); + /* xl_rmgrinfo is located immediately behind the xl_payload_len field */ + offset = XLogSizeClassToByteLength(recSizeClass); + + return (uint8) record->xl_hdrdata[offset]; +} + + +/* Works on any partial record */ +inline static Size XLogRecordTotalLength(XLogRecHdr record) +{ + uint8 xl_info = record->xl_info; + XLogSizeClass sizeClass; + uint32 length = 0; + sizeClass = XLR_SIZECLASS(xl_info); + + XLogReadLength(&length, sizeClass, XLS_UINT32, + &record->xl_hdrdata[0], 6); + + return (Size) length + XLogRecordHdrLen(xl_info); +} + +inline static void XLogReadRecHdrInto(XLogRecHdr recdata, Size length, + XLogRecord *record) +{ + Size offset = 0; + Size hdr_size PG_USED_FOR_ASSERTS_ONLY = 0; + XLogSizeClass sizeClass; + + Assert(length >= XLogRecordMinHdrSize); + + record->xl_info = recdata->xl_info; + + hdr_size = XLogRecordHdrLen(record->xl_info); + Assert(length >= hdr_size); + + record->xl_rmid = recdata->xl_rmid; + + sizeClass = XLR_SIZECLASS(record->xl_info); + offset += XLogReadLength(&record->xl_payload_len, sizeClass, + XLS_UINT32, &recdata->xl_hdrdata[offset], length - offset); + + if (record->xl_info & XLR_HAS_RMGRINFO) + { + record->xl_rmgrinfo = recdata->xl_hdrdata[offset]; + offset += sizeof(uint8); + } + else + { + record->xl_rmgrinfo = 0; + } + + if (record->xl_info & XLR_HAS_XID) + { + memcpy(&record->xl_xid, &recdata->xl_hdrdata[offset], sizeof(TransactionId)); + offset += sizeof(TransactionId); + } + else + { + record->xl_xid = InvalidTransactionId; + } + + memcpy(&record->xl_prev, &recdata->xl_hdrdata[offset], sizeof(XLogRecPtr)); + offset += sizeof(XLogRecPtr); + + memcpy(&record->xl_crc, &recdata->xl_hdrdata[offset], sizeof(pg_crc32c)); + offset += sizeof(pg_crc32c); + + Assert(hdr_size - 2 == offset); +} + /* * Method table for resource managers. * diff --git a/src/include/access/xlogreader.h b/src/include/access/xlogreader.h index a1d0216404..bb64f89dba 100644 --- a/src/include/access/xlogreader.h +++ b/src/include/access/xlogreader.h @@ -397,7 +397,7 @@ extern bool WALRead(XLogReaderState *state, extern size_t DecodeXLogRecordRequiredSpace(size_t xl_tot_len); extern bool DecodeXLogRecord(XLogReaderState *state, DecodedXLogRecord *decoded, - XLogRecord *record, + XLogRecHdr record, XLogRecPtr lsn, char **errormsg); @@ -405,7 +405,7 @@ extern bool DecodeXLogRecord(XLogReaderState *state, * Macros that provide access to parts of the record most recently returned by * XLogReadRecord() or XLogNextRecord(). */ -#define XLogRecGetTotalLen(decoder) ((decoder)->record->header.xl_tot_len) +#define XLogRecGetTotalLen(decoder) ((decoder)->record->header.xl_payload_len) #define XLogRecGetPrev(decoder) ((decoder)->record->header.xl_prev) #define XLogRecGetInfo(decoder) ((decoder)->record->header.xl_info) #define XLogRecGetRmgrInfo(decoder) ((decoder)->record->header.xl_rmgrinfo) diff --git a/src/include/access/xlogrecord.h b/src/include/access/xlogrecord.h index 42b06f163e..af615e2866 100644 --- a/src/include/access/xlogrecord.h +++ b/src/include/access/xlogrecord.h @@ -38,9 +38,38 @@ typedef enum XLogSizeClass { XLS_UINT32 = 3 /* length <= UINT32_MAX; stored in uint32 (4B) */ } XLogSizeClass; +static inline int XLogSizeClassToByteLength(XLogSizeClass sz) +{ + switch (sz) { + case XLS_EMPTY: + return 0; + case XLS_UINT8: + return sizeof(uint8); + case XLS_UINT16: + return sizeof(uint16); + case XLS_UINT32: + return sizeof(uint32); + default: + pg_unreachable(); + } +} + +typedef struct XLogRecord +{ + uint8 xl_info; /* flag bits, see below */ + RmgrId xl_rmid; /* resource manager for this record */ + uint32 xl_payload_len; /* total len of entire record */ + uint8 xl_rmgrinfo; /* rmgr flag bits, see below */ + TransactionId xl_xid; /* xact id */ + XLogRecPtr xl_prev; /* ptr to previous record in log */ + pg_crc32c xl_crc; /* CRC for this record */ +} XLogRecord; + +#define SizeOfXLogRecord (offsetof(XLogRecord, xl_crc) + sizeof(pg_crc32c)) + /* * The overall layout of an XLOG record is: - * Fixed-size header (XLogRecord struct) + * Variable-size header (containing the data of the XLogRecord struct) * XLogRecordBlockHeader struct * XLogRecordBlockHeader struct * ... @@ -50,6 +79,15 @@ typedef enum XLogSizeClass { * ... * main data * + * Different xlog record headers need to store different header fields, so + * depending on flags in the xl_info field the layout may change. + * + * Note that for records with a payload larger than 0xFFFFFFF, the size field + * will be 4 bytes long, which can be larger than the minimum MAXALIGN quantum. + * To still be able to read the record correctly, we always align WAL to 8 + * bytes, so that the length bytes are always present in the first aligned + * quantum of data. + * * There can be zero or more XLogRecordBlockHeaders, and 0 or more bytes of * rmgr-specific data not associated with a block. XLogRecord structs * always start on MAXALIGN boundaries in the WAL files, but the rest of @@ -59,22 +97,25 @@ typedef enum XLogSizeClass { * XLogRecordDataHeaderLong structs all begin with a single 'id' byte. It's * used to distinguish between block references, and the main data structs. */ -typedef struct XLogRecord +typedef struct XLogRecHdrData { - uint32 xl_tot_len; /* total len of entire record */ - TransactionId xl_xid; /* xact id */ - XLogRecPtr xl_prev; /* ptr to previous record in log */ uint8 xl_info; /* flag bits, see below */ RmgrId xl_rmid; /* resource manager for this record */ - uint8 xl_rmgrinfo; /* rmgr flag bits, see below */ - /* 1 byte of padding here, initialize to zero */ - pg_crc32c xl_crc; /* CRC for this record */ - - /* XLogRecordBlockHeaders and XLogRecordDataHeader follow, no padding */ + char xl_hdrdata[FLEXIBLE_ARRAY_MEMBER]; /* variable length record data */ -} XLogRecord; + /* + * Payload is as follows, in order: + * XLogRecord data encoded + * xl_payload_len; 0, 1, 2 or 4 bytes + * xl_rmgrinfo; 0 or 1 byte + * xl_xid; 0 or 4 bytes + * xl_prev; 8 bytes + * xl_prev follows; 8 bytes + * record payload, if present. + */ +} XLogRecHdrData; -#define SizeOfXLogRecord (offsetof(XLogRecord, xl_crc) + sizeof(pg_crc32c)) +typedef XLogRecHdrData *XLogRecHdr; /* * XLogReader needs to allocate all the data of a WAL record in a single @@ -87,13 +128,28 @@ typedef struct XLogRecord */ #define XLogRecordMaxSize (1020 * 1024 * 1024) +#define XLR_SIZECLASS_MASK 0x03 +#define XLR_SIZECLASS(xl_info) ((XLogSizeClass) ((xl_info) & XLR_SIZECLASS_MASK)) + +/* + * There are several rmgrs which don't use info bits, so we omit that + * byte whenever possible. + */ +#define XLR_HAS_RMGRINFO 0x04 + +/* + * If a WAL record uses the current transaction ID, that will be included + * in the record header. In all other cases we omit the XID to save bytes. + */ +#define XLR_HAS_XID 0x08 + /* * If a WAL record modifies any relation files, in ways not covered by the * usual block references, this flag is set. This is not used for anything * by PostgreSQL itself, but it allows external tools that read WAL and keep * track of modified blocks to recognize such special record types. */ -#define XLR_SPECIAL_REL_UPDATE 0x01 +#define XLR_SPECIAL_REL_UPDATE 0x10 /* * Enforces consistency checks of replayed WAL at recovery. If enabled, @@ -102,7 +158,48 @@ typedef struct XLogRecord * of XLogInsert can use this value if necessary, but if * wal_consistency_checking is enabled for a rmgr this is set unconditionally. */ -#define XLR_CHECK_CONSISTENCY 0x02 +#define XLR_CHECK_CONSISTENCY (0x20) + +#define XLogRecordMaxHdrSize ( \ + sizeof(uint8) /* xl_info */ + \ + sizeof(uint8) /* xl_rmid */ + \ + sizeof(uint32) /* xl_payload_len */ + \ + sizeof(uint8) /* xl_rmgrinfo */ + \ + sizeof(TransactionId) /* xl_xid */ + \ + sizeof(XLogRecPtr) /* xl_prev */ + \ + sizeof(pg_crc32c) /* xl_crc */ \ +) + +#define XLogRecordMinHdrSize ( \ + sizeof(uint8) /* xl_info */ + \ + sizeof(uint8) /* xl_rmid */ + \ + 0 /* xl_payload_len */ + \ + 0 /* xl_rmgrinfo */ + \ + 0 /* xl_xid */ + \ + sizeof(XLogRecPtr) /* xl_prev */ + \ + sizeof(pg_crc32c) /* xl_crc */ \ +) + +static inline Size XLogRecordHdrLen(uint8 info) +{ + Size size = 0; + /* xl_info */ + size += sizeof(uint8); + /* xl_rmid */ + size += sizeof(uint8); + /* xl_payload_len */ + size += XLogSizeClassToByteLength(XLR_SIZECLASS(info)); + /* xl_rmgrinfo */ + size += ((info) & XLR_HAS_RMGRINFO) ? sizeof(uint8) : 0; + /* xl_xid */ + size += ((info) & XLR_HAS_XID) ? sizeof(TransactionId) : 0; + /* xl_prev */ + size += sizeof(XLogRecPtr); + /* xl_crc */ + size += sizeof(pg_crc32c); + + return size; +} /* * Header info for block data appended to an XLOG record. -- 2.40.1 ^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: XLog size reductions: Reduced XLog record header size for PG17 @ 2023-07-12 12:50 Matthias van de Meent <[email protected]> parent: Matthias van de Meent <[email protected]> 0 siblings, 1 reply; 35+ messages in thread From: Matthias van de Meent @ 2023-07-12 12:50 UTC (permalink / raw) To: pgsql-hackers; +Cc: Heikki Linnakangas <[email protected]>; Robert Haas <[email protected]>; Dilip Kumar <[email protected]>; Andres Freund <[email protected]> On Mon, 3 Jul 2023 at 13:08, Matthias van de Meent <[email protected]> wrote: > > On Fri, 30 Jun 2023 at 17:36, Matthias van de Meent > <[email protected]> wrote: > > > > Hi, > > > > The attached v2 patchset contains some small fixes for the failing > > cfbot 32-bit tests - at least locally it does so. > > > > I'd overlooked one remaining use of MAXALIGN64() in xlog.c in the last > > patch of the set, which has now been updated to XLP_ALIGN as well. > > Additionally, XLP_ALIGN has been updated to use TYPEALIGN64 instead of > > TYPEALIGN so that we don't lose bits of the aligned value in 32-bit > > systems. > > Apparently there was some usage of MAXALIGN() in xlogreader that I'd > missed, and which only shows up in TAP tests. In v3 I've fixed that, > together with some improved early detection of invalid record headers. Another fix for CFBot - pg_waldump tests which were added in 96063e28 exposed an issue in my patchset related to RM_INVALID_ID. v4 splits former patch 0006 into two: now 0006 adds RM_INVALID and does the rmgr-related changes in the code, and 0007 does the WAL disk format overhaul. Kind regards, Matthias van de Meent Neon (https://neon.tech/) Attachments: [application/octet-stream] v4-0003-Rename-rmgr_identify-info-bits-argument-to-rmgrin.patch (21.6K, ../../CAEze2WjVNdfP9sbMW=6zbgfae-UF_XVCpgkxp7BcZUcFBYa7tQ@mail.gmail.com/2-v4-0003-Rename-rmgr_identify-info-bits-argument-to-rmgrin.patch) download | inline diff: From 088841ca507a582ef98a550c97e6932eeee347d9 Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Mon, 23 Jan 2023 17:31:18 +0100 Subject: [PATCH v4 3/7] Rename rmgr_identify 'info'-bits argument to rmgrinfo This indicates the provided bits are fully owned by the rmgr, and none of the xlog record's owned bits are shared with the rmgr anymore. --- src/backend/access/rmgrdesc/brindesc.c | 4 ++-- src/backend/access/rmgrdesc/clogdesc.c | 4 ++-- src/backend/access/rmgrdesc/dbasedesc.c | 4 ++-- src/backend/access/rmgrdesc/gindesc.c | 4 ++-- src/backend/access/rmgrdesc/gistdesc.c | 4 ++-- src/backend/access/rmgrdesc/hashdesc.c | 4 ++-- src/backend/access/rmgrdesc/heapdesc.c | 8 ++++---- src/backend/access/rmgrdesc/logicalmsgdesc.c | 4 ++-- src/backend/access/rmgrdesc/mxactdesc.c | 4 ++-- src/backend/access/rmgrdesc/nbtdesc.c | 4 ++-- src/backend/access/rmgrdesc/relmapdesc.c | 4 ++-- src/backend/access/rmgrdesc/seqdesc.c | 4 ++-- src/backend/access/rmgrdesc/smgrdesc.c | 4 ++-- src/backend/access/rmgrdesc/spgdesc.c | 4 ++-- src/backend/access/rmgrdesc/standbydesc.c | 4 ++-- src/backend/access/rmgrdesc/tblspcdesc.c | 4 ++-- src/backend/access/rmgrdesc/xlogdesc.c | 4 ++-- src/include/access/brin_xlog.h | 2 +- src/include/access/clog.h | 2 +- src/include/access/ginxlog.h | 2 +- src/include/access/gistxlog.h | 2 +- src/include/access/hash_xlog.h | 2 +- src/include/access/heapam_xlog.h | 4 ++-- src/include/access/multixact.h | 2 +- src/include/access/nbtxlog.h | 2 +- src/include/access/spgxlog.h | 2 +- src/include/access/xlog.h | 2 +- src/include/catalog/storage_xlog.h | 2 +- src/include/commands/dbcommands_xlog.h | 2 +- src/include/commands/sequence.h | 2 +- src/include/commands/tablespace.h | 2 +- src/include/replication/message.h | 2 +- src/include/storage/standbydefs.h | 2 +- src/include/utils/relmapper.h | 2 +- src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c | 4 ++-- 35 files changed, 56 insertions(+), 56 deletions(-) diff --git a/src/backend/access/rmgrdesc/brindesc.c b/src/backend/access/rmgrdesc/brindesc.c index 1d3e5ee03e..fa272f0720 100644 --- a/src/backend/access/rmgrdesc/brindesc.c +++ b/src/backend/access/rmgrdesc/brindesc.c @@ -71,11 +71,11 @@ brin_desc(StringInfo buf, XLogReaderState *record) } const char * -brin_identify(uint8 info) +brin_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_BRIN_CREATE_INDEX: id = "CREATE_INDEX"; diff --git a/src/backend/access/rmgrdesc/clogdesc.c b/src/backend/access/rmgrdesc/clogdesc.c index 915f9945d0..b368af595e 100644 --- a/src/backend/access/rmgrdesc/clogdesc.c +++ b/src/backend/access/rmgrdesc/clogdesc.c @@ -41,11 +41,11 @@ clog_desc(StringInfo buf, XLogReaderState *record) } const char * -clog_identify(uint8 info) +clog_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case CLOG_ZEROPAGE: id = "ZEROPAGE"; diff --git a/src/backend/access/rmgrdesc/dbasedesc.c b/src/backend/access/rmgrdesc/dbasedesc.c index a9529bde05..a405c0738a 100644 --- a/src/backend/access/rmgrdesc/dbasedesc.c +++ b/src/backend/access/rmgrdesc/dbasedesc.c @@ -54,11 +54,11 @@ dbase_desc(StringInfo buf, XLogReaderState *record) } const char * -dbase_identify(uint8 info) +dbase_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_DBASE_CREATE_FILE_COPY: id = "CREATE_FILE_COPY"; diff --git a/src/backend/access/rmgrdesc/gindesc.c b/src/backend/access/rmgrdesc/gindesc.c index 08fc7ba49b..e7ed7a6094 100644 --- a/src/backend/access/rmgrdesc/gindesc.c +++ b/src/backend/access/rmgrdesc/gindesc.c @@ -179,11 +179,11 @@ gin_desc(StringInfo buf, XLogReaderState *record) } const char * -gin_identify(uint8 info) +gin_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_GIN_CREATE_PTREE: id = "CREATE_PTREE"; diff --git a/src/backend/access/rmgrdesc/gistdesc.c b/src/backend/access/rmgrdesc/gistdesc.c index 6473069c1a..899e12bcc3 100644 --- a/src/backend/access/rmgrdesc/gistdesc.c +++ b/src/backend/access/rmgrdesc/gistdesc.c @@ -86,11 +86,11 @@ gist_desc(StringInfo buf, XLogReaderState *record) } const char * -gist_identify(uint8 info) +gist_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_GIST_PAGE_UPDATE: id = "PAGE_UPDATE"; diff --git a/src/backend/access/rmgrdesc/hashdesc.c b/src/backend/access/rmgrdesc/hashdesc.c index bcb2013048..451251caca 100644 --- a/src/backend/access/rmgrdesc/hashdesc.c +++ b/src/backend/access/rmgrdesc/hashdesc.c @@ -122,11 +122,11 @@ hash_desc(StringInfo buf, XLogReaderState *record) } const char * -hash_identify(uint8 info) +hash_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_HASH_INIT_META_PAGE: id = "INIT_META_PAGE"; diff --git a/src/backend/access/rmgrdesc/heapdesc.c b/src/backend/access/rmgrdesc/heapdesc.c index 15f943051a..5224869af1 100644 --- a/src/backend/access/rmgrdesc/heapdesc.c +++ b/src/backend/access/rmgrdesc/heapdesc.c @@ -302,11 +302,11 @@ heap2_desc(StringInfo buf, XLogReaderState *record) } const char * -heap_identify(uint8 info) +heap_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_HEAP_INSERT: id = "INSERT"; @@ -347,11 +347,11 @@ heap_identify(uint8 info) } const char * -heap2_identify(uint8 info) +heap2_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_HEAP2_PRUNE: id = "PRUNE"; diff --git a/src/backend/access/rmgrdesc/logicalmsgdesc.c b/src/backend/access/rmgrdesc/logicalmsgdesc.c index 2bb6cb06c3..7566630961 100644 --- a/src/backend/access/rmgrdesc/logicalmsgdesc.c +++ b/src/backend/access/rmgrdesc/logicalmsgdesc.c @@ -43,9 +43,9 @@ logicalmsg_desc(StringInfo buf, XLogReaderState *record) } const char * -logicalmsg_identify(uint8 info) +logicalmsg_identify(uint8 rmgrinfo) { - if ((info & ~XLR_INFO_MASK) == XLOG_LOGICAL_MESSAGE) + if (rmgrinfo == XLOG_LOGICAL_MESSAGE) return "MESSAGE"; return NULL; diff --git a/src/backend/access/rmgrdesc/mxactdesc.c b/src/backend/access/rmgrdesc/mxactdesc.c index cdcf8a4f45..7676778239 100644 --- a/src/backend/access/rmgrdesc/mxactdesc.c +++ b/src/backend/access/rmgrdesc/mxactdesc.c @@ -81,11 +81,11 @@ multixact_desc(StringInfo buf, XLogReaderState *record) } const char * -multixact_identify(uint8 info) +multixact_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_MULTIXACT_ZERO_OFF_PAGE: id = "ZERO_OFF_PAGE"; diff --git a/src/backend/access/rmgrdesc/nbtdesc.c b/src/backend/access/rmgrdesc/nbtdesc.c index ede8df72c0..ff9e28ca55 100644 --- a/src/backend/access/rmgrdesc/nbtdesc.c +++ b/src/backend/access/rmgrdesc/nbtdesc.c @@ -134,11 +134,11 @@ btree_desc(StringInfo buf, XLogReaderState *record) } const char * -btree_identify(uint8 info) +btree_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_BTREE_INSERT_LEAF: id = "INSERT_LEAF"; diff --git a/src/backend/access/rmgrdesc/relmapdesc.c b/src/backend/access/rmgrdesc/relmapdesc.c index f4f0e5ce8b..9c9f191700 100644 --- a/src/backend/access/rmgrdesc/relmapdesc.c +++ b/src/backend/access/rmgrdesc/relmapdesc.c @@ -32,11 +32,11 @@ relmap_desc(StringInfo buf, XLogReaderState *record) } const char * -relmap_identify(uint8 info) +relmap_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_RELMAP_UPDATE: id = "UPDATE"; diff --git a/src/backend/access/rmgrdesc/seqdesc.c b/src/backend/access/rmgrdesc/seqdesc.c index 6def7b653b..eed631b5d8 100644 --- a/src/backend/access/rmgrdesc/seqdesc.c +++ b/src/backend/access/rmgrdesc/seqdesc.c @@ -31,11 +31,11 @@ seq_desc(StringInfo buf, XLogReaderState *record) } const char * -seq_identify(uint8 info) +seq_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_SEQ_LOG: id = "LOG"; diff --git a/src/backend/access/rmgrdesc/smgrdesc.c b/src/backend/access/rmgrdesc/smgrdesc.c index 8604bee0c0..653caf58e4 100644 --- a/src/backend/access/rmgrdesc/smgrdesc.c +++ b/src/backend/access/rmgrdesc/smgrdesc.c @@ -43,11 +43,11 @@ smgr_desc(StringInfo buf, XLogReaderState *record) } const char * -smgr_identify(uint8 info) +smgr_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_SMGR_CREATE: id = "CREATE"; diff --git a/src/backend/access/rmgrdesc/spgdesc.c b/src/backend/access/rmgrdesc/spgdesc.c index 9e1e48240b..0d1af9a5b6 100644 --- a/src/backend/access/rmgrdesc/spgdesc.c +++ b/src/backend/access/rmgrdesc/spgdesc.c @@ -128,11 +128,11 @@ spg_desc(StringInfo buf, XLogReaderState *record) } const char * -spg_identify(uint8 info) +spg_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_SPGIST_ADD_LEAF: id = "ADD_LEAF"; diff --git a/src/backend/access/rmgrdesc/standbydesc.c b/src/backend/access/rmgrdesc/standbydesc.c index 9a08e10fe1..56acea1407 100644 --- a/src/backend/access/rmgrdesc/standbydesc.c +++ b/src/backend/access/rmgrdesc/standbydesc.c @@ -76,11 +76,11 @@ standby_desc(StringInfo buf, XLogReaderState *record) } const char * -standby_identify(uint8 info) +standby_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_STANDBY_LOCK: id = "LOCK"; diff --git a/src/backend/access/rmgrdesc/tblspcdesc.c b/src/backend/access/rmgrdesc/tblspcdesc.c index c1feb4fa05..c5849ccf7c 100644 --- a/src/backend/access/rmgrdesc/tblspcdesc.c +++ b/src/backend/access/rmgrdesc/tblspcdesc.c @@ -38,11 +38,11 @@ tblspc_desc(StringInfo buf, XLogReaderState *record) } const char * -tblspc_identify(uint8 info) +tblspc_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_TBLSPC_CREATE: id = "CREATE"; diff --git a/src/backend/access/rmgrdesc/xlogdesc.c b/src/backend/access/rmgrdesc/xlogdesc.c index 3692dfb96a..4db4f3331f 100644 --- a/src/backend/access/rmgrdesc/xlogdesc.c +++ b/src/backend/access/rmgrdesc/xlogdesc.c @@ -151,11 +151,11 @@ xlog_desc(StringInfo buf, XLogReaderState *record) } const char * -xlog_identify(uint8 info) +xlog_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_CHECKPOINT_SHUTDOWN: id = "CHECKPOINT_SHUTDOWN"; diff --git a/src/include/access/brin_xlog.h b/src/include/access/brin_xlog.h index ae263a371f..45a688d09b 100644 --- a/src/include/access/brin_xlog.h +++ b/src/include/access/brin_xlog.h @@ -145,7 +145,7 @@ typedef struct xl_brin_desummarize extern void brin_redo(XLogReaderState *record); extern void brin_desc(StringInfo buf, XLogReaderState *record); -extern const char *brin_identify(uint8 info); +extern const char *brin_identify(uint8 rmgrinfo); extern void brin_mask(char *pagedata, BlockNumber blkno); #endif /* BRIN_XLOG_H */ diff --git a/src/include/access/clog.h b/src/include/access/clog.h index d99444f073..224cad4415 100644 --- a/src/include/access/clog.h +++ b/src/include/access/clog.h @@ -58,6 +58,6 @@ extern int clogsyncfiletag(const FileTag *ftag, char *path); extern void clog_redo(XLogReaderState *record); extern void clog_desc(StringInfo buf, XLogReaderState *record); -extern const char *clog_identify(uint8 info); +extern const char *clog_identify(uint8 rmgrinfo); #endif /* CLOG_H */ diff --git a/src/include/access/ginxlog.h b/src/include/access/ginxlog.h index 37095e5f17..adf1f6b6d8 100644 --- a/src/include/access/ginxlog.h +++ b/src/include/access/ginxlog.h @@ -208,7 +208,7 @@ typedef struct ginxlogDeleteListPages extern void gin_redo(XLogReaderState *record); extern void gin_desc(StringInfo buf, XLogReaderState *record); -extern const char *gin_identify(uint8 info); +extern const char *gin_identify(uint8 rmgrinfo); extern void gin_xlog_startup(void); extern void gin_xlog_cleanup(void); extern void gin_mask(char *pagedata, BlockNumber blkno); diff --git a/src/include/access/gistxlog.h b/src/include/access/gistxlog.h index aff2ffbdcc..0ca37f3a07 100644 --- a/src/include/access/gistxlog.h +++ b/src/include/access/gistxlog.h @@ -109,7 +109,7 @@ typedef struct gistxlogPageReuse extern void gist_redo(XLogReaderState *record); extern void gist_desc(StringInfo buf, XLogReaderState *record); -extern const char *gist_identify(uint8 info); +extern const char *gist_identify(uint8 rmgrinfo); extern void gist_xlog_startup(void); extern void gist_xlog_cleanup(void); extern void gist_mask(char *pagedata, BlockNumber blkno); diff --git a/src/include/access/hash_xlog.h b/src/include/access/hash_xlog.h index b93619d1a8..74859a2a47 100644 --- a/src/include/access/hash_xlog.h +++ b/src/include/access/hash_xlog.h @@ -263,7 +263,7 @@ typedef struct xl_hash_vacuum_one_page extern void hash_redo(XLogReaderState *record); extern void hash_desc(StringInfo buf, XLogReaderState *record); -extern const char *hash_identify(uint8 info); +extern const char *hash_identify(uint8 rmgrinfo); extern void hash_mask(char *pagedata, BlockNumber blkno); #endif /* HASH_XLOG_H */ diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h index a038450787..70e9acd350 100644 --- a/src/include/access/heapam_xlog.h +++ b/src/include/access/heapam_xlog.h @@ -406,11 +406,11 @@ extern void HeapTupleHeaderAdvanceConflictHorizon(HeapTupleHeader tuple, extern void heap_redo(XLogReaderState *record); extern void heap_desc(StringInfo buf, XLogReaderState *record); -extern const char *heap_identify(uint8 info); +extern const char *heap_identify(uint8 rmgrinfo); extern void heap_mask(char *pagedata, BlockNumber blkno); extern void heap2_redo(XLogReaderState *record); extern void heap2_desc(StringInfo buf, XLogReaderState *record); -extern const char *heap2_identify(uint8 info); +extern const char *heap2_identify(uint8 rmgrinfo); extern void heap_xlog_logical_rewrite(XLogReaderState *r); extern XLogRecPtr log_heap_visible(Relation rel, Buffer heap_buffer, diff --git a/src/include/access/multixact.h b/src/include/access/multixact.h index 246f757f6a..d52e69aa05 100644 --- a/src/include/access/multixact.h +++ b/src/include/access/multixact.h @@ -158,7 +158,7 @@ extern void multixact_twophase_postabort(TransactionId xid, uint16 info, extern void multixact_redo(XLogReaderState *record); extern void multixact_desc(StringInfo buf, XLogReaderState *record); -extern const char *multixact_identify(uint8 info); +extern const char *multixact_identify(uint8 rmgrinfo); extern char *mxid_to_string(MultiXactId multi, int nmembers, MultiXactMember *members); diff --git a/src/include/access/nbtxlog.h b/src/include/access/nbtxlog.h index 7dd9fd0206..422ac13d56 100644 --- a/src/include/access/nbtxlog.h +++ b/src/include/access/nbtxlog.h @@ -362,6 +362,6 @@ extern void btree_mask(char *pagedata, BlockNumber blkno); * prototypes for functions in nbtdesc.c */ extern void btree_desc(StringInfo buf, XLogReaderState *record); -extern const char *btree_identify(uint8 info); +extern const char *btree_identify(uint8 rmgrinfo); #endif /* NBTXLOG_H */ diff --git a/src/include/access/spgxlog.h b/src/include/access/spgxlog.h index a7f10bf2d3..3533894baf 100644 --- a/src/include/access/spgxlog.h +++ b/src/include/access/spgxlog.h @@ -251,7 +251,7 @@ typedef struct spgxlogVacuumRedirect extern void spg_redo(XLogReaderState *record); extern void spg_desc(StringInfo buf, XLogReaderState *record); -extern const char *spg_identify(uint8 info); +extern const char *spg_identify(uint8 rmgrinfo); extern void spg_xlog_startup(void); extern void spg_xlog_cleanup(void); extern void spg_mask(char *pagedata, BlockNumber blkno); diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h index 48ca852381..9007f559f3 100644 --- a/src/include/access/xlog.h +++ b/src/include/access/xlog.h @@ -211,7 +211,7 @@ extern void XLogSetReplicationSlotMinimumLSN(XLogRecPtr lsn); extern void xlog_redo(struct XLogReaderState *record); extern void xlog_desc(StringInfo buf, struct XLogReaderState *record); -extern const char *xlog_identify(uint8 info); +extern const char *xlog_identify(uint8 rmgrinfo); extern void issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli); diff --git a/src/include/catalog/storage_xlog.h b/src/include/catalog/storage_xlog.h index 6b0a7aa3df..27514163be 100644 --- a/src/include/catalog/storage_xlog.h +++ b/src/include/catalog/storage_xlog.h @@ -54,6 +54,6 @@ extern void log_smgrcreate(const RelFileLocator *rlocator, ForkNumber forkNum); extern void smgr_redo(XLogReaderState *record); extern void smgr_desc(StringInfo buf, XLogReaderState *record); -extern const char *smgr_identify(uint8 info); +extern const char *smgr_identify(uint8 rmgrinfo); #endif /* STORAGE_XLOG_H */ diff --git a/src/include/commands/dbcommands_xlog.h b/src/include/commands/dbcommands_xlog.h index a1433275f5..f3c11060b6 100644 --- a/src/include/commands/dbcommands_xlog.h +++ b/src/include/commands/dbcommands_xlog.h @@ -55,6 +55,6 @@ typedef struct xl_dbase_drop_rec extern void dbase_redo(XLogReaderState *record); extern void dbase_desc(StringInfo buf, XLogReaderState *record); -extern const char *dbase_identify(uint8 info); +extern const char *dbase_identify(uint8 rmgrinfo); #endif /* DBCOMMANDS_XLOG_H */ diff --git a/src/include/commands/sequence.h b/src/include/commands/sequence.h index 7db7b3da7b..fb7059b8f6 100644 --- a/src/include/commands/sequence.h +++ b/src/include/commands/sequence.h @@ -64,7 +64,7 @@ extern void ResetSequenceCaches(void); extern void seq_redo(XLogReaderState *record); extern void seq_desc(StringInfo buf, XLogReaderState *record); -extern const char *seq_identify(uint8 info); +extern const char *seq_identify(uint8 rmgrinfo); extern void seq_mask(char *page, BlockNumber blkno); #endif /* SEQUENCE_H */ diff --git a/src/include/commands/tablespace.h b/src/include/commands/tablespace.h index f1961c1813..db21eb59ee 100644 --- a/src/include/commands/tablespace.h +++ b/src/include/commands/tablespace.h @@ -64,6 +64,6 @@ extern void remove_tablespace_symlink(const char *linkloc); extern void tblspc_redo(XLogReaderState *record); extern void tblspc_desc(StringInfo buf, XLogReaderState *record); -extern const char *tblspc_identify(uint8 info); +extern const char *tblspc_identify(uint8 rmgrinfo); #endif /* TABLESPACE_H */ diff --git a/src/include/replication/message.h b/src/include/replication/message.h index 6ce7f2038b..2fe985a6fe 100644 --- a/src/include/replication/message.h +++ b/src/include/replication/message.h @@ -36,6 +36,6 @@ extern XLogRecPtr LogLogicalMessage(const char *prefix, const char *message, #define XLOG_LOGICAL_MESSAGE 0x00 extern void logicalmsg_redo(XLogReaderState *record); extern void logicalmsg_desc(StringInfo buf, XLogReaderState *record); -extern const char *logicalmsg_identify(uint8 info); +extern const char *logicalmsg_identify(uint8 rmgrinfo); #endif /* PG_LOGICAL_MESSAGE_H */ diff --git a/src/include/storage/standbydefs.h b/src/include/storage/standbydefs.h index 188e348618..84e1f03c8f 100644 --- a/src/include/storage/standbydefs.h +++ b/src/include/storage/standbydefs.h @@ -22,7 +22,7 @@ /* Recovery handlers for the Standby Rmgr (RM_STANDBY_ID) */ extern void standby_redo(XLogReaderState *record); extern void standby_desc(StringInfo buf, XLogReaderState *record); -extern const char *standby_identify(uint8 info); +extern const char *standby_identify(uint8 rmgrinfo); extern void standby_desc_invalidations(StringInfo buf, int nmsgs, SharedInvalidationMessage *msgs, Oid dbId, Oid tsId, diff --git a/src/include/utils/relmapper.h b/src/include/utils/relmapper.h index 5c173bdbb3..e3c41ff093 100644 --- a/src/include/utils/relmapper.h +++ b/src/include/utils/relmapper.h @@ -68,6 +68,6 @@ extern void RestoreRelationMap(char *startAddress); extern void relmap_redo(XLogReaderState *record); extern void relmap_desc(StringInfo buf, XLogReaderState *record); -extern const char *relmap_identify(uint8 info); +extern const char *relmap_identify(uint8 rmgrinfo); #endif /* RELMAPPER_H */ diff --git a/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c b/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c index b5537a5f5b..21514f8e33 100644 --- a/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c +++ b/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c @@ -103,9 +103,9 @@ testcustomrmgrs_desc(StringInfo buf, XLogReaderState *record) } const char * -testcustomrmgrs_identify(uint8 info) +testcustomrmgrs_identify(uint8 rmgrinfo) { - if ((info & ~XLR_INFO_MASK) == XLOG_TEST_CUSTOM_RMGRS_MESSAGE) + if (rmgrinfo == XLOG_TEST_CUSTOM_RMGRS_MESSAGE) return "TEST_CUSTOM_RMGRS_MESSAGE"; return NULL; -- 2.40.1 [application/octet-stream] v4-0005-Stop-the-logging-of-XIDs-in-records-where-the-XID.patch (10.4K, ../../CAEze2WjVNdfP9sbMW=6zbgfae-UF_XVCpgkxp7BcZUcFBYa7tQ@mail.gmail.com/3-v4-0005-Stop-the-logging-of-XIDs-in-records-where-the-XID.patch) download | inline diff: From 7a98a4bf52eb38da49147f7c007667f9573b9062 Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Wed, 25 Jan 2023 15:40:30 +0100 Subject: [PATCH v4 5/7] Stop the logging of XIDs in records where the XID is unused. Many WAL record types do not need to contain an XID. This change prepares for the removal of the xid from the main xlog record header, by requiring the user of the XLog insertion API to explicitly request the inclusion of the current xid in the log record - in all other cases we'll log, assume and track InvalidXid instead. --- src/backend/access/heap/heapam.c | 16 +++++++++------- src/backend/access/heap/pruneheap.c | 1 + src/backend/access/heap/rewriteheap.c | 1 + src/backend/access/transam/generic_xlog.c | 3 +++ src/backend/access/transam/multixact.c | 1 + src/backend/access/transam/xact.c | 4 ++-- src/backend/access/transam/xloginsert.c | 12 ++++++++++-- src/backend/commands/tablecmds.c | 2 +- src/backend/replication/logical/message.c | 2 +- src/backend/utils/cache/inval.c | 1 + src/include/access/xlog.h | 1 + 11 files changed, 31 insertions(+), 13 deletions(-) diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 1512f22c76..30b7b05f54 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -1969,7 +1969,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid, heaptup->t_len - SizeofHeapTupleHeader); /* filtering by origin on a row level is much more efficient */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP_ID, info); @@ -2365,7 +2365,7 @@ heap_multi_insert(Relation relation, TupleTableSlot **slots, int ntuples, XLogRegisterBufData(0, tupledata, totaldatalen); /* filtering by origin on a row level is much more efficient */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP2_ID, info); @@ -2865,7 +2865,7 @@ l1: } /* filtering by origin on a row level is much more efficient */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP_ID, XLOG_HEAP_DELETE); @@ -5535,6 +5535,7 @@ l4: cleared_all_frozen ? XLH_LOCK_ALL_FROZEN_CLEARED : 0; XLogRegisterData((char *) &xlrec, SizeOfHeapLockUpdated); + XLogSetRecordFlags(XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP2_ID, XLOG_HEAP2_LOCK_UPDATED); @@ -5685,7 +5686,7 @@ heap_finish_speculative(Relation relation, ItemPointer tid) XLogBeginInsert(); /* We want the same filtering on this as on a plain insert */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); XLogRegisterData((char *) &xlrec, SizeOfHeapConfirm); XLogRegisterBuffer(0, buffer, REGBUF_STANDARD); @@ -5831,7 +5832,8 @@ heap_abort_speculative(Relation relation, ItemPointer tid) XLogRegisterData((char *) &xlrec, SizeOfHeapDelete); XLogRegisterBuffer(0, buffer, REGBUF_STANDARD); - /* No replica identity & replication origin logged */ + /* No replica identity & replication origin logged, but XID is required */ + XLogSetRecordFlags(XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP_ID, XLOG_HEAP_DELETE); @@ -8563,7 +8565,7 @@ log_heap_update(Relation reln, Buffer oldbuf, } /* filtering by origin on a row level is much more efficient */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP_ID, info); @@ -8638,7 +8640,7 @@ log_heap_new_cid(Relation relation, HeapTuple tup) XLogRegisterData((char *) &xlrec, SizeOfHeapNewCid); /* will be looked at irrespective of origin */ - + XLogSetRecordFlags(XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP2_ID, XLOG_HEAP2_NEW_CID); return recptr; diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index 47b9e20915..24e1bbe1ad 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -445,6 +445,7 @@ heap_page_prune(Relation relation, Buffer buffer, XLogRegisterBufData(0, (char *) prstate.nowunused, prstate.nunused * sizeof(OffsetNumber)); + XLogSetRecordFlags(XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP2_ID, XLOG_HEAP2_PRUNE); PageSetLSN(BufferGetPage(buffer), recptr); diff --git a/src/backend/access/heap/rewriteheap.c b/src/backend/access/heap/rewriteheap.c index 424958912c..9f77537335 100644 --- a/src/backend/access/heap/rewriteheap.c +++ b/src/backend/access/heap/rewriteheap.c @@ -921,6 +921,7 @@ logical_heap_rewrite_flush_mappings(RewriteState state) src->off += len; XLogBeginInsert(); + XLogSetRecordFlags(XLOG_INCLUDE_XID); XLogRegisterData((char *) (&xlrec), sizeof(xlrec)); XLogRegisterData(waldata_start, len); diff --git a/src/backend/access/transam/generic_xlog.c b/src/backend/access/transam/generic_xlog.c index 6c68191ca6..c8983b40ad 100644 --- a/src/backend/access/transam/generic_xlog.c +++ b/src/backend/access/transam/generic_xlog.c @@ -399,6 +399,9 @@ GenericXLogFinish(GenericXLogState *state) } } + /* Assume XID is required */ + XLogSetRecordFlags(XLOG_INCLUDE_XID); + /* Insert xlog record */ lsn = XLogInsert(RM_GENERIC_ID, 0); diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c index c70b6b7358..883da6a33b 100644 --- a/src/backend/access/transam/multixact.c +++ b/src/backend/access/transam/multixact.c @@ -834,6 +834,7 @@ MultiXactIdCreateFromMembers(int nmembers, MultiXactMember *members) * Not clear that it's worth the trouble though. */ XLogBeginInsert(); + XLogSetRecordFlags(XLOG_INCLUDE_XID); XLogRegisterData((char *) (&xlrec), SizeOfMultiXactCreate); XLogRegisterData((char *) members, nmembers * sizeof(MultiXactMember)); diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index da0ac6c8d3..57504594b0 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -5793,7 +5793,7 @@ XactLogCommitRecord(TimestampTz commit_time, XLogRegisterData((char *) (&xl_origin), sizeof(xl_xact_origin)); /* we allow filtering by xacts */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); return XLogInsertExtended(RM_XACT_ID, info, rmgr_info); } @@ -5940,7 +5940,7 @@ XactLogAbortRecord(TimestampTz abort_time, XLogRegisterData((char *) (&xl_origin), sizeof(xl_xact_origin)); /* Include the replication origin */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); return XLogInsertExtended(RM_XACT_ID, info, rmgr_info); } diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c index 9a3eb4fb4b..4816c5284b 100644 --- a/src/backend/access/transam/xloginsert.c +++ b/src/backend/access/transam/xloginsert.c @@ -456,6 +456,10 @@ XLogSetRecordFlags(uint8 flags) * (LSN is the XLOG point up to which the XLOG must be flushed to disk * before the data page can be written out. This implements the basic * WAL rule "write the log before the data".) + * + * Note: To include the current backend's TransactionID in the record, + * you have to set the XLOG_INCLUDE_XID flag using XLogRecordSetFlags + * before calling XLogInsert. */ XLogRecPtr XLogInsert(RmgrId rmid, uint8 rmgr_info) @@ -847,7 +851,7 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP } /* followed by toplevel XID, if not already included in previous record */ - if (IsSubxactTopXidLogPending()) + if (curinsert_flags & XLOG_INCLUDE_XID && IsSubxactTopXidLogPending()) { TransactionId xid = GetTopTransactionIdIfAny(); @@ -924,7 +928,11 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP * once we know where in the WAL the record will be inserted. The CRC does * not include the record header yet. */ - rechdr->xl_xid = GetCurrentTransactionIdIfAny(); + if (curinsert_flags & XLOG_INCLUDE_XID) + rechdr->xl_xid = GetCurrentTransactionIdIfAny(); + else + rechdr->xl_xid = InvalidTransactionId; + rechdr->xl_tot_len = (uint32) total_len; rechdr->xl_info = info; rechdr->xl_rmid = rmid; diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 8fff036b73..585c58c8d6 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -2133,7 +2133,7 @@ ExecuteTruncateGuts(List *explicit_rels, XLogRegisterData((char *) &xlrec, SizeOfHeapTruncate); XLogRegisterData((char *) logrelids, list_length(relids_logged) * sizeof(Oid)); - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); (void) XLogInsert(RM_HEAP_ID, XLOG_HEAP_TRUNCATE); } diff --git a/src/backend/replication/logical/message.c b/src/backend/replication/logical/message.c index c31dc1203f..af86d18457 100644 --- a/src/backend/replication/logical/message.c +++ b/src/backend/replication/logical/message.c @@ -69,7 +69,7 @@ LogLogicalMessage(const char *prefix, const char *message, size_t size, XLogRegisterData(unconstify(char *, message), size); /* allow origin filtering */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); return XLogInsert(RM_LOGICALMSG_ID, XLOG_LOGICAL_MESSAGE); } diff --git a/src/backend/utils/cache/inval.c b/src/backend/utils/cache/inval.c index 0008826f67..b0afdd6f90 100644 --- a/src/backend/utils/cache/inval.c +++ b/src/backend/utils/cache/inval.c @@ -1632,6 +1632,7 @@ LogLogicalInvalidations(void) ProcessMessageSubGroupMulti(group, RelCacheMsgs, XLogRegisterData((char *) msgs, n * sizeof(SharedInvalidationMessage))); + XLogSetRecordFlags(XLOG_INCLUDE_XID); XLogInsert(RM_XACT_ID, XLOG_XACT_INVALIDATIONS); } } diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h index 9007f559f3..f4dfaeda29 100644 --- a/src/include/access/xlog.h +++ b/src/include/access/xlog.h @@ -150,6 +150,7 @@ extern PGDLLIMPORT bool XLOG_DEBUG; */ #define XLOG_INCLUDE_ORIGIN 0x01 /* include the replication origin */ #define XLOG_MARK_UNIMPORTANT 0x02 /* record not important for durability */ +#define XLOG_INCLUDE_XID 0x04 /* include the transaction ID */ /* Checkpoint statistics */ -- 2.40.1 [application/octet-stream] v4-0004-Move-rmgr-info-bits-into-their-own-field-in-the-x.patch (26.1K, ../../CAEze2WjVNdfP9sbMW=6zbgfae-UF_XVCpgkxp7BcZUcFBYa7tQ@mail.gmail.com/4-v4-0004-Move-rmgr-info-bits-into-their-own-field-in-the-x.patch) download | inline diff: From a063782e13084ca2082d8f18502bbee7c555cead Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Mon, 23 Jan 2023 17:21:12 +0100 Subject: [PATCH v4 4/7] Move rmgr info bits into their own field in the xlog record header. This takes one byte of the 2 previously lost to alignment, and allows each rmgr to use more info bits (useful for e.g. merging the heap rmgrs back into one). --- contrib/pg_walinspect/pg_walinspect.c | 7 +++--- src/backend/access/transam/xact.c | 22 +++++++++-------- src/backend/access/transam/xlog.c | 7 +++--- src/backend/access/transam/xloginsert.c | 27 ++++++++++++--------- src/backend/access/transam/xlogprefetcher.c | 2 +- src/backend/access/transam/xlogreader.c | 2 +- src/backend/access/transam/xlogrecovery.c | 10 ++++---- src/backend/access/transam/xlogstats.c | 12 ++++----- src/backend/catalog/storage.c | 7 +++--- src/backend/commands/dbcommands.c | 20 +++++++++------ src/backend/utils/probes.d | 2 +- src/bin/pg_resetwal/pg_resetwal.c | 3 ++- src/bin/pg_rewind/parsexlog.c | 4 +-- src/bin/pg_waldump/pg_waldump.c | 7 +++--- src/include/access/brin_xlog.h | 3 +-- src/include/access/heapam_xlog.h | 4 +-- src/include/access/nbtxlog.h | 3 +-- src/include/access/xact.h | 6 ++--- src/include/access/xlog_internal.h | 2 +- src/include/access/xloginsert.h | 3 ++- src/include/access/xlogreader.h | 4 +-- src/include/access/xlogrecord.h | 11 ++------- src/include/access/xlogstats.h | 5 ++-- 23 files changed, 89 insertions(+), 84 deletions(-) diff --git a/contrib/pg_walinspect/pg_walinspect.c b/contrib/pg_walinspect/pg_walinspect.c index 1b6f4cb178..909700f9a1 100644 --- a/contrib/pg_walinspect/pg_walinspect.c +++ b/contrib/pg_walinspect/pg_walinspect.c @@ -697,7 +697,7 @@ GetXLogSummaryStats(XLogStats *stats, ReturnSetInfo *rsinfo, { int rj; - for (rj = 0; rj < MAX_XLINFO_TYPES; rj++) + for (rj = 0; rj < MAX_XLRMGRINFO_TYPES; rj++) { const char *id; @@ -712,10 +712,9 @@ GetXLogSummaryStats(XLogStats *stats, ReturnSetInfo *rsinfo, old_cxt = MemoryContextSwitchTo(tmp_cxt); - /* the upper four bits in xl_info are the rmgr's */ - id = desc.rm_identify(rj << 4); + id = desc.rm_identify(rj); if (id == NULL) - id = psprintf("UNKNOWN (%x)", rj << 4); + id = psprintf("UNKNOWN (%x)", rj); FillXLogStatsRow(psprintf("%s/%s", desc.rm_name, id), count, total_count, rec_len, total_rec_len, fpi_len, diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c560f07390..da0ac6c8d3 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -5650,7 +5650,8 @@ XactLogCommitRecord(TimestampTz commit_time, xl_xact_invals xl_invals; xl_xact_twophase xl_twophase; xl_xact_origin xl_origin; - uint8 info; + uint8 info = 0; + uint8 rmgr_info; Assert(CritSectionCount > 0); @@ -5658,9 +5659,9 @@ XactLogCommitRecord(TimestampTz commit_time, /* decide between a plain and 2pc commit */ if (!TransactionIdIsValid(twophase_xid)) - info = XLOG_XACT_COMMIT; + rmgr_info = XLOG_XACT_COMMIT; else - info = XLOG_XACT_COMMIT_PREPARED; + rmgr_info = XLOG_XACT_COMMIT_PREPARED; /* First figure out and collect all the information needed */ @@ -5736,7 +5737,7 @@ XactLogCommitRecord(TimestampTz commit_time, } if (xl_xinfo.xinfo != 0) - info |= XLOG_XACT_HAS_INFO; + rmgr_info |= XLOG_XACT_HAS_INFO; /* Then include all the collected data into the commit record. */ @@ -5794,7 +5795,7 @@ XactLogCommitRecord(TimestampTz commit_time, /* we allow filtering by xacts */ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); - return XLogInsert(RM_XACT_ID, info); + return XLogInsertExtended(RM_XACT_ID, info, rmgr_info); } /* @@ -5820,7 +5821,8 @@ XactLogAbortRecord(TimestampTz abort_time, xl_xact_dbinfo xl_dbinfo; xl_xact_origin xl_origin; - uint8 info; + uint8 info = 0; + uint8 rmgr_info; Assert(CritSectionCount > 0); @@ -5828,9 +5830,9 @@ XactLogAbortRecord(TimestampTz abort_time, /* decide between a plain and 2pc abort */ if (!TransactionIdIsValid(twophase_xid)) - info = XLOG_XACT_ABORT; + rmgr_info = XLOG_XACT_ABORT; else - info = XLOG_XACT_ABORT_PREPARED; + rmgr_info = XLOG_XACT_ABORT_PREPARED; /* First figure out and collect all the information needed */ @@ -5889,7 +5891,7 @@ XactLogAbortRecord(TimestampTz abort_time, } if (xl_xinfo.xinfo != 0) - info |= XLOG_XACT_HAS_INFO; + rmgr_info |= XLOG_XACT_HAS_INFO; /* Then include all the collected data into the abort record. */ @@ -5940,7 +5942,7 @@ XactLogAbortRecord(TimestampTz abort_time, /* Include the replication origin */ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); - return XLogInsert(RM_XACT_ID, info); + return XLogInsertExtended(RM_XACT_ID, info, rmgr_info); } /* diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index b01cd6cc23..4b1ff0d1aa 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -738,9 +738,9 @@ XLogInsertRecord(XLogRecData *rdata, pg_crc32c rdata_crc; bool inserted; XLogRecord *rechdr = (XLogRecord *) rdata->data; - uint8 info = rechdr->xl_info & ~XLR_INFO_MASK; + uint8 rmgrinfo = rechdr->xl_rmgrinfo; bool isLogSwitch = (rechdr->xl_rmid == RM_XLOG_ID && - info == XLOG_SWITCH); + rmgrinfo == XLOG_SWITCH); XLogRecPtr StartPos; XLogRecPtr EndPos; bool prevDoPageWrites = doPageWrites; @@ -4730,7 +4730,8 @@ BootStrapXLOG(void) record->xl_prev = 0; record->xl_xid = InvalidTransactionId; record->xl_tot_len = SizeOfXLogRecord + SizeOfXLogRecordDataHeaderShort + sizeof(checkPoint); - record->xl_info = XLOG_CHECKPOINT_SHUTDOWN; + record->xl_info = 0; + record->xl_rmgrinfo = XLOG_CHECKPOINT_SHUTDOWN; record->xl_rmid = RM_XLOG_ID; recptr += SizeOfXLogRecord; /* fill the XLogRecordDataHeaderShort struct */ diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c index 0fce063ca4..9a3eb4fb4b 100644 --- a/src/backend/access/transam/xloginsert.c +++ b/src/backend/access/transam/xloginsert.c @@ -135,7 +135,7 @@ static bool begininsert_called = false; /* Memory context to hold the registered buffer and data references. */ static MemoryContext xloginsert_cxt; -static XLogRecData *XLogRecordAssemble(RmgrId rmid, uint8 info, +static XLogRecData *XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecPtr, bool doPageWrites, XLogRecPtr *fpw_lsn, int *num_fpi, bool *topxid_included); @@ -447,9 +447,9 @@ XLogSetRecordFlags(uint8 flags) } /* - * Insert an XLOG record having the specified RMID and info bytes, with the - * body of the record being the data and buffer references registered earlier - * with XLogRegister* calls. + * Insert an XLOG record having the specified RMID and rmgr_info bytes, with + * the body of the record being the data and buffer references registered + * earlier with XLogRegister* calls. * * Returns XLOG pointer to end of record (beginning of next record). * This can be used as LSN for data pages affected by the logged action. @@ -458,7 +458,13 @@ XLogSetRecordFlags(uint8 flags) * WAL rule "write the log before the data".) */ XLogRecPtr -XLogInsert(RmgrId rmid, uint8 info) +XLogInsert(RmgrId rmid, uint8 rmgr_info) +{ + return XLogInsertExtended(rmid, 0, rmgr_info); +} + +XLogRecPtr +XLogInsertExtended(RmgrId rmid, uint8 info, uint8 rmgr_info) { XLogRecPtr EndPos; @@ -470,12 +476,11 @@ XLogInsert(RmgrId rmid, uint8 info) * The caller can set rmgr bits, XLR_SPECIAL_REL_UPDATE and * XLR_CHECK_CONSISTENCY; the rest are reserved for use by me. */ - if ((info & ~(XLR_RMGR_INFO_MASK | - XLR_SPECIAL_REL_UPDATE | + if ((info & ~(XLR_SPECIAL_REL_UPDATE | XLR_CHECK_CONSISTENCY)) != 0) elog(PANIC, "invalid xlog info mask %02X", info); - TRACE_POSTGRESQL_WAL_INSERT(rmid, info); + TRACE_POSTGRESQL_WAL_INSERT(rmid, info, rmgr_info); /* * In bootstrap mode, we don't actually log anything but XLOG resources; @@ -504,7 +509,7 @@ XLogInsert(RmgrId rmid, uint8 info) */ GetFullPageWriteInfo(&RedoRecPtr, &doPageWrites); - rdt = XLogRecordAssemble(rmid, info, RedoRecPtr, doPageWrites, + rdt = XLogRecordAssemble(rmid, info, rmgr_info, RedoRecPtr, doPageWrites, &fpw_lsn, &num_fpi, &topxid_included); EndPos = XLogInsertRecord(rdt, fpw_lsn, curinsert_flags, num_fpi, @@ -532,8 +537,7 @@ XLogInsert(RmgrId rmid, uint8 info) * current subtransaction. */ static XLogRecData * -XLogRecordAssemble(RmgrId rmid, uint8 info, - XLogRecPtr RedoRecPtr, bool doPageWrites, +XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecPtr, bool doPageWrites, XLogRecPtr *fpw_lsn, int *num_fpi, bool *topxid_included) { XLogRecData *rdt; @@ -924,6 +928,7 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, rechdr->xl_tot_len = (uint32) total_len; rechdr->xl_info = info; rechdr->xl_rmid = rmid; + rechdr->xl_rmgrinfo = rmgr_info; rechdr->xl_prev = InvalidXLogRecPtr; rechdr->xl_crc = rdata_crc; diff --git a/src/backend/access/transam/xlogprefetcher.c b/src/backend/access/transam/xlogprefetcher.c index 539928cb85..dd4ca866a4 100644 --- a/src/backend/access/transam/xlogprefetcher.c +++ b/src/backend/access/transam/xlogprefetcher.c @@ -536,7 +536,7 @@ XLogPrefetcherNextBlock(uintptr_t pgsr_private, XLogRecPtr *lsn) if (replaying_lsn < record->lsn) { uint8 rmid = record->header.xl_rmid; - uint8 record_type = record->header.xl_info & ~XLR_INFO_MASK; + uint8 record_type = record->header.xl_rmgrinfo; if (rmid == RM_XLOG_ID) { diff --git a/src/backend/access/transam/xlogreader.c b/src/backend/access/transam/xlogreader.c index 11267369bf..47bfb2266c 100644 --- a/src/backend/access/transam/xlogreader.c +++ b/src/backend/access/transam/xlogreader.c @@ -860,7 +860,7 @@ restart: * Special processing if it's an XLOG SWITCH record */ if (record->xl_rmid == RM_XLOG_ID && - (record->xl_info & ~XLR_INFO_MASK) == XLOG_SWITCH) + record->xl_rmgrinfo == XLOG_SWITCH) { /* Pretend it extends to end of segment */ state->NextRecPtr += state->segcxt.ws_segsize - 1; diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index 6508246999..56fa2f74a2 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -632,7 +632,7 @@ InitWalRecovery(ControlFileData *ControlFile, bool *wasShutdown_ptr, if (record != NULL) { memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint)); - wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN); + wasShutdown = (record->xl_rmgrinfo == XLOG_CHECKPOINT_SHUTDOWN); ereport(DEBUG1, (errmsg_internal("checkpoint record is at %X/%X", LSN_FORMAT_ARGS(CheckPointLoc)))); @@ -786,7 +786,7 @@ InitWalRecovery(ControlFileData *ControlFile, bool *wasShutdown_ptr, (errmsg("could not locate a valid checkpoint record"))); } memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint)); - wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN); + wasShutdown = (record->xl_rmgrinfo == XLOG_CHECKPOINT_SHUTDOWN); } /* @@ -1857,7 +1857,7 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl { TimeLineID newReplayTLI = *replayTLI; TimeLineID prevReplayTLI = *replayTLI; - uint8 info = record->xl_info & ~XLR_INFO_MASK; + uint8 info = record->xl_rmgrinfo; if (info == XLOG_CHECKPOINT_SHUTDOWN) { @@ -3992,12 +3992,12 @@ ReadCheckpointRecord(XLogPrefetcher *xlogprefetcher, XLogRecPtr RecPtr, (errmsg("invalid resource manager ID in checkpoint record"))); return NULL; } - info = record->xl_info & ~XLR_INFO_MASK; + info = record->xl_rmgrinfo; if (info != XLOG_CHECKPOINT_SHUTDOWN && info != XLOG_CHECKPOINT_ONLINE) { ereport(LOG, - (errmsg("invalid xl_info in checkpoint record"))); + (errmsg("invalid xl_rmgrinfo in checkpoint record"))); return NULL; } if (record->xl_tot_len != SizeOfXLogRecord + SizeOfXLogRecordDataHeaderShort + sizeof(CheckPoint)) diff --git a/src/backend/access/transam/xlogstats.c b/src/backend/access/transam/xlogstats.c index 267b108943..28d6d971ac 100644 --- a/src/backend/access/transam/xlogstats.c +++ b/src/backend/access/transam/xlogstats.c @@ -74,21 +74,21 @@ XLogRecStoreStats(XLogStats *stats, XLogReaderState *record) /* * Update per-record statistics, where the record is identified by a - * combination of the RmgrId and the four bits of the xl_info field that - * are the rmgr's domain (resulting in sixteen possible entries per + * combination of the RmgrId and the eight bits of the xl_rmgrinfo field + * that are the rmgr's domain (resulting in 256 possible entries per * RmgrId). */ - recid = XLogRecGetRmgrInfo(record) >> 4; + recid = XLogRecGetRmgrInfo(record); /* * XACT records need to be handled differently. Those records use the * first bit of those four bits for an optional flag variable and the - * following three bits for the opcode. We filter opcode out of xl_info - * and use it as the identifier of the record. + * following three bits for the opcode. We filter opcode out of + * xl_rmgrinfo and use it as the identifier of the record. */ if (rmid == RM_XACT_ID) - recid &= 0x07; + recid &= 0x70; stats->record_stats[rmid][recid].count++; stats->record_stats[rmid][recid].rec_len += rec_len; diff --git a/src/backend/catalog/storage.c b/src/backend/catalog/storage.c index 4d73023a65..5b48b24e7b 100644 --- a/src/backend/catalog/storage.c +++ b/src/backend/catalog/storage.c @@ -194,7 +194,7 @@ log_smgrcreate(const RelFileLocator *rlocator, ForkNumber forkNum) XLogBeginInsert(); XLogRegisterData((char *) &xlrec, sizeof(xlrec)); - XLogInsert(RM_SMGR_ID, XLOG_SMGR_CREATE | XLR_SPECIAL_REL_UPDATE); + XLogInsertExtended(RM_SMGR_ID, XLR_SPECIAL_REL_UPDATE, XLOG_SMGR_CREATE); } /* @@ -375,8 +375,9 @@ RelationTruncate(Relation rel, BlockNumber nblocks) XLogBeginInsert(); XLogRegisterData((char *) &xlrec, sizeof(xlrec)); - lsn = XLogInsert(RM_SMGR_ID, - XLOG_SMGR_TRUNCATE | XLR_SPECIAL_REL_UPDATE); + lsn = XLogInsertExtended(RM_SMGR_ID, + XLR_SPECIAL_REL_UPDATE, + XLOG_SMGR_TRUNCATE); /* * Flush, because otherwise the truncation of the main relation might diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c index dffb683e74..fe89540aed 100644 --- a/src/backend/commands/dbcommands.c +++ b/src/backend/commands/dbcommands.c @@ -624,8 +624,9 @@ CreateDatabaseUsingFileCopy(Oid src_dboid, Oid dst_dboid, Oid src_tsid, XLogRegisterData((char *) &xlrec, sizeof(xl_dbase_create_file_copy_rec)); - (void) XLogInsert(RM_DBASE_ID, - XLOG_DBASE_CREATE_FILE_COPY | XLR_SPECIAL_REL_UPDATE); + (void) XLogInsertExtended(RM_DBASE_ID, + XLR_SPECIAL_REL_UPDATE, + XLOG_DBASE_CREATE_FILE_COPY); } pfree(srcpath); pfree(dstpath); @@ -2086,8 +2087,9 @@ movedb(const char *dbname, const char *tblspcname) XLogRegisterData((char *) &xlrec, sizeof(xl_dbase_create_file_copy_rec)); - (void) XLogInsert(RM_DBASE_ID, - XLOG_DBASE_CREATE_FILE_COPY | XLR_SPECIAL_REL_UPDATE); + (void) XLogInsertExtended(RM_DBASE_ID, + XLR_SPECIAL_REL_UPDATE, + XLOG_DBASE_CREATE_FILE_COPY); } /* @@ -2180,8 +2182,9 @@ movedb(const char *dbname, const char *tblspcname) XLogRegisterData((char *) &xlrec, sizeof(xl_dbase_drop_rec)); XLogRegisterData((char *) &src_tblspcoid, sizeof(Oid)); - (void) XLogInsert(RM_DBASE_ID, - XLOG_DBASE_DROP | XLR_SPECIAL_REL_UPDATE); + (void) XLogInsertExtended(RM_DBASE_ID, + XLR_SPECIAL_REL_UPDATE, + XLOG_DBASE_DROP); } /* Now it's safe to release the database lock */ @@ -2903,8 +2906,9 @@ remove_dbtablespaces(Oid db_id) XLogRegisterData((char *) &xlrec, MinSizeOfDbaseDropRec); XLogRegisterData((char *) tablespace_ids, ntblspc * sizeof(Oid)); - (void) XLogInsert(RM_DBASE_ID, - XLOG_DBASE_DROP | XLR_SPECIAL_REL_UPDATE); + (void) XLogInsertExtended(RM_DBASE_ID, + XLR_SPECIAL_REL_UPDATE, + XLOG_DBASE_DROP); } list_free(ltblspc); diff --git a/src/backend/utils/probes.d b/src/backend/utils/probes.d index 0af275587b..131067b5ac 100644 --- a/src/backend/utils/probes.d +++ b/src/backend/utils/probes.d @@ -87,7 +87,7 @@ provider postgresql { probe smgr__md__write__start(ForkNumber, BlockNumber, Oid, Oid, Oid, int); probe smgr__md__write__done(ForkNumber, BlockNumber, Oid, Oid, Oid, int, int, int); - probe wal__insert(unsigned char, unsigned char); + probe wal__insert(unsigned char, unsigned char, unsigned char); probe wal__switch(); probe wal__buffer__write__dirty__start(); probe wal__buffer__write__dirty__done(); diff --git a/src/bin/pg_resetwal/pg_resetwal.c b/src/bin/pg_resetwal/pg_resetwal.c index e7ef2b8bd0..80cb9a5bc4 100644 --- a/src/bin/pg_resetwal/pg_resetwal.c +++ b/src/bin/pg_resetwal/pg_resetwal.c @@ -1066,7 +1066,8 @@ WriteEmptyXLOG(void) record->xl_prev = 0; record->xl_xid = InvalidTransactionId; record->xl_tot_len = SizeOfXLogRecord + SizeOfXLogRecordDataHeaderShort + sizeof(CheckPoint); - record->xl_info = XLOG_CHECKPOINT_SHUTDOWN; + record->xl_info = 0; + record->xl_rmgrinfo = XLOG_CHECKPOINT_SHUTDOWN; record->xl_rmid = RM_XLOG_ID; recptr += SizeOfXLogRecord; diff --git a/src/bin/pg_rewind/parsexlog.c b/src/bin/pg_rewind/parsexlog.c index 2b6b46f0e3..aa03fe8031 100644 --- a/src/bin/pg_rewind/parsexlog.c +++ b/src/bin/pg_rewind/parsexlog.c @@ -438,9 +438,9 @@ extractPageInfo(XLogReaderState *record) * track that change. */ pg_fatal("WAL record modifies a relation, but record type is not recognized: " - "lsn: %X/%X, rmid: %d, rmgr: %s, info: %02X", + "lsn: %X/%X, rmid: %d, rmgr: %s, info: %02X, rmgrinfo: %02X", LSN_FORMAT_ARGS(record->ReadRecPtr), - rmid, RmgrName(rmid), info | rminfo); + rmid, RmgrName(rmid), info, rminfo); } for (block_id = 0; block_id <= XLogRecMaxBlockId(record); block_id++) diff --git a/src/bin/pg_waldump/pg_waldump.c b/src/bin/pg_waldump/pg_waldump.c index eb6a7df119..cb16e72f58 100644 --- a/src/bin/pg_waldump/pg_waldump.c +++ b/src/bin/pg_waldump/pg_waldump.c @@ -694,7 +694,7 @@ XLogDumpDisplayStats(XLogDumpConfig *config, XLogStats *stats) } else { - for (rj = 0; rj < MAX_XLINFO_TYPES; rj++) + for (rj = 0; rj < MAX_XLRMGRINFO_TYPES; rj++) { const char *id; @@ -707,10 +707,9 @@ XLogDumpDisplayStats(XLogDumpConfig *config, XLogStats *stats) if (count == 0) continue; - /* the upper four bits in xl_info are the rmgr's */ - id = desc->rm_identify(rj << 4); + id = desc->rm_identify(rj); if (id == NULL) - id = psprintf("UNKNOWN (%x)", rj << 4); + id = psprintf("UNKNOWN (%x)", rj); XLogDumpStatsRow(psprintf("%s/%s", desc->rm_name, id), count, total_count, rec_len, total_rec_len, diff --git a/src/include/access/brin_xlog.h b/src/include/access/brin_xlog.h index 45a688d09b..92190688c9 100644 --- a/src/include/access/brin_xlog.h +++ b/src/include/access/brin_xlog.h @@ -25,8 +25,7 @@ /* * WAL record definitions for BRIN's WAL operations * - * XLOG allows to store some information in high 4 bits of log - * record xl_info field. + * XLOG allows to store some information in the 8-bit xl_rmgrinfo field. */ #define XLOG_BRIN_CREATE_INDEX 0x00 #define XLOG_BRIN_INSERT 0x10 diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h index 70e9acd350..6705c8a316 100644 --- a/src/include/access/heapam_xlog.h +++ b/src/include/access/heapam_xlog.h @@ -26,8 +26,8 @@ /* * WAL record definitions for heapam.c's WAL operations * - * XLOG allows to store some information in high 4 bits of log - * record xl_info field. We use 3 for opcode and one for init bit. + * XLOG allows to store some information in the 8-bit xl_rmgrinfo field. + * We use 3 for opcode and one for init bit. */ #define XLOG_HEAP_INSERT 0x00 #define XLOG_HEAP_DELETE 0x10 diff --git a/src/include/access/nbtxlog.h b/src/include/access/nbtxlog.h index 422ac13d56..6d1231a01e 100644 --- a/src/include/access/nbtxlog.h +++ b/src/include/access/nbtxlog.h @@ -21,8 +21,7 @@ /* * XLOG records for btree operations * - * XLOG allows to store some information in high 4 bits of log - * record xl_info field + * XLOG allows to store some information in the 8-bit xl_rmgrinfo field. */ #define XLOG_BTREE_INSERT_LEAF 0x00 /* add index tuple without split */ #define XLOG_BTREE_INSERT_UPPER 0x10 /* same, on a non-leaf page */ diff --git a/src/include/access/xact.h b/src/include/access/xact.h index b94b264402..2a9a708279 100644 --- a/src/include/access/xact.h +++ b/src/include/access/xact.h @@ -163,8 +163,8 @@ typedef struct SavedTransactionCharacteristics */ /* - * XLOG allows to store some information in high 4 bits of log record xl_info - * field. We use 3 for the opcode, and one about an optional flag variable. + * XLOG allows to store some information in the 8-bit xl_rmgrinfo field. + * We use 3 for the opcode, and one about an optional flag variable. */ #define XLOG_XACT_COMMIT 0x00 #define XLOG_XACT_PREPARE 0x10 @@ -232,7 +232,7 @@ typedef struct xl_xact_assignment * A minimal commit/abort record only consists of a xl_xact_commit/abort * struct. The presence of additional information is indicated by bits set in * 'xl_xact_xinfo->xinfo'. The presence of the xinfo field itself is signaled - * by a set XLOG_XACT_HAS_INFO bit in the xl_info field. + * by a set XLOG_XACT_HAS_INFO bit in the xl_rmgrinfo field. * * NB: All the individual data chunks should be sized to multiples of * sizeof(int) and only require int32 alignment. If they require bigger diff --git a/src/include/access/xlog_internal.h b/src/include/access/xlog_internal.h index 42b3c66547..876e2790f9 100644 --- a/src/include/access/xlog_internal.h +++ b/src/include/access/xlog_internal.h @@ -448,7 +448,7 @@ static inline Size XLogReadLength(uint32 *length, XLogSizeClass sizeClass, * This struct must be kept in sync with the PG_RMGR definition in * rmgr.c. * - * rm_identify must return a name for the record based on xl_info (without + * rm_identify must return a name for the record based on xl_rmgrinfo (without * reference to the rmid). For example, XLOG_BTREE_VACUUM would be named * "VACUUM". rm_desc can then be called to obtain additional detail for the * record, if available (e.g. the last block). diff --git a/src/include/access/xloginsert.h b/src/include/access/xloginsert.h index 31785dc578..c6c46e5409 100644 --- a/src/include/access/xloginsert.h +++ b/src/include/access/xloginsert.h @@ -41,7 +41,8 @@ /* prototypes for public functions in xloginsert.c: */ extern void XLogBeginInsert(void); extern void XLogSetRecordFlags(uint8 flags); -extern XLogRecPtr XLogInsert(RmgrId rmid, uint8 info); +extern XLogRecPtr XLogInsertExtended(RmgrId rmid, uint8 info, uint8 rmgr_info); +extern XLogRecPtr XLogInsert(RmgrId rmid, uint8 rmgr_info); extern void XLogEnsureRecordSpace(int max_block_id, int ndatas); extern void XLogRegisterData(char *data, uint32 len); extern void XLogRegisterBuffer(uint8 block_id, Buffer buffer, uint8 flags); diff --git a/src/include/access/xlogreader.h b/src/include/access/xlogreader.h index 2b976dcc03..a1d0216404 100644 --- a/src/include/access/xlogreader.h +++ b/src/include/access/xlogreader.h @@ -407,8 +407,8 @@ extern bool DecodeXLogRecord(XLogReaderState *state, */ #define XLogRecGetTotalLen(decoder) ((decoder)->record->header.xl_tot_len) #define XLogRecGetPrev(decoder) ((decoder)->record->header.xl_prev) -#define XLogRecGetInfo(decoder) ((decoder)->record->header.xl_info & XLR_INFO_MASK) -#define XLogRecGetRmgrInfo(decoder) (((decoder)->record->header.xl_info) & XLR_RMGR_INFO_MASK) +#define XLogRecGetInfo(decoder) ((decoder)->record->header.xl_info) +#define XLogRecGetRmgrInfo(decoder) ((decoder)->record->header.xl_rmgrinfo) #define XLogRecGetRmid(decoder) ((decoder)->record->header.xl_rmid) #define XLogRecGetXid(decoder) ((decoder)->record->header.xl_xid) #define XLogRecGetOrigin(decoder) ((decoder)->record->record_origin) diff --git a/src/include/access/xlogrecord.h b/src/include/access/xlogrecord.h index b0aada1031..42b06f163e 100644 --- a/src/include/access/xlogrecord.h +++ b/src/include/access/xlogrecord.h @@ -66,7 +66,8 @@ typedef struct XLogRecord XLogRecPtr xl_prev; /* ptr to previous record in log */ uint8 xl_info; /* flag bits, see below */ RmgrId xl_rmid; /* resource manager for this record */ - /* 2 bytes of padding here, initialize to zero */ + uint8 xl_rmgrinfo; /* rmgr flag bits, see below */ + /* 1 byte of padding here, initialize to zero */ pg_crc32c xl_crc; /* CRC for this record */ /* XLogRecordBlockHeaders and XLogRecordDataHeader follow, no padding */ @@ -75,14 +76,6 @@ typedef struct XLogRecord #define SizeOfXLogRecord (offsetof(XLogRecord, xl_crc) + sizeof(pg_crc32c)) -/* - * The high 4 bits in xl_info may be used freely by rmgr. The - * XLR_SPECIAL_REL_UPDATE and XLR_CHECK_CONSISTENCY bits can be passed by - * XLogInsert caller. The rest are set internally by XLogInsert. - */ -#define XLR_INFO_MASK 0x0F -#define XLR_RMGR_INFO_MASK 0xF0 - /* * XLogReader needs to allocate all the data of a WAL record in a single * chunk. This means that a single XLogRecord cannot exceed MaxAllocSize diff --git a/src/include/access/xlogstats.h b/src/include/access/xlogstats.h index 89410ce92b..30974f5ba7 100644 --- a/src/include/access/xlogstats.h +++ b/src/include/access/xlogstats.h @@ -16,7 +16,8 @@ #include "access/rmgr.h" #include "access/xlogreader.h" -#define MAX_XLINFO_TYPES 16 +/* all 8 bits of xl_rmgrinfo are available */ +#define MAX_XLRMGRINFO_TYPES 256 typedef struct XLogRecStats { @@ -33,7 +34,7 @@ typedef struct XLogStats XLogRecPtr endptr; #endif XLogRecStats rmgr_stats[RM_MAX_ID + 1]; - XLogRecStats record_stats[RM_MAX_ID + 1][MAX_XLINFO_TYPES]; + XLogRecStats record_stats[RM_MAX_ID + 1][MAX_XLRMGRINFO_TYPES]; } XLogStats; extern void XLogRecGetLen(XLogReaderState *record, uint32 *rec_len, -- 2.40.1 [application/octet-stream] v4-0002-Replace-accesses-to-rmgr-s-XLogRecord-info-bit-ac.patch (44.3K, ../../CAEze2WjVNdfP9sbMW=6zbgfae-UF_XVCpgkxp7BcZUcFBYa7tQ@mail.gmail.com/5-v4-0002-Replace-accesses-to-rmgr-s-XLogRecord-info-bit-ac.patch) download | inline diff: From 14ab6b04f6613ff5a3bd152ddb3682dbb8cc69b8 Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Mon, 23 Jan 2023 16:24:01 +0100 Subject: [PATCH v4 2/7] Replace accesses to rmgr's XLogRecord info bit accesses with XLogRecGetRmgrInfo This allows refactoring of the info bits into its own struct field, instead of sharing the field with the records own info bits. --- contrib/pg_walinspect/pg_walinspect.c | 8 ++--- src/backend/access/brin/brin_xlog.c | 4 +-- src/backend/access/gin/ginxlog.c | 2 +- src/backend/access/gist/gistxlog.c | 2 +- src/backend/access/hash/hash_xlog.c | 2 +- src/backend/access/heap/heapam.c | 10 +++---- src/backend/access/nbtree/nbtxlog.c | 2 +- src/backend/access/rmgrdesc/brindesc.c | 2 +- src/backend/access/rmgrdesc/clogdesc.c | 2 +- src/backend/access/rmgrdesc/committsdesc.c | 2 +- src/backend/access/rmgrdesc/dbasedesc.c | 2 +- src/backend/access/rmgrdesc/gindesc.c | 2 +- src/backend/access/rmgrdesc/gistdesc.c | 2 +- src/backend/access/rmgrdesc/hashdesc.c | 2 +- src/backend/access/rmgrdesc/heapdesc.c | 6 ++-- src/backend/access/rmgrdesc/logicalmsgdesc.c | 2 +- src/backend/access/rmgrdesc/mxactdesc.c | 2 +- src/backend/access/rmgrdesc/nbtdesc.c | 2 +- src/backend/access/rmgrdesc/relmapdesc.c | 2 +- src/backend/access/rmgrdesc/replorigindesc.c | 2 +- src/backend/access/rmgrdesc/seqdesc.c | 2 +- src/backend/access/rmgrdesc/smgrdesc.c | 2 +- src/backend/access/rmgrdesc/spgdesc.c | 2 +- src/backend/access/rmgrdesc/standbydesc.c | 2 +- src/backend/access/rmgrdesc/tblspcdesc.c | 2 +- src/backend/access/rmgrdesc/xactdesc.c | 30 +++++++++---------- src/backend/access/rmgrdesc/xlogdesc.c | 2 +- src/backend/access/spgist/spgxlog.c | 2 +- src/backend/access/transam/clog.c | 2 +- src/backend/access/transam/commit_ts.c | 2 +- src/backend/access/transam/multixact.c | 2 +- src/backend/access/transam/twophase.c | 2 +- src/backend/access/transam/xact.c | 10 +++---- src/backend/access/transam/xlog.c | 2 +- src/backend/access/transam/xlogrecovery.c | 22 +++++++------- src/backend/access/transam/xlogstats.c | 2 +- src/backend/catalog/storage.c | 2 +- src/backend/commands/dbcommands.c | 2 +- src/backend/commands/sequence.c | 2 +- src/backend/commands/tablespace.c | 2 +- src/backend/replication/logical/decode.c | 20 +++++++------ src/backend/replication/logical/message.c | 2 +- src/backend/replication/logical/origin.c | 2 +- src/backend/storage/ipc/standby.c | 2 +- src/backend/utils/cache/relmapper.c | 2 +- src/bin/pg_rewind/parsexlog.c | 6 ++-- src/bin/pg_waldump/pg_waldump.c | 4 +-- src/include/access/xact.h | 6 ++-- src/include/access/xlogreader.h | 3 +- .../test_custom_rmgrs/test_custom_rmgrs.c | 4 +-- 50 files changed, 105 insertions(+), 102 deletions(-) diff --git a/contrib/pg_walinspect/pg_walinspect.c b/contrib/pg_walinspect/pg_walinspect.c index 796a74f322..1b6f4cb178 100644 --- a/contrib/pg_walinspect/pg_walinspect.c +++ b/contrib/pg_walinspect/pg_walinspect.c @@ -196,10 +196,10 @@ GetWALRecordInfo(XLogReaderState *record, Datum *values, int i = 0; desc = GetRmgr(XLogRecGetRmid(record)); - record_type = desc.rm_identify(XLogRecGetInfo(record)); + record_type = desc.rm_identify(XLogRecGetRmgrInfo(record)); if (record_type == NULL) - record_type = psprintf("UNKNOWN (%x)", XLogRecGetInfo(record) & ~XLR_INFO_MASK); + record_type = psprintf("UNKNOWN (%x)", XLogRecGetRmgrInfo(record)); initStringInfo(&rec_desc); desc.rm_desc(&rec_desc, record); @@ -258,11 +258,11 @@ GetWALBlockInfo(FunctionCallInfo fcinfo, XLogReaderState *record, Assert(XLogRecHasAnyBlockRefs(record)); desc = GetRmgr(XLogRecGetRmid(record)); - record_type = desc.rm_identify(XLogRecGetInfo(record)); + record_type = desc.rm_identify(XLogRecGetRmgrInfo(record)); if (record_type == NULL) record_type = psprintf("UNKNOWN (%x)", - XLogRecGetInfo(record) & ~XLR_INFO_MASK); + XLogRecGetRmgrInfo(record)); initStringInfo(&rec_desc); desc.rm_desc(&rec_desc, record); diff --git a/src/backend/access/brin/brin_xlog.c b/src/backend/access/brin/brin_xlog.c index 89145b68f6..00dd85bdc5 100644 --- a/src/backend/access/brin/brin_xlog.c +++ b/src/backend/access/brin/brin_xlog.c @@ -56,7 +56,7 @@ brin_xlog_insert_update(XLogReaderState *record, * If we inserted the first and only tuple on the page, re-initialize the * page from scratch. */ - if (XLogRecGetInfo(record) & XLOG_BRIN_INIT_PAGE) + if (XLogRecGetRmgrInfo(record) & XLOG_BRIN_INIT_PAGE) { buffer = XLogInitBufferForRedo(record, 0); page = BufferGetPage(buffer); @@ -308,7 +308,7 @@ brin_xlog_desummarize_page(XLogReaderState *record) void brin_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info & XLOG_BRIN_OPMASK) { diff --git a/src/backend/access/gin/ginxlog.c b/src/backend/access/gin/ginxlog.c index f7c84beef8..feb675ed19 100644 --- a/src/backend/access/gin/ginxlog.c +++ b/src/backend/access/gin/ginxlog.c @@ -725,7 +725,7 @@ ginRedoDeleteListPages(XLogReaderState *record) void gin_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); MemoryContext oldCtx; /* diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index 15249aa921..a49210b64f 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -398,7 +398,7 @@ gistRedoPageReuse(XLogReaderState *record) void gist_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); MemoryContext oldCxt; /* diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index e8e06c62a9..3a45b7ee21 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -1048,7 +1048,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) void hash_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 7ed72abe59..1512f22c76 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -9255,7 +9255,7 @@ heap_xlog_insert(XLogReaderState *record) * If we inserted the first and only tuple on the page, re-initialize the * page from scratch. */ - if (XLogRecGetInfo(record) & XLOG_HEAP_INIT_PAGE) + if (XLogRecGetRmgrInfo(record) & XLOG_HEAP_INIT_PAGE) { buffer = XLogInitBufferForRedo(record, 0); page = BufferGetPage(buffer); @@ -9349,7 +9349,7 @@ heap_xlog_multi_insert(XLogReaderState *record) uint32 newlen; Size freespace = 0; int i; - bool isinit = (XLogRecGetInfo(record) & XLOG_HEAP_INIT_PAGE) != 0; + bool isinit = (XLogRecGetRmgrInfo(record) & XLOG_HEAP_INIT_PAGE) != 0; XLogRedoAction action; /* @@ -9597,7 +9597,7 @@ heap_xlog_update(XLogReaderState *record, bool hot_update) nbuffer = obuffer; newaction = oldaction; } - else if (XLogRecGetInfo(record) & XLOG_HEAP_INIT_PAGE) + else if (XLogRecGetRmgrInfo(record) & XLOG_HEAP_INIT_PAGE) { nbuffer = XLogInitBufferForRedo(record, 0); page = (Page) BufferGetPage(nbuffer); @@ -9961,7 +9961,7 @@ heap_xlog_inplace(XLogReaderState *record) void heap_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* * These operations don't overwrite MVCC data so no conflict processing is @@ -10007,7 +10007,7 @@ heap_redo(XLogReaderState *record) void heap2_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info & XLOG_HEAP_OPMASK) { diff --git a/src/backend/access/nbtree/nbtxlog.c b/src/backend/access/nbtree/nbtxlog.c index c87e46ed66..97d8d068c6 100644 --- a/src/backend/access/nbtree/nbtxlog.c +++ b/src/backend/access/nbtree/nbtxlog.c @@ -1015,7 +1015,7 @@ btree_xlog_reuse_page(XLogReaderState *record) void btree_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); MemoryContext oldCtx; oldCtx = MemoryContextSwitchTo(opCtx); diff --git a/src/backend/access/rmgrdesc/brindesc.c b/src/backend/access/rmgrdesc/brindesc.c index 1466a31bbb..1d3e5ee03e 100644 --- a/src/backend/access/rmgrdesc/brindesc.c +++ b/src/backend/access/rmgrdesc/brindesc.c @@ -20,7 +20,7 @@ void brin_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); info &= XLOG_BRIN_OPMASK; if (info == XLOG_BRIN_CREATE_INDEX) diff --git a/src/backend/access/rmgrdesc/clogdesc.c b/src/backend/access/rmgrdesc/clogdesc.c index e60b76f9da..915f9945d0 100644 --- a/src/backend/access/rmgrdesc/clogdesc.c +++ b/src/backend/access/rmgrdesc/clogdesc.c @@ -21,7 +21,7 @@ void clog_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == CLOG_ZEROPAGE) { diff --git a/src/backend/access/rmgrdesc/committsdesc.c b/src/backend/access/rmgrdesc/committsdesc.c index e7155cd507..3d673c2302 100644 --- a/src/backend/access/rmgrdesc/committsdesc.c +++ b/src/backend/access/rmgrdesc/committsdesc.c @@ -22,7 +22,7 @@ void commit_ts_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == COMMIT_TS_ZEROPAGE) { diff --git a/src/backend/access/rmgrdesc/dbasedesc.c b/src/backend/access/rmgrdesc/dbasedesc.c index 3922120d64..a9529bde05 100644 --- a/src/backend/access/rmgrdesc/dbasedesc.c +++ b/src/backend/access/rmgrdesc/dbasedesc.c @@ -22,7 +22,7 @@ void dbase_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_DBASE_CREATE_FILE_COPY) { diff --git a/src/backend/access/rmgrdesc/gindesc.c b/src/backend/access/rmgrdesc/gindesc.c index 246a6a6b85..08fc7ba49b 100644 --- a/src/backend/access/rmgrdesc/gindesc.c +++ b/src/backend/access/rmgrdesc/gindesc.c @@ -74,7 +74,7 @@ void gin_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/access/rmgrdesc/gistdesc.c b/src/backend/access/rmgrdesc/gistdesc.c index 5dc6e1dcee..6473069c1a 100644 --- a/src/backend/access/rmgrdesc/gistdesc.c +++ b/src/backend/access/rmgrdesc/gistdesc.c @@ -60,7 +60,7 @@ void gist_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/access/rmgrdesc/hashdesc.c b/src/backend/access/rmgrdesc/hashdesc.c index b6810a9320..bcb2013048 100644 --- a/src/backend/access/rmgrdesc/hashdesc.c +++ b/src/backend/access/rmgrdesc/hashdesc.c @@ -20,7 +20,7 @@ void hash_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/access/rmgrdesc/heapdesc.c b/src/backend/access/rmgrdesc/heapdesc.c index d73248abdd..15f943051a 100644 --- a/src/backend/access/rmgrdesc/heapdesc.c +++ b/src/backend/access/rmgrdesc/heapdesc.c @@ -95,7 +95,7 @@ void heap_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); info &= XLOG_HEAP_OPMASK; if (info == XLOG_HEAP_INSERT) @@ -172,7 +172,7 @@ void heap2_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); info &= XLOG_HEAP_OPMASK; if (info == XLOG_HEAP2_PRUNE) @@ -265,7 +265,7 @@ heap2_desc(StringInfo buf, XLogReaderState *record) else if (info == XLOG_HEAP2_MULTI_INSERT) { xl_heap_multi_insert *xlrec = (xl_heap_multi_insert *) rec; - bool isinit = (XLogRecGetInfo(record) & XLOG_HEAP_INIT_PAGE) != 0; + bool isinit = (XLogRecGetRmgrInfo(record) & XLOG_HEAP_INIT_PAGE) != 0; appendStringInfo(buf, "ntuples: %d, flags: 0x%02X", xlrec->ntuples, xlrec->flags); diff --git a/src/backend/access/rmgrdesc/logicalmsgdesc.c b/src/backend/access/rmgrdesc/logicalmsgdesc.c index 0578ec5535..2bb6cb06c3 100644 --- a/src/backend/access/rmgrdesc/logicalmsgdesc.c +++ b/src/backend/access/rmgrdesc/logicalmsgdesc.c @@ -19,7 +19,7 @@ void logicalmsg_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_LOGICAL_MESSAGE) { diff --git a/src/backend/access/rmgrdesc/mxactdesc.c b/src/backend/access/rmgrdesc/mxactdesc.c index a2fa1eca18..cdcf8a4f45 100644 --- a/src/backend/access/rmgrdesc/mxactdesc.c +++ b/src/backend/access/rmgrdesc/mxactdesc.c @@ -50,7 +50,7 @@ void multixact_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_MULTIXACT_ZERO_OFF_PAGE || info == XLOG_MULTIXACT_ZERO_MEM_PAGE) diff --git a/src/backend/access/rmgrdesc/nbtdesc.c b/src/backend/access/rmgrdesc/nbtdesc.c index e4fbaa4d5d..ede8df72c0 100644 --- a/src/backend/access/rmgrdesc/nbtdesc.c +++ b/src/backend/access/rmgrdesc/nbtdesc.c @@ -24,7 +24,7 @@ void btree_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/access/rmgrdesc/relmapdesc.c b/src/backend/access/rmgrdesc/relmapdesc.c index 82ad51eb9a..f4f0e5ce8b 100644 --- a/src/backend/access/rmgrdesc/relmapdesc.c +++ b/src/backend/access/rmgrdesc/relmapdesc.c @@ -20,7 +20,7 @@ void relmap_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_RELMAP_UPDATE) { diff --git a/src/backend/access/rmgrdesc/replorigindesc.c b/src/backend/access/rmgrdesc/replorigindesc.c index 8ce4588fda..539dc57939 100644 --- a/src/backend/access/rmgrdesc/replorigindesc.c +++ b/src/backend/access/rmgrdesc/replorigindesc.c @@ -19,7 +19,7 @@ void replorigin_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/access/rmgrdesc/seqdesc.c b/src/backend/access/rmgrdesc/seqdesc.c index ba60544085..6def7b653b 100644 --- a/src/backend/access/rmgrdesc/seqdesc.c +++ b/src/backend/access/rmgrdesc/seqdesc.c @@ -21,7 +21,7 @@ void seq_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); xl_seq_rec *xlrec = (xl_seq_rec *) rec; if (info == XLOG_SEQ_LOG) diff --git a/src/backend/access/rmgrdesc/smgrdesc.c b/src/backend/access/rmgrdesc/smgrdesc.c index bd841b96e8..8604bee0c0 100644 --- a/src/backend/access/rmgrdesc/smgrdesc.c +++ b/src/backend/access/rmgrdesc/smgrdesc.c @@ -21,7 +21,7 @@ void smgr_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_SMGR_CREATE) { diff --git a/src/backend/access/rmgrdesc/spgdesc.c b/src/backend/access/rmgrdesc/spgdesc.c index 87f62f0fb4..9e1e48240b 100644 --- a/src/backend/access/rmgrdesc/spgdesc.c +++ b/src/backend/access/rmgrdesc/spgdesc.c @@ -20,7 +20,7 @@ void spg_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/access/rmgrdesc/standbydesc.c b/src/backend/access/rmgrdesc/standbydesc.c index f2bce9a37a..9a08e10fe1 100644 --- a/src/backend/access/rmgrdesc/standbydesc.c +++ b/src/backend/access/rmgrdesc/standbydesc.c @@ -47,7 +47,7 @@ void standby_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_STANDBY_LOCK) { diff --git a/src/backend/access/rmgrdesc/tblspcdesc.c b/src/backend/access/rmgrdesc/tblspcdesc.c index b8c89f8c54..c1feb4fa05 100644 --- a/src/backend/access/rmgrdesc/tblspcdesc.c +++ b/src/backend/access/rmgrdesc/tblspcdesc.c @@ -21,7 +21,7 @@ void tblspc_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_TBLSPC_CREATE) { diff --git a/src/backend/access/rmgrdesc/xactdesc.c b/src/backend/access/rmgrdesc/xactdesc.c index 01610c5ddc..8903bb32de 100644 --- a/src/backend/access/rmgrdesc/xactdesc.c +++ b/src/backend/access/rmgrdesc/xactdesc.c @@ -32,7 +32,7 @@ */ void -ParseCommitRecord(uint8 info, xl_xact_commit *xlrec, xl_xact_parsed_commit *parsed) +ParseCommitRecord(uint8 rmgrinfo, xl_xact_commit *xlrec, xl_xact_parsed_commit *parsed) { char *data = ((char *) xlrec) + MinSizeOfXactCommit; @@ -43,7 +43,7 @@ ParseCommitRecord(uint8 info, xl_xact_commit *xlrec, xl_xact_parsed_commit *pars parsed->xact_time = xlrec->xact_time; - if (info & XLOG_XACT_HAS_INFO) + if (rmgrinfo & XLOG_XACT_HAS_INFO) { xl_xact_xinfo *xl_xinfo = (xl_xact_xinfo *) data; @@ -138,7 +138,7 @@ ParseCommitRecord(uint8 info, xl_xact_commit *xlrec, xl_xact_parsed_commit *pars } void -ParseAbortRecord(uint8 info, xl_xact_abort *xlrec, xl_xact_parsed_abort *parsed) +ParseAbortRecord(uint8 rmgrinfo, xl_xact_abort *xlrec, xl_xact_parsed_abort *parsed) { char *data = ((char *) xlrec) + MinSizeOfXactAbort; @@ -149,7 +149,7 @@ ParseAbortRecord(uint8 info, xl_xact_abort *xlrec, xl_xact_parsed_abort *parsed) parsed->xact_time = xlrec->xact_time; - if (info & XLOG_XACT_HAS_INFO) + if (rmgrinfo & XLOG_XACT_HAS_INFO) { xl_xact_xinfo *xl_xinfo = (xl_xact_xinfo *) data; @@ -236,7 +236,7 @@ ParseAbortRecord(uint8 info, xl_xact_abort *xlrec, xl_xact_parsed_abort *parsed) * ParsePrepareRecord */ void -ParsePrepareRecord(uint8 info, xl_xact_prepare *xlrec, xl_xact_parsed_prepare *parsed) +ParsePrepareRecord(uint8 rmgrinfo, xl_xact_prepare *xlrec, xl_xact_parsed_prepare *parsed) { char *bufptr; @@ -328,11 +328,11 @@ xact_desc_stats(StringInfo buf, const char *label, } static void -xact_desc_commit(StringInfo buf, uint8 info, xl_xact_commit *xlrec, RepOriginId origin_id) +xact_desc_commit(StringInfo buf, uint8 rmgrinfo, xl_xact_commit *xlrec, RepOriginId origin_id) { xl_xact_parsed_commit parsed; - ParseCommitRecord(info, xlrec, &parsed); + ParseCommitRecord(rmgrinfo, xlrec, &parsed); /* If this is a prepared xact, show the xid of the original xact */ if (TransactionIdIsValid(parsed.twophase_xid)) @@ -364,11 +364,11 @@ xact_desc_commit(StringInfo buf, uint8 info, xl_xact_commit *xlrec, RepOriginId } static void -xact_desc_abort(StringInfo buf, uint8 info, xl_xact_abort *xlrec, RepOriginId origin_id) +xact_desc_abort(StringInfo buf, uint8 rmgrinfo, xl_xact_abort *xlrec, RepOriginId origin_id) { xl_xact_parsed_abort parsed; - ParseAbortRecord(info, xlrec, &parsed); + ParseAbortRecord(rmgrinfo, xlrec, &parsed); /* If this is a prepared xact, show the xid of the original xact */ if (TransactionIdIsValid(parsed.twophase_xid)) @@ -391,11 +391,11 @@ xact_desc_abort(StringInfo buf, uint8 info, xl_xact_abort *xlrec, RepOriginId or } static void -xact_desc_prepare(StringInfo buf, uint8 info, xl_xact_prepare *xlrec, RepOriginId origin_id) +xact_desc_prepare(StringInfo buf, uint8 rmgrinfo, xl_xact_prepare *xlrec, RepOriginId origin_id) { xl_xact_parsed_prepare parsed; - ParsePrepareRecord(info, xlrec, &parsed); + ParsePrepareRecord(rmgrinfo, xlrec, &parsed); appendStringInfo(buf, "gid %s: ", parsed.twophase_gid); appendStringInfoString(buf, timestamptz_to_str(parsed.xact_time)); @@ -436,27 +436,27 @@ void xact_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & XLOG_XACT_OPMASK; + uint8 info = XLogRecGetRmgrInfo(record) & XLOG_XACT_OPMASK; if (info == XLOG_XACT_COMMIT || info == XLOG_XACT_COMMIT_PREPARED) { xl_xact_commit *xlrec = (xl_xact_commit *) rec; - xact_desc_commit(buf, XLogRecGetInfo(record), xlrec, + xact_desc_commit(buf, XLogRecGetRmgrInfo(record), xlrec, XLogRecGetOrigin(record)); } else if (info == XLOG_XACT_ABORT || info == XLOG_XACT_ABORT_PREPARED) { xl_xact_abort *xlrec = (xl_xact_abort *) rec; - xact_desc_abort(buf, XLogRecGetInfo(record), xlrec, + xact_desc_abort(buf, XLogRecGetRmgrInfo(record), xlrec, XLogRecGetOrigin(record)); } else if (info == XLOG_XACT_PREPARE) { xl_xact_prepare *xlrec = (xl_xact_prepare *) rec; - xact_desc_prepare(buf, XLogRecGetInfo(record), xlrec, + xact_desc_prepare(buf, XLogRecGetRmgrInfo(record), xlrec, XLogRecGetOrigin(record)); } else if (info == XLOG_XACT_ASSIGNMENT) diff --git a/src/backend/access/rmgrdesc/xlogdesc.c b/src/backend/access/rmgrdesc/xlogdesc.c index f390c177e4..3692dfb96a 100644 --- a/src/backend/access/rmgrdesc/xlogdesc.c +++ b/src/backend/access/rmgrdesc/xlogdesc.c @@ -37,7 +37,7 @@ void xlog_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_CHECKPOINT_SHUTDOWN || info == XLOG_CHECKPOINT_ONLINE) diff --git a/src/backend/access/spgist/spgxlog.c b/src/backend/access/spgist/spgxlog.c index 459ac929ba..53190a67f9 100644 --- a/src/backend/access/spgist/spgxlog.c +++ b/src/backend/access/spgist/spgxlog.c @@ -936,7 +936,7 @@ spgRedoVacuumRedirect(XLogReaderState *record) void spg_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); MemoryContext oldCxt; oldCxt = MemoryContextSwitchTo(opCtx); diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c index 4a431d5876..11c10b936d 100644 --- a/src/backend/access/transam/clog.c +++ b/src/backend/access/transam/clog.c @@ -984,7 +984,7 @@ WriteTruncateXlogRec(int pageno, TransactionId oldestXact, Oid oldestXactDb) void clog_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in clog records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/backend/access/transam/commit_ts.c b/src/backend/access/transam/commit_ts.c index b897fabc70..a351ee84ce 100644 --- a/src/backend/access/transam/commit_ts.c +++ b/src/backend/access/transam/commit_ts.c @@ -966,7 +966,7 @@ WriteTruncateXlogRec(int pageno, TransactionId oldestXid) void commit_ts_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in commit_ts records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c index abb022e067..c70b6b7358 100644 --- a/src/backend/access/transam/multixact.c +++ b/src/backend/access/transam/multixact.c @@ -3232,7 +3232,7 @@ WriteMTruncateXlogRec(Oid oldestMultiDB, void multixact_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in multixact records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c index 068e59bec0..f9ab5a9c7f 100644 --- a/src/backend/access/transam/twophase.c +++ b/src/backend/access/transam/twophase.c @@ -1429,7 +1429,7 @@ XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len) } if (XLogRecGetRmid(xlogreader) != RM_XACT_ID || - (XLogRecGetInfo(xlogreader) & XLOG_XACT_OPMASK) != XLOG_XACT_PREPARE) + (XLogRecGetRmgrInfo(xlogreader) & XLOG_XACT_OPMASK) != XLOG_XACT_PREPARE) ereport(ERROR, (errcode_for_file_access(), errmsg("expected two-phase state data is not present in WAL at %X/%X", diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index 8daaa535ed..c560f07390 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -6183,7 +6183,7 @@ xact_redo_abort(xl_xact_parsed_abort *parsed, TransactionId xid, void xact_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & XLOG_XACT_OPMASK; + uint8 info = XLogRecGetRmgrInfo(record) & XLOG_XACT_OPMASK; /* Backup blocks are not used in xact records */ Assert(!XLogRecHasAnyBlockRefs(record)); @@ -6193,7 +6193,7 @@ xact_redo(XLogReaderState *record) xl_xact_commit *xlrec = (xl_xact_commit *) XLogRecGetData(record); xl_xact_parsed_commit parsed; - ParseCommitRecord(XLogRecGetInfo(record), xlrec, &parsed); + ParseCommitRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); xact_redo_commit(&parsed, XLogRecGetXid(record), record->EndRecPtr, XLogRecGetOrigin(record)); } @@ -6202,7 +6202,7 @@ xact_redo(XLogReaderState *record) xl_xact_commit *xlrec = (xl_xact_commit *) XLogRecGetData(record); xl_xact_parsed_commit parsed; - ParseCommitRecord(XLogRecGetInfo(record), xlrec, &parsed); + ParseCommitRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); xact_redo_commit(&parsed, parsed.twophase_xid, record->EndRecPtr, XLogRecGetOrigin(record)); @@ -6216,7 +6216,7 @@ xact_redo(XLogReaderState *record) xl_xact_abort *xlrec = (xl_xact_abort *) XLogRecGetData(record); xl_xact_parsed_abort parsed; - ParseAbortRecord(XLogRecGetInfo(record), xlrec, &parsed); + ParseAbortRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); xact_redo_abort(&parsed, XLogRecGetXid(record), record->EndRecPtr, XLogRecGetOrigin(record)); } @@ -6225,7 +6225,7 @@ xact_redo(XLogReaderState *record) xl_xact_abort *xlrec = (xl_xact_abort *) XLogRecGetData(record); xl_xact_parsed_abort parsed; - ParseAbortRecord(XLogRecGetInfo(record), xlrec, &parsed); + ParseAbortRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); xact_redo_abort(&parsed, parsed.twophase_xid, record->EndRecPtr, XLogRecGetOrigin(record)); diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 8b0710abe6..b01cd6cc23 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -7719,7 +7719,7 @@ UpdateFullPageWrites(void) void xlog_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); XLogRecPtr lsn = record->EndRecPtr; /* diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index becc2bda62..6508246999 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -1995,7 +1995,7 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl static void xlogrecovery_redo(XLogReaderState *record, TimeLineID replayTLI) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); XLogRecPtr lsn = record->EndRecPtr; Assert(XLogRecGetRmid(record) == RM_XLOG_ID); @@ -2213,7 +2213,7 @@ void xlog_outdesc(StringInfo buf, XLogReaderState *record) { RmgrData rmgr = GetRmgr(XLogRecGetRmid(record)); - uint8 info = XLogRecGetInfo(record); + uint8 info = XLogRecGetRmgrInfo(record); const char *id; appendStringInfoString(buf, rmgr.rm_name); @@ -2221,7 +2221,7 @@ xlog_outdesc(StringInfo buf, XLogReaderState *record) id = rmgr.rm_identify(info); if (id == NULL) - appendStringInfo(buf, "UNKNOWN (%X): ", info & ~XLR_INFO_MASK); + appendStringInfo(buf, "UNKNOWN (%X): ", info); else appendStringInfo(buf, "%s: ", id); @@ -2341,7 +2341,7 @@ checkTimeLineSwitch(XLogRecPtr lsn, TimeLineID newTLI, TimeLineID prevTLI, static bool getRecordTimestamp(XLogReaderState *record, TimestampTz *recordXtime) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); uint8 xact_info = info & XLOG_XACT_OPMASK; uint8 rmid = XLogRecGetRmid(record); @@ -2535,7 +2535,7 @@ recoveryStopsBefore(XLogReaderState *record) if (XLogRecGetRmid(record) != RM_XACT_ID) return false; - xact_info = XLogRecGetInfo(record) & XLOG_XACT_OPMASK; + xact_info = XLogRecGetRmgrInfo(record); if (xact_info == XLOG_XACT_COMMIT) { @@ -2548,7 +2548,7 @@ recoveryStopsBefore(XLogReaderState *record) xl_xact_parsed_commit parsed; isCommit = true; - ParseCommitRecord(XLogRecGetInfo(record), + ParseCommitRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); recordXid = parsed.twophase_xid; @@ -2564,7 +2564,7 @@ recoveryStopsBefore(XLogReaderState *record) xl_xact_parsed_abort parsed; isCommit = false; - ParseAbortRecord(XLogRecGetInfo(record), + ParseAbortRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); recordXid = parsed.twophase_xid; @@ -2653,7 +2653,7 @@ recoveryStopsAfter(XLogReaderState *record) if (!ArchiveRecoveryRequested) return false; - info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + info = XLogRecGetRmgrInfo(record); rmid = XLogRecGetRmid(record); /* @@ -2721,7 +2721,7 @@ recoveryStopsAfter(XLogReaderState *record) xl_xact_commit *xlrec = (xl_xact_commit *) XLogRecGetData(record); xl_xact_parsed_commit parsed; - ParseCommitRecord(XLogRecGetInfo(record), + ParseCommitRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); recordXid = parsed.twophase_xid; @@ -2731,7 +2731,7 @@ recoveryStopsAfter(XLogReaderState *record) xl_xact_abort *xlrec = (xl_xact_abort *) XLogRecGetData(record); xl_xact_parsed_abort parsed; - ParseAbortRecord(XLogRecGetInfo(record), + ParseAbortRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); recordXid = parsed.twophase_xid; @@ -2925,7 +2925,7 @@ recoveryApplyDelay(XLogReaderState *record) if (XLogRecGetRmid(record) != RM_XACT_ID) return false; - xact_info = XLogRecGetInfo(record) & XLOG_XACT_OPMASK; + xact_info = XLogRecGetRmgrInfo(record) & XLOG_XACT_OPMASK; if (xact_info != XLOG_XACT_COMMIT && xact_info != XLOG_XACT_COMMIT_PREPARED) diff --git a/src/backend/access/transam/xlogstats.c b/src/backend/access/transam/xlogstats.c index 2d315df67a..267b108943 100644 --- a/src/backend/access/transam/xlogstats.c +++ b/src/backend/access/transam/xlogstats.c @@ -79,7 +79,7 @@ XLogRecStoreStats(XLogStats *stats, XLogReaderState *record) * RmgrId). */ - recid = XLogRecGetInfo(record) >> 4; + recid = XLogRecGetRmgrInfo(record) >> 4; /* * XACT records need to be handled differently. Those records use the diff --git a/src/backend/catalog/storage.c b/src/backend/catalog/storage.c index 2add053489..4d73023a65 100644 --- a/src/backend/catalog/storage.c +++ b/src/backend/catalog/storage.c @@ -958,7 +958,7 @@ void smgr_redo(XLogReaderState *record) { XLogRecPtr lsn = record->EndRecPtr; - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in smgr records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c index 09f1ab41ad..dffb683e74 100644 --- a/src/backend/commands/dbcommands.c +++ b/src/backend/commands/dbcommands.c @@ -3109,7 +3109,7 @@ recovery_create_dbdir(char *path, bool only_tblspc) void dbase_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in dbase records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c index ef01449678..eae2cb09b1 100644 --- a/src/backend/commands/sequence.c +++ b/src/backend/commands/sequence.c @@ -1829,7 +1829,7 @@ void seq_redo(XLogReaderState *record) { XLogRecPtr lsn = record->EndRecPtr; - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); Buffer buffer; Page page; Page localpage; diff --git a/src/backend/commands/tablespace.c b/src/backend/commands/tablespace.c index 13b0dee146..1de6803df7 100644 --- a/src/backend/commands/tablespace.c +++ b/src/backend/commands/tablespace.c @@ -1516,7 +1516,7 @@ get_tablespace_name(Oid spc_oid) void tblspc_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in tblspc records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c index 7039d425e2..a38c625cea 100644 --- a/src/backend/replication/logical/decode.c +++ b/src/backend/replication/logical/decode.c @@ -132,7 +132,7 @@ void xlog_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) { SnapBuild *builder = ctx->snapshot_builder; - uint8 info = XLogRecGetInfo(buf->record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(buf->record); ReorderBufferProcessXid(ctx->reorder, XLogRecGetXid(buf->record), buf->origptr); @@ -205,7 +205,7 @@ xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) SnapBuild *builder = ctx->snapshot_builder; ReorderBuffer *reorder = ctx->reorder; XLogReaderState *r = buf->record; - uint8 info = XLogRecGetInfo(r) & XLOG_XACT_OPMASK; + uint8 info = XLogRecGetRmgrInfo(r) & XLOG_XACT_OPMASK; /* * If the snapshot isn't yet fully built, we cannot decode anything, so @@ -225,7 +225,9 @@ xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) bool two_phase = false; xlrec = (xl_xact_commit *) XLogRecGetData(r); - ParseCommitRecord(XLogRecGetInfo(buf->record), xlrec, &parsed); + ParseCommitRecord(XLogRecGetRmgrInfo(buf->record), + xlrec, + &parsed); if (!TransactionIdIsValid(parsed.twophase_xid)) xid = XLogRecGetXid(r); @@ -253,7 +255,7 @@ xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) bool two_phase = false; xlrec = (xl_xact_abort *) XLogRecGetData(r); - ParseAbortRecord(XLogRecGetInfo(buf->record), xlrec, &parsed); + ParseAbortRecord(XLogRecGetRmgrInfo(buf->record), xlrec, &parsed); if (!TransactionIdIsValid(parsed.twophase_xid)) xid = XLogRecGetXid(r); @@ -316,7 +318,7 @@ xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) /* ok, parse it */ xlrec = (xl_xact_prepare *) XLogRecGetData(r); - ParsePrepareRecord(XLogRecGetInfo(buf->record), + ParsePrepareRecord(XLogRecGetRmgrInfo(buf->record), xlrec, &parsed); /* @@ -361,7 +363,7 @@ standby_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) { SnapBuild *builder = ctx->snapshot_builder; XLogReaderState *r = buf->record; - uint8 info = XLogRecGetInfo(r) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(r); ReorderBufferProcessXid(ctx->reorder, XLogRecGetXid(r), buf->origptr); @@ -405,7 +407,7 @@ standby_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) void heap2_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) { - uint8 info = XLogRecGetInfo(buf->record) & XLOG_HEAP_OPMASK; + uint8 info = XLogRecGetRmgrInfo(buf->record) & XLOG_HEAP_OPMASK; TransactionId xid = XLogRecGetXid(buf->record); SnapBuild *builder = ctx->snapshot_builder; @@ -464,7 +466,7 @@ heap2_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) void heap_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) { - uint8 info = XLogRecGetInfo(buf->record) & XLOG_HEAP_OPMASK; + uint8 info = XLogRecGetRmgrInfo(buf->record) & XLOG_HEAP_OPMASK; TransactionId xid = XLogRecGetXid(buf->record); SnapBuild *builder = ctx->snapshot_builder; @@ -589,7 +591,7 @@ logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) SnapBuild *builder = ctx->snapshot_builder; XLogReaderState *r = buf->record; TransactionId xid = XLogRecGetXid(r); - uint8 info = XLogRecGetInfo(r) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(r); RepOriginId origin_id = XLogRecGetOrigin(r); Snapshot snapshot = NULL; xl_logical_message *message; diff --git a/src/backend/replication/logical/message.c b/src/backend/replication/logical/message.c index c5de14afc6..c31dc1203f 100644 --- a/src/backend/replication/logical/message.c +++ b/src/backend/replication/logical/message.c @@ -80,7 +80,7 @@ LogLogicalMessage(const char *prefix, const char *message, size_t size, void logicalmsg_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info != XLOG_LOGICAL_MESSAGE) elog(PANIC, "logicalmsg_redo: unknown op code %u", info); diff --git a/src/backend/replication/logical/origin.c b/src/backend/replication/logical/origin.c index b0255ffd25..4977d2eb9a 100644 --- a/src/backend/replication/logical/origin.c +++ b/src/backend/replication/logical/origin.c @@ -826,7 +826,7 @@ StartupReplicationOrigin(void) void replorigin_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c index cc22d2e87c..44338d6de3 100644 --- a/src/backend/storage/ipc/standby.c +++ b/src/backend/storage/ipc/standby.c @@ -1160,7 +1160,7 @@ StandbyReleaseOldLocks(TransactionId oldxid) void standby_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in standby records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/backend/utils/cache/relmapper.c b/src/backend/utils/cache/relmapper.c index 26575cae6c..b825dda353 100644 --- a/src/backend/utils/cache/relmapper.c +++ b/src/backend/utils/cache/relmapper.c @@ -1084,7 +1084,7 @@ perform_relmap_update(bool shared, const RelMapFile *updates) void relmap_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in relmap records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/bin/pg_rewind/parsexlog.c b/src/bin/pg_rewind/parsexlog.c index 27782237d0..2b6b46f0e3 100644 --- a/src/bin/pg_rewind/parsexlog.c +++ b/src/bin/pg_rewind/parsexlog.c @@ -222,7 +222,7 @@ findLastCheckpoint(const char *datadir, XLogRecPtr forkptr, int tliIndex, * be the latest checkpoint before WAL forked and not the checkpoint * where the primary has been stopped to be rewound. */ - info = XLogRecGetInfo(xlogreader) & ~XLR_INFO_MASK; + info = XLogRecGetRmgrInfo(xlogreader); if (searchptr < forkptr && XLogRecGetRmid(xlogreader) == RM_XLOG_ID && (info == XLOG_CHECKPOINT_SHUTDOWN || @@ -370,7 +370,7 @@ extractPageInfo(XLogReaderState *record) int block_id; RmgrId rmid = XLogRecGetRmid(record); uint8 info = XLogRecGetInfo(record); - uint8 rminfo = info & ~XLR_INFO_MASK; + uint8 rminfo = XLogRecGetRmgrInfo(record); /* Is this a special record type that I recognize? */ @@ -440,7 +440,7 @@ extractPageInfo(XLogReaderState *record) pg_fatal("WAL record modifies a relation, but record type is not recognized: " "lsn: %X/%X, rmid: %d, rmgr: %s, info: %02X", LSN_FORMAT_ARGS(record->ReadRecPtr), - rmid, RmgrName(rmid), info); + rmid, RmgrName(rmid), info | rminfo); } for (block_id = 0; block_id <= XLogRecMaxBlockId(record); block_id++) diff --git a/src/bin/pg_waldump/pg_waldump.c b/src/bin/pg_waldump/pg_waldump.c index e8b5a6cd61..eb6a7df119 100644 --- a/src/bin/pg_waldump/pg_waldump.c +++ b/src/bin/pg_waldump/pg_waldump.c @@ -545,7 +545,7 @@ XLogDumpDisplayRecord(XLogDumpConfig *config, XLogReaderState *record) const RmgrDescData *desc = GetRmgrDesc(XLogRecGetRmid(record)); uint32 rec_len; uint32 fpi_len; - uint8 info = XLogRecGetInfo(record); + uint8 info = XLogRecGetRmgrInfo(record); XLogRecPtr xl_prev = XLogRecGetPrev(record); StringInfoData s; @@ -560,7 +560,7 @@ XLogDumpDisplayRecord(XLogDumpConfig *config, XLogReaderState *record) id = desc->rm_identify(info); if (id == NULL) - printf("desc: UNKNOWN (%x) ", info & ~XLR_INFO_MASK); + printf("desc: UNKNOWN (%x) ", info); else printf("desc: %s ", id); diff --git a/src/include/access/xact.h b/src/include/access/xact.h index 7d3b9446e6..b94b264402 100644 --- a/src/include/access/xact.h +++ b/src/include/access/xact.h @@ -519,9 +519,9 @@ extern void xact_desc(StringInfo buf, XLogReaderState *record); extern const char *xact_identify(uint8 info); /* also in xactdesc.c, so they can be shared between front/backend code */ -extern void ParseCommitRecord(uint8 info, xl_xact_commit *xlrec, xl_xact_parsed_commit *parsed); -extern void ParseAbortRecord(uint8 info, xl_xact_abort *xlrec, xl_xact_parsed_abort *parsed); -extern void ParsePrepareRecord(uint8 info, xl_xact_prepare *xlrec, xl_xact_parsed_prepare *parsed); +extern void ParseCommitRecord(uint8 rmgrinfo, xl_xact_commit *xlrec, xl_xact_parsed_commit *parsed); +extern void ParseAbortRecord(uint8 rmgrinfo, xl_xact_abort *xlrec, xl_xact_parsed_abort *parsed); +extern void ParsePrepareRecord(uint8 rmgrinfo, xl_xact_prepare *xlrec, xl_xact_parsed_prepare *parsed); extern void EnterParallelMode(void); extern void ExitParallelMode(void); diff --git a/src/include/access/xlogreader.h b/src/include/access/xlogreader.h index da32c7db77..2b976dcc03 100644 --- a/src/include/access/xlogreader.h +++ b/src/include/access/xlogreader.h @@ -407,7 +407,8 @@ extern bool DecodeXLogRecord(XLogReaderState *state, */ #define XLogRecGetTotalLen(decoder) ((decoder)->record->header.xl_tot_len) #define XLogRecGetPrev(decoder) ((decoder)->record->header.xl_prev) -#define XLogRecGetInfo(decoder) ((decoder)->record->header.xl_info) +#define XLogRecGetInfo(decoder) ((decoder)->record->header.xl_info & XLR_INFO_MASK) +#define XLogRecGetRmgrInfo(decoder) (((decoder)->record->header.xl_info) & XLR_RMGR_INFO_MASK) #define XLogRecGetRmid(decoder) ((decoder)->record->header.xl_rmid) #define XLogRecGetXid(decoder) ((decoder)->record->header.xl_xid) #define XLogRecGetOrigin(decoder) ((decoder)->record->record_origin) diff --git a/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c b/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c index a304ba54bb..b5537a5f5b 100644 --- a/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c +++ b/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c @@ -81,7 +81,7 @@ _PG_init(void) void testcustomrmgrs_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info != XLOG_TEST_CUSTOM_RMGRS_MESSAGE) elog(PANIC, "testcustomrmgrs_redo: unknown op code %u", info); @@ -91,7 +91,7 @@ void testcustomrmgrs_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_TEST_CUSTOM_RMGRS_MESSAGE) { -- 2.40.1 [application/octet-stream] v4-0001-Reduce-overhead-of-small-block-data-in-xlog-recor.patch (10.9K, ../../CAEze2WjVNdfP9sbMW=6zbgfae-UF_XVCpgkxp7BcZUcFBYa7tQ@mail.gmail.com/6-v4-0001-Reduce-overhead-of-small-block-data-in-xlog-recor.patch) download | inline diff: From a9e0f82e1669b8a5ce1219b71a874194b9fc9480 Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Tue, 24 Jan 2023 12:37:09 +0100 Subject: [PATCH v4 1/7] Reduce overhead of small block data in xlog records We reserve 2 bits in the block ID header for how large the block's registered data is: 0, 1 or 2 bytes, for 0, 1-255 and 256-UINT16_MAX bytes to the block, respectively. This reduces the minimum size of any block-referencing xlog record from 48 bytes to 46 bytes, and reduces the size of any non-hot heap update record by at least 2 bytes. It includes more infrastructure for future value-dependent encoding of length fields, even though it is currently only used in one place. --- src/backend/access/transam/xloginsert.c | 13 ++- src/backend/access/transam/xlogreader.c | 34 +++++-- src/include/access/xlog_internal.h | 114 ++++++++++++++++++++++++ src/include/access/xlogrecord.h | 43 +++++++-- 4 files changed, 188 insertions(+), 16 deletions(-) diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c index 54247e1d81..0fce063ca4 100644 --- a/src/backend/access/transam/xloginsert.c +++ b/src/backend/access/transam/xloginsert.c @@ -581,6 +581,8 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, XLogRecordBlockHeader bkpb; XLogRecordBlockImageHeader bimg; XLogRecordBlockCompressHeader cbimg = {0}; + int data_length = 0; + XLogSizeClass data_sizeclass = XLS_EMPTY; bool samerel; bool is_compressed = false; bool include_image; @@ -622,7 +624,7 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, bkpb.id = block_id; bkpb.fork_flags = regbuf->forkno; - bkpb.data_length = 0; + data_length = 0; if ((regbuf->flags & REGBUF_WILL_INIT) == REGBUF_WILL_INIT) bkpb.fork_flags |= BKPBLOCK_WILL_INIT; @@ -785,13 +787,16 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, * overall list. */ bkpb.fork_flags |= BKPBLOCK_HAS_DATA; - bkpb.data_length = (uint16) regbuf->rdata_len; + data_length = (uint16) regbuf->rdata_len; + data_sizeclass = XLogLengthToSizeClass(data_length, XLS_UINT16); total_len += regbuf->rdata_len; rdt_datas_last->next = regbuf->rdata_head; rdt_datas_last = regbuf->rdata_tail; } + bkpb.id |= data_sizeclass << XLR_BLOCKID_SZCLASS_SHIFT; + if (prev_regbuf && RelFileLocatorEquals(regbuf->rlocator, prev_regbuf->rlocator)) { samerel = true; @@ -804,6 +809,10 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, /* Ok, copy the header to the scratch buffer */ memcpy(scratch, &bkpb, SizeOfXLogRecordBlockHeader); scratch += SizeOfXLogRecordBlockHeader; + + scratch += XLogWriteLength(data_length, data_sizeclass, + XLS_UINT16, scratch); + if (include_image) { memcpy(scratch, &bimg, SizeOfXLogRecordBlockImageHeader); diff --git a/src/backend/access/transam/xlogreader.c b/src/backend/access/transam/xlogreader.c index c9f9f6e98f..11267369bf 100644 --- a/src/backend/access/transam/xlogreader.c +++ b/src/backend/access/transam/xlogreader.c @@ -1656,11 +1656,11 @@ DecodeXLogRecord(XLogReaderState *state, */ #define COPY_HEADER_FIELD(_dst, _size) \ do { \ - if (remaining < _size) \ + if (remaining < (_size)) \ goto shortdata_err; \ - memcpy(_dst, ptr, _size); \ - ptr += _size; \ - remaining -= _size; \ + memcpy((_dst), ptr, (_size)); \ + ptr += (_size); \ + remaining -= (_size); \ } while(0) char *ptr; @@ -1686,8 +1686,13 @@ DecodeXLogRecord(XLogReaderState *state, datatotal = 0; while (remaining > datatotal) { + XLogSizeClass sizeClass; COPY_HEADER_FIELD(&block_id, sizeof(uint8)); + sizeClass = (block_id & XLR_BLOCKID_SZCLASS_MASK) >> + XLR_BLOCKID_SZCLASS_SHIFT; + block_id &= XLR_BLOCK_ID_MASK; + if (block_id == XLR_BLOCK_ID_DATA_SHORT) { /* XLogRecordDataHeaderShort */ @@ -1751,7 +1756,26 @@ DecodeXLogRecord(XLogReaderState *state, blk->prefetch_buffer = InvalidBuffer; - COPY_HEADER_FIELD(&blk->data_len, sizeof(uint16)); + { + Size read; + uint32 length = 0; + read = XLogReadLength(&length, sizeClass, + XLS_UINT16, + ptr, remaining); + + if (read < 0) + { + report_invalid_record(state, + "Could not read length from record at %X/%X", + LSN_FORMAT_ARGS(state->ReadRecPtr)); + goto err; + } + + ptr += read; + remaining -= read; + blk->data_len = length; + } + /* cross-check that the HAS_DATA flag is set iff data_length > 0 */ if (blk->has_data && blk->data_len == 0) { diff --git a/src/include/access/xlog_internal.h b/src/include/access/xlog_internal.h index b0fd338a00..42b3c66547 100644 --- a/src/include/access/xlog_internal.h +++ b/src/include/access/xlog_internal.h @@ -328,6 +328,120 @@ typedef enum struct LogicalDecodingContext; struct XLogRecordBuffer; +/* + * XLogSizeClass interactions + */ + +/* + * XLogLengthToSizeClass + * Turns a length into a size class + * MaxSizeClass is used to allow the compiler to eliminate branches that + * can never be hit in the caller's code path; e.g. when processing uint16. + */ +static inline XLogSizeClass XLogLengthToSizeClass(uint32 length, + const XLogSizeClass maxSizeClass) +{ + XLogSizeClass sizeClass; + + if (length == 0) + sizeClass = XLS_EMPTY; + else if (length <= UINT8_MAX && maxSizeClass >= XLS_UINT8) + sizeClass = XLS_UINT8; + else if (length <= UINT16_MAX && maxSizeClass >= XLS_UINT16) + sizeClass = XLS_UINT16; + else + { + Assert(maxSizeClass == XLS_UINT32); + sizeClass = XLS_UINT32; + } + + Assert(sizeClass <= maxSizeClass); + + return sizeClass; +} + +/* + * XLogWriteLength + * + * Write a length with size determined by sizeClass into the output. + * Returns the Size of bytes written. The user is responsible for making + * sure that out-of-bounds write is impossible. + * Writes at most 4 bytes. + */ +static inline Size XLogWriteLength(uint32 length, XLogSizeClass sizeClass, + const XLogSizeClass maxSizeClass, + char *output) +{ + Size written = -1; + + Assert(sizeClass <= maxSizeClass && + XLogLengthToSizeClass(length, maxSizeClass) == sizeClass); + +#define WRITE_OP(caseSizeClass, field_type) \ + case caseSizeClass: \ + if ((caseSizeClass) <= maxSizeClass) \ + { \ + field_type typedLength = length; \ + memcpy(output, &typedLength, sizeof(field_type)); \ + written = sizeof(field_type); \ + } \ + break + + switch (sizeClass) { + case XLS_EMPTY: + written = 0; + break; + WRITE_OP(XLS_UINT8, uint8); + WRITE_OP(XLS_UINT16, uint16); + WRITE_OP(XLS_UINT32, uint32); + } + +#undef WRITE_OP + return written; +} + +/* + * XLogReadLength + * + * Read a length with size determined by sizeClass from the input into + * *length. Returns the Size of bytes read, or -1 if the input size was + * too small to read the relevant length field. + */ +static inline Size XLogReadLength(uint32 *length, XLogSizeClass sizeClass, + const XLogSizeClass maxSizeClass, + char *input, Size inSize) +{ + Size readSize = -1; + + Assert(sizeClass <= maxSizeClass); + +#define READ_OP(caseSizeClass, field_type) \ + case caseSizeClass: \ + if ((caseSizeClass) <= maxSizeClass) \ + { \ + field_type typedLength; \ + if (inSize < sizeof(field_type)) \ + return -1; \ + memcpy(&typedLength, input, sizeof(field_type)); \ + readSize = sizeof(field_type); \ + *length = typedLength; \ + } \ + break + + switch (sizeClass) { + case XLS_EMPTY: + readSize = 0; + *length = 0; + break; + READ_OP(XLS_UINT8, uint8); + READ_OP(XLS_UINT16, uint16); + READ_OP(XLS_UINT32, uint32); + } + +#undef READ_OP + return readSize; +} + /* * Method table for resource managers. * diff --git a/src/include/access/xlogrecord.h b/src/include/access/xlogrecord.h index f355e08e1d..b0aada1031 100644 --- a/src/include/access/xlogrecord.h +++ b/src/include/access/xlogrecord.h @@ -17,6 +17,27 @@ #include "storage/block.h" #include "storage/relfilelocator.h" +/* + * XLogSizeClass + * + * XLog data segments registered to e.g. blocks are often quite small. + * This XLogSizeClass infrastructure allows us to do variable length encoding + * on these fields, so that we may save some bytes in each record on length + * fields. + * + * Note that it is possible to encode smaller sizes in the larger size classes, + * but the reverse is not true. For maximum efficiency, it is important to only + * use + * + * Values > UINT32_MAX are not supported in this scheme. + */ +typedef enum XLogSizeClass { + XLS_EMPTY = 0, /* length == 0 */ + XLS_UINT8 = 1, /* length <= UINT8_MAX; stored in uint8 (1B) */ + XLS_UINT16 = 2, /* length <= UINT16_MAX; stored in uint16 (2B) */ + XLS_UINT32 = 3 /* length <= UINT32_MAX; stored in uint32 (4B) */ +} XLogSizeClass; + /* * The overall layout of an XLOG record is: * Fixed-size header (XLogRecord struct) @@ -104,15 +125,14 @@ typedef struct XLogRecordBlockHeader { uint8 id; /* block reference ID */ uint8 fork_flags; /* fork within the relation, and flags */ - uint16 data_length; /* number of payload bytes (not including page - * image) */ + /* Depending on XLR_BLOCKID_SZCLASS; 0, 1 or 2 bytes of data_length follow */ /* If BKPBLOCK_HAS_IMAGE, an XLogRecordBlockImageHeader struct follows */ /* If BKPBLOCK_SAME_REL is not set, a RelFileLocator follows */ /* BlockNumber follows */ } XLogRecordBlockHeader; -#define SizeOfXLogRecordBlockHeader (offsetof(XLogRecordBlockHeader, data_length) + sizeof(uint16)) +#define SizeOfXLogRecordBlockHeader (offsetof(XLogRecordBlockHeader, fork_flags) + sizeof(uint8)) /* * Additional header information when a full-page image is included @@ -226,6 +246,11 @@ typedef struct XLogRecordDataHeaderLong #define SizeOfXLogRecordDataHeaderLong (sizeof(uint8) + sizeof(uint32)) +#define XLR_BLOCK_ID_MASK 0x3F + +#define XLR_BLOCKID_SZCLASS_SHIFT 6 +#define XLR_BLOCKID_SZCLASS_MASK (3 << XLR_BLOCKID_SZCLASS_SHIFT) + /* * Block IDs used to distinguish different kinds of record fragments. Block * references are numbered from 0 to XLR_MAX_BLOCK_ID. A rmgr is free to use @@ -234,15 +259,15 @@ typedef struct XLogRecordDataHeaderLong * numbers are reserved to denote the "main" data portion of the record, * as well as replication-supporting transaction metadata. * - * The maximum is currently set at 32, quite arbitrarily. Most records only + * The maximum is currently set at 0x20 (32), quite arbitrarily. Most records only * need a handful of block references, but there are a few exceptions that * need more. */ -#define XLR_MAX_BLOCK_ID 32 +#define XLR_MAX_BLOCK_ID 0x20 -#define XLR_BLOCK_ID_DATA_SHORT 255 -#define XLR_BLOCK_ID_DATA_LONG 254 -#define XLR_BLOCK_ID_ORIGIN 253 -#define XLR_BLOCK_ID_TOPLEVEL_XID 252 +#define XLR_BLOCK_ID_DATA_SHORT 0x3F +#define XLR_BLOCK_ID_DATA_LONG 0x3E +#define XLR_BLOCK_ID_ORIGIN 0x3D +#define XLR_BLOCK_ID_TOPLEVEL_XID 0x3C #endif /* XLOGRECORD_H */ -- 2.40.1 [application/octet-stream] v4-0006-Add-RM_INVALID-an-invalid-resource-manager.patch (3.7K, ../../CAEze2WjVNdfP9sbMW=6zbgfae-UF_XVCpgkxp7BcZUcFBYa7tQ@mail.gmail.com/7-v4-0006-Add-RM_INVALID-an-invalid-resource-manager.patch) download | inline diff: From 1a17ef50cc4df58347a86c4b4a1a25c236565260 Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Wed, 12 Jul 2023 14:08:36 +0200 Subject: [PATCH v4 6/7] Add RM_INVALID, an invalid resource manager The resource manager has ID = 0, thus requiring some special handling in other code. Apart from being generally useful, it is used in future patches to detect the end of wal in lieu of a zero-ed fixed-size xl_tot_len field. --- src/backend/access/transam/xlogreader.c | 13 +++++++++++++ src/bin/pg_waldump/pg_waldump.c | 4 ++-- src/include/access/rmgr.h | 2 +- src/include/access/rmgrlist.h | 1 + src/include/access/xlog_internal.h | 2 +- 5 files changed, 18 insertions(+), 4 deletions(-) diff --git a/src/backend/access/transam/xlogreader.c b/src/backend/access/transam/xlogreader.c index 47bfb2266c..9f8687ae45 100644 --- a/src/backend/access/transam/xlogreader.c +++ b/src/backend/access/transam/xlogreader.c @@ -670,6 +670,19 @@ restart: else { /* XXX: more validation should be done here */ + + /* + * RM_INVALID_ID can be an indication of end-of-WAL, or other signs of + * corruption - that byte should be non-0. + */ + if (record->xl_rmid == RM_INVALID_ID) + { + report_invalid_record(state, + "invalid resource manager id %u at %X/%X", + (uint32) record->xl_rmid, + LSN_FORMAT_ARGS(RecPtr)); + goto err; + } if (total_len < SizeOfXLogRecord) { report_invalid_record(state, diff --git a/src/bin/pg_waldump/pg_waldump.c b/src/bin/pg_waldump/pg_waldump.c index cb16e72f58..b8034d8212 100644 --- a/src/bin/pg_waldump/pg_waldump.c +++ b/src/bin/pg_waldump/pg_waldump.c @@ -99,7 +99,7 @@ print_rmgr_list(void) { int i; - for (i = 0; i <= RM_MAX_BUILTIN_ID; i++) + for (i = RM_INVALID_ID + 1; i <= RM_MAX_BUILTIN_ID; i++) { printf("%s\n", GetRmgrDesc(i)->rm_name); } @@ -963,7 +963,7 @@ main(int argc, char **argv) else { /* then look for builtin rmgrs */ - for (rmid = 0; rmid <= RM_MAX_BUILTIN_ID; rmid++) + for (rmid = RM_INVALID_ID + 1; rmid <= RM_MAX_BUILTIN_ID; rmid++) { if (pg_strcasecmp(optarg, GetRmgrDesc(rmid)->rm_name) == 0) { diff --git a/src/include/access/rmgr.h b/src/include/access/rmgr.h index 3b6a497e1b..7038ba0833 100644 --- a/src/include/access/rmgr.h +++ b/src/include/access/rmgr.h @@ -41,7 +41,7 @@ typedef enum RmgrIds static inline bool RmgrIdIsBuiltin(int rmid) { - return rmid <= RM_MAX_BUILTIN_ID; + return rmid <= RM_MAX_BUILTIN_ID && rmid != RM_INVALID_ID; } static inline bool diff --git a/src/include/access/rmgrlist.h b/src/include/access/rmgrlist.h index 463bcb67c5..b53d731f56 100644 --- a/src/include/access/rmgrlist.h +++ b/src/include/access/rmgrlist.h @@ -25,6 +25,7 @@ */ /* symbol name, textual name, redo, desc, identify, startup, cleanup, mask, decode */ +PG_RMGR(RM_INVALID_ID, "invalid", NULL, NULL, NULL, NULL, NULL, NULL, NULL) PG_RMGR(RM_XLOG_ID, "XLOG", xlog_redo, xlog_desc, xlog_identify, NULL, NULL, NULL, xlog_decode) PG_RMGR(RM_XACT_ID, "Transaction", xact_redo, xact_desc, xact_identify, NULL, NULL, NULL, xact_decode) PG_RMGR(RM_SMGR_ID, "Storage", smgr_redo, smgr_desc, smgr_identify, NULL, NULL, NULL, NULL) diff --git a/src/include/access/xlog_internal.h b/src/include/access/xlog_internal.h index 876e2790f9..8b3c9b9598 100644 --- a/src/include/access/xlog_internal.h +++ b/src/include/access/xlog_internal.h @@ -31,7 +31,7 @@ /* * Each page of XLOG file has a header like this: */ -#define XLOG_PAGE_MAGIC 0xD113 /* can be used as WAL version indicator */ +#define XLOG_PAGE_MAGIC 0xD114 /* can be used as WAL version indicator */ typedef struct XLogPageHeaderData { -- 2.40.1 [application/octet-stream] v4-0007-Reformat-the-XLog-record-header.patch (42.6K, ../../CAEze2WjVNdfP9sbMW=6zbgfae-UF_XVCpgkxp7BcZUcFBYa7tQ@mail.gmail.com/8-v4-0007-Reformat-the-XLog-record-header.patch) download | inline diff: From e887248dc62f76510267472b195834452ce299e5 Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Mon, 19 Jun 2023 17:46:45 +0200 Subject: [PATCH v4 7/7] Reformat the XLog record header Many WAL records don't utilize all bytes of the XLog header, resulting in a lot of wasted bytes. This new XLog header format saves bytes in several ways: - Records that don't need the XID won't include it This potentially saves 4 bytes /record - Records that are small enough don't need 4 bytes to describe their length By varint encoding the length, we save up to 3 bytes /record (up to 4 bytes /record if dataless records are considered) - Only include xl_rmgrinfo if it is non-0 Several RMGRs have a record that they use most often. We save 1 byte/record if that record id is 0. - Lose some alignment losses We save 1 byte/record by removing one byte of alignment losses --- src/backend/access/transam/xlog.c | 133 ++++++++++++++------- src/backend/access/transam/xloginsert.c | 135 ++++++++++++++++------ src/backend/access/transam/xlogreader.c | 105 +++++++++-------- src/backend/access/transam/xlogrecovery.c | 4 +- src/bin/pg_resetwal/pg_resetwal.c | 38 +++--- src/include/access/xlog_internal.h | 100 +++++++++++++++- src/include/access/xlogreader.h | 4 +- src/include/access/xlogrecord.h | 125 +++++++++++++++++--- 8 files changed, 486 insertions(+), 158 deletions(-) diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 4b1ff0d1aa..5937b43f50 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -737,17 +737,18 @@ XLogInsertRecord(XLogRecData *rdata, XLogCtlInsert *Insert = &XLogCtl->Insert; pg_crc32c rdata_crc; bool inserted; - XLogRecord *rechdr = (XLogRecord *) rdata->data; - uint8 rmgrinfo = rechdr->xl_rmgrinfo; - bool isLogSwitch = (rechdr->xl_rmid == RM_XLOG_ID && - rmgrinfo == XLOG_SWITCH); + XLogRecHdr rechdr; + bool isLogSwitch; XLogRecPtr StartPos; XLogRecPtr EndPos; + XLogRecPtr xl_prev; bool prevDoPageWrites = doPageWrites; TimeLineID insertTLI; + int rec_payload_len, + rec_hdr_len; - /* we assume that all of the record header is in the first chunk */ - Assert(rdata->len >= SizeOfXLogRecord); + /* we assume that all data of the record header is in the first chunk */ + Assert(rdata->len >= XLogRecordMinHdrSize); /* cross-check on whether we should be here or not */ if (!XLogInsertAllowed()) @@ -758,6 +759,17 @@ XLogInsertRecord(XLogRecData *rdata, * change, so we can read it without a lock. */ insertTLI = XLogCtl->InsertTimeLineID; + rechdr = (XLogRecHdr) rdata->data; + + { + XLogRecord record = {0}; + XLogReadRecHdrInto(rechdr, (Size) rdata->len, &record); + + isLogSwitch = (record.xl_rmid == RM_XLOG_ID && + record.xl_rmgrinfo == XLOG_SWITCH); + rec_hdr_len = (int) XLogRecordHdrLen(rechdr->xl_info); + rec_payload_len = (int) record.xl_payload_len; + } /*---------- * @@ -834,34 +846,44 @@ XLogInsertRecord(XLogRecData *rdata, } /* - * Reserve space for the record in the WAL. This also sets the xl_prev - * pointer. + * Reserve space for the record in the WAL. */ if (isLogSwitch) - inserted = ReserveXLogSwitch(&StartPos, &EndPos, &rechdr->xl_prev); + inserted = ReserveXLogSwitch(&StartPos, &EndPos, &xl_prev); else { - ReserveXLogInsertLocation(rechdr->xl_tot_len, &StartPos, &EndPos, - &rechdr->xl_prev); + ReserveXLogInsertLocation(rec_hdr_len + rec_payload_len, + &StartPos, &EndPos, &xl_prev); inserted = true; } if (inserted) { + char *rec = (char *) rechdr; + + /* fill in xl_prev */ + memcpy(rec + rec_hdr_len - sizeof(pg_crc32c) - sizeof(XLogRecPtr), + &xl_prev, sizeof(XLogRecPtr)); + /* * Now that xl_prev has been filled in, calculate CRC of the record * header. */ - rdata_crc = rechdr->xl_crc; - COMP_CRC32C(rdata_crc, rechdr, offsetof(XLogRecord, xl_crc)); + + memcpy(&rdata_crc, rec + rec_hdr_len - sizeof(pg_crc32c), + sizeof(pg_crc32c)); + + COMP_CRC32C(rdata_crc, rec, rec_hdr_len - sizeof(pg_crc32c)); FIN_CRC32C(rdata_crc); - rechdr->xl_crc = rdata_crc; + + memcpy(rec + rec_hdr_len - sizeof(pg_crc32c), + &rdata_crc, sizeof(pg_crc32c)); /* * All the record data, including the header, is now ready to be * inserted. Copy the record in the space reserved. */ - CopyXLogRecordToWAL(rechdr->xl_tot_len, isLogSwitch, rdata, + CopyXLogRecordToWAL(rec_hdr_len + rec_payload_len, isLogSwitch, rdata, StartPos, EndPos, insertTLI); /* @@ -932,7 +954,8 @@ XLogInsertRecord(XLogRecData *rdata, */ if (inserted) { - EndPos = StartPos + SizeOfXLogRecord; + /* xlog switch is minimal record header, plus a byte for rmgrinfo */ + EndPos = StartPos + XLogRecordMinHdrSize + sizeof(uint8); if (StartPos / XLOG_BLCKSZ != EndPos / XLOG_BLCKSZ) { uint64 offset = XLogSegmentOffset(EndPos, wal_segment_size); @@ -949,7 +972,7 @@ XLogInsertRecord(XLogRecData *rdata, if (XLOG_DEBUG) { static XLogReaderState *debug_reader = NULL; - XLogRecord *record; + XLogRecHdr record; DecodedXLogRecord *decoded; StringInfoData buf; StringInfoData recordBuf; @@ -971,9 +994,9 @@ XLogInsertRecord(XLogRecData *rdata, appendBinaryStringInfo(&recordBuf, rdata->data, rdata->len); /* We also need temporary space to decode the record. */ - record = (XLogRecord *) recordBuf.data; + record = (XLogRecHdr) recordBuf.data; decoded = (DecodedXLogRecord *) - palloc(DecodeXLogRecordRequiredSpace(record->xl_tot_len)); + palloc(DecodeXLogRecordRequiredSpace(recordBuf.len)); if (!debug_reader) debug_reader = XLogReaderAllocate(wal_segment_size, NULL, @@ -1018,7 +1041,7 @@ XLogInsertRecord(XLogRecData *rdata, /* Report WAL traffic to the instrumentation. */ if (inserted) { - pgWalUsage.wal_bytes += rechdr->xl_tot_len; + pgWalUsage.wal_bytes += rec_hdr_len + rec_payload_len; pgWalUsage.wal_records++; pgWalUsage.wal_fpi += num_fpi; } @@ -1049,10 +1072,10 @@ ReserveXLogInsertLocation(int size, XLogRecPtr *StartPos, XLogRecPtr *EndPos, uint64 endbytepos; uint64 prevbytepos; - size = MAXALIGN(size); + size = XLP_ALIGN(size); /* All (non xlog-switch) records should contain data. */ - Assert(size > SizeOfXLogRecord); + Assert(size > XLogRecordMinHdrSize); /* * The duration the spinlock needs to be held is minimized by minimizing @@ -1103,7 +1126,7 @@ ReserveXLogSwitch(XLogRecPtr *StartPos, XLogRecPtr *EndPos, XLogRecPtr *PrevPtr) uint64 startbytepos; uint64 endbytepos; uint64 prevbytepos; - uint32 size = MAXALIGN(SizeOfXLogRecord); + uint32 size = XLP_ALIGN(XLogRecordMinHdrSize + sizeof(uint8)); XLogRecPtr ptr; uint32 segleft; @@ -1176,10 +1199,10 @@ CopyXLogRecordToWAL(int write_len, bool isLogSwitch, XLogRecData *rdata, freespace = INSERT_FREESPACE(CurrPos); /* - * there should be enough space for at least the first field (xl_tot_len) + * there should be enough space for at least the first XL_ALIGN quantum * on this page. */ - Assert(freespace >= sizeof(uint32)); + Assert(freespace >= XLP_ALIGN(1)); /* Copy record data */ written = 0; @@ -1247,7 +1270,7 @@ CopyXLogRecordToWAL(int write_len, bool isLogSwitch, XLogRecData *rdata, if (isLogSwitch && XLogSegmentOffset(CurrPos, wal_segment_size) != 0) { /* An xlog-switch record doesn't contain any data besides the header */ - Assert(write_len == SizeOfXLogRecord); + Assert(write_len == XLogRecordMinHdrSize + sizeof(uint8)); /* Assert that we did reserve the right amount of space */ Assert(XLogSegmentOffset(EndPos, wal_segment_size) == 0); @@ -1291,7 +1314,7 @@ CopyXLogRecordToWAL(int write_len, bool isLogSwitch, XLogRecData *rdata, else { /* Align the end position, so that the next record starts aligned */ - CurrPos = MAXALIGN64(CurrPos); + CurrPos = XLP_ALIGN(CurrPos); } if (CurrPos != EndPos) @@ -4649,11 +4672,15 @@ BootStrapXLOG(void) char *buffer; XLogPageHeader page; XLogLongPageHeader longpage; - XLogRecord *record; + XLogRecHdr rec_hdr; + char *baserecptr; char *recptr; uint64 sysidentifier; struct timeval tv; pg_crc32c crc; + const int rec_hdr_len = offsetof(XLogRecHdrData, xl_hdrdata) + + sizeof(uint8) + sizeof(XLogRecPtr) + sizeof(pg_crc32c); + const int rec_payload_len = sizeof(uint8) * 2 + sizeof(CheckPoint); /* allow ordinary WAL segment creation, like StartupXLOG() would */ SetInstallXLogFileSegmentActive(); @@ -4724,28 +4751,52 @@ BootStrapXLOG(void) longpage->xlp_seg_size = wal_segment_size; longpage->xlp_xlog_blcksz = XLOG_BLCKSZ; + /* if this changes, we need to update the code below */ + Assert(XLogLengthToSizeClass(SizeOfXLogRecordDataHeaderShort + sizeof(checkPoint), XLS_UINT32) == XLS_UINT8); + /* if this changes, we need to add XLR_HAS_RMGRINFO */ + Assert(XLOG_CHECKPOINT_SHUTDOWN == 0); + /* Insert the initial checkpoint record */ - recptr = ((char *) page + SizeOfXLogLongPHD); - record = (XLogRecord *) recptr; - record->xl_prev = 0; - record->xl_xid = InvalidTransactionId; - record->xl_tot_len = SizeOfXLogRecord + SizeOfXLogRecordDataHeaderShort + sizeof(checkPoint); - record->xl_info = 0; - record->xl_rmgrinfo = XLOG_CHECKPOINT_SHUTDOWN; - record->xl_rmid = RM_XLOG_ID; - recptr += SizeOfXLogRecord; + baserecptr = recptr = (((char *) page) + SizeOfXLogLongPHD); + + rec_hdr = (XLogRecHdr) baserecptr; + rec_hdr->xl_info = (char) XLS_UINT8; + rec_hdr->xl_rmid = (char) RM_XLOG_ID; + recptr += offsetof(XLogRecHdrData, xl_hdrdata); + + recptr += XLogWriteLength(SizeOfXLogRecordDataHeaderShort + sizeof(checkPoint), + XLS_UINT8, XLS_UINT8, recptr); + + /* include prevptr */ + { + XLogRecPtr prevptr = 0; + memcpy(recptr, &prevptr, sizeof(XLogRecPtr)); + recptr += sizeof(XLogRecPtr); + } + + /* reserve location of crc */ + recptr += sizeof(pg_crc32c); + + Assert(recptr - baserecptr == XLogRecordHdrLen(rec_hdr->xl_info)); + Assert(recptr - baserecptr == rec_hdr_len); + /* fill the XLogRecordDataHeaderShort struct */ *(recptr++) = (char) XLR_BLOCK_ID_DATA_SHORT; *(recptr++) = sizeof(checkPoint); memcpy(recptr, &checkPoint, sizeof(checkPoint)); recptr += sizeof(checkPoint); - Assert(recptr - (char *) record == record->xl_tot_len); + + /* Assert length of record matches expectations */ + Assert(recptr - baserecptr == XLogRecordTotalLength(rec_hdr)); + Assert((rec_hdr)->xl_info == XLS_UINT8); INIT_CRC32C(crc); - COMP_CRC32C(crc, ((char *) record) + SizeOfXLogRecord, record->xl_tot_len - SizeOfXLogRecord); - COMP_CRC32C(crc, (char *) record, offsetof(XLogRecord, xl_crc)); + COMP_CRC32C(crc, baserecptr + rec_hdr_len, rec_payload_len); + COMP_CRC32C(crc, baserecptr, rec_hdr_len - sizeof(pg_crc32c)); FIN_CRC32C(crc); - record->xl_crc = crc; + + memcpy(baserecptr + rec_hdr_len - sizeof(pg_crc32c), + &crc, sizeof(pg_crc32c)); /* Create first XLOG segment file */ openLogTLI = BootstrapTimeLineID; diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c index 4816c5284b..194196f596 100644 --- a/src/backend/access/transam/xloginsert.c +++ b/src/backend/access/transam/xloginsert.c @@ -545,12 +545,16 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP XLogRecPtr *fpw_lsn, int *num_fpi, bool *topxid_included) { XLogRecData *rdt; - uint64 total_len = 0; + uint64 payload_len = 0; + XLogSizeClass payload_sizeclass = XLS_EMPTY; int block_id; pg_crc32c rdata_crc; registered_buffer *prev_regbuf = NULL; XLogRecData *rdt_datas_last; - XLogRecord *rechdr; + TransactionId xid; + XLogRecHdr rechdr; + uint8 xlr_flags = 0; + uint32 rec_hdr_len; char *scratch = hdr_scratch; /* @@ -558,9 +562,10 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP * All the modifications we do to the rdata chains below must handle that. */ - /* The record begins with the fixed-size header */ - rechdr = (XLogRecord *) scratch; - scratch += SizeOfXLogRecord; + /* The record begins with the variable-size header */ + rechdr = (XLogRecHdr) scratch; + + scratch += XLogRecordMaxHdrSize; hdr_rdt.next = NULL; rdt_datas_last = &hdr_rdt; @@ -779,7 +784,7 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP } } - total_len += bimg.length; + payload_len += bimg.length; } if (needs_data) @@ -797,7 +802,7 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP bkpb.fork_flags |= BKPBLOCK_HAS_DATA; data_length = (uint16) regbuf->rdata_len; data_sizeclass = XLogLengthToSizeClass(data_length, XLS_UINT16); - total_len += regbuf->rdata_len; + payload_len += regbuf->rdata_len; rdt_datas_last->next = regbuf->rdata_head; rdt_datas_last = regbuf->rdata_tail; @@ -851,16 +856,33 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP } /* followed by toplevel XID, if not already included in previous record */ - if (curinsert_flags & XLOG_INCLUDE_XID && IsSubxactTopXidLogPending()) + if (curinsert_flags & XLOG_INCLUDE_XID) { - TransactionId xid = GetTopTransactionIdIfAny(); + xid = GetCurrentTransactionIdIfAny(); + + if (IsSubxactTopXidLogPending()) + { + TransactionId txid = GetTopTransactionIdIfAny(); + + xlr_flags |= XLR_HAS_XID; + Assert(TransactionIdIsValid(xid)); - /* Set the flag that the top xid is included in the WAL */ - *topxid_included = true; + /* Set the flag that the top xid is included in the WAL */ + *topxid_included = true; - *(scratch++) = (char) XLR_BLOCK_ID_TOPLEVEL_XID; - memcpy(scratch, &xid, sizeof(TransactionId)); - scratch += sizeof(TransactionId); + *(scratch++) = (char) XLR_BLOCK_ID_TOPLEVEL_XID; + memcpy(scratch, &txid, sizeof(TransactionId)); + scratch += sizeof(TransactionId); + } + else if (TransactionIdIsValid(xid)) + { + xlr_flags |= XLR_HAS_XID; + } + } + else + { + xid = InvalidTransactionId; + Assert((xlr_flags & XLR_HAS_XID) == 0); } /* followed by main data, if any */ @@ -889,12 +911,64 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP } rdt_datas_last->next = mainrdata_head; rdt_datas_last = mainrdata_last; - total_len += mainrdata_len; + payload_len += mainrdata_len; } rdt_datas_last->next = NULL; - hdr_rdt.len = (scratch - hdr_scratch); - total_len += hdr_rdt.len; + /* Add the block headers section's length to the payload */ + payload_len += scratch - (hdr_scratch + XLogRecordMaxHdrSize); + + /* + * Fill in the fields in the record header. Prev-link is filled in later, + * once we know where in the WAL the record will be inserted. The CRC does + * not include the record header yet. + */ + payload_sizeclass = XLogLengthToSizeClass(payload_len, XLS_UINT32); + + xlr_flags |= payload_sizeclass; + + if (rmgr_info != 0) + xlr_flags |= XLR_HAS_RMGRINFO; + + /* Set up the xlog header. and xl_rmgr */ + rechdr->xl_info = xlr_flags; + rechdr->xl_rmid = rmid; + + rec_hdr_len = 0; + + /* next, xl_payload_len */ + rec_hdr_len += XLogWriteLength(payload_len, payload_sizeclass, + XLS_UINT32, + &rechdr->xl_hdrdata[rec_hdr_len]); + + if (xlr_flags & XLR_HAS_RMGRINFO) + rechdr->xl_hdrdata[rec_hdr_len++] = (char) rmgr_info; + + if (xlr_flags & XLR_HAS_XID) + { + Assert(curinsert_flags & XLOG_INCLUDE_XID); + Assert(TransactionIdIsValid(xid)); + + memcpy(&rechdr->xl_hdrdata[rec_hdr_len], &xid, sizeof(TransactionId)); + rec_hdr_len += sizeof(TransactionId); + } + + /* reserve space for XLogRecPtr and checksum */ + rec_hdr_len += sizeof(XLogRecPtr); + rec_hdr_len += sizeof(pg_crc32c); + /* Add static header length */ + rec_hdr_len += offsetof(XLogRecHdrData, xl_hdrdata); + + Assert(rec_hdr_len == XLogRecordHdrLen(rechdr->xl_info)); + + /* move the record to be placed the rest of the payload */ + memmove(hdr_scratch + XLogRecordMaxHdrSize - rec_hdr_len, + hdr_scratch, rec_hdr_len); + + rechdr = (XLogRecHdr) (hdr_scratch + XLogRecordMaxHdrSize - rec_hdr_len); + + hdr_rdt.data = (char *) rechdr; + hdr_rdt.len = (scratch - hdr_rdt.data); /* * Calculate CRC of the data @@ -905,10 +979,14 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP * header. */ INIT_CRC32C(rdata_crc); - COMP_CRC32C(rdata_crc, hdr_scratch + SizeOfXLogRecord, hdr_rdt.len - SizeOfXLogRecord); + COMP_CRC32C(rdata_crc, hdr_scratch + XLogRecordMaxHdrSize, + hdr_rdt.len - rec_hdr_len); for (rdt = hdr_rdt.next; rdt != NULL; rdt = rdt->next) COMP_CRC32C(rdata_crc, rdt->data, rdt->len); + memcpy(hdr_rdt.data + rec_hdr_len - sizeof(pg_crc32c), + &rdata_crc, sizeof(pg_crc32c)); + /* * Ensure that the XLogRecord is not too large. * @@ -917,28 +995,11 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP * not emit records larger than the sizes advertised to be supported. This * cap is based on DecodeXLogRecordRequiredSpace(). */ - if (total_len >= XLogRecordMaxSize) + if (payload_len + rec_hdr_len >= XLogRecordMaxSize) ereport(ERROR, (errmsg_internal("oversized WAL record"), errdetail_internal("WAL record would be %llu bytes (of maximum %u bytes); rmid %u flags %u.", - (unsigned long long) total_len, XLogRecordMaxSize, rmid, info))); - - /* - * Fill in the fields in the record header. Prev-link is filled in later, - * once we know where in the WAL the record will be inserted. The CRC does - * not include the record header yet. - */ - if (curinsert_flags & XLOG_INCLUDE_XID) - rechdr->xl_xid = GetCurrentTransactionIdIfAny(); - else - rechdr->xl_xid = InvalidTransactionId; - - rechdr->xl_tot_len = (uint32) total_len; - rechdr->xl_info = info; - rechdr->xl_rmid = rmid; - rechdr->xl_rmgrinfo = rmgr_info; - rechdr->xl_prev = InvalidXLogRecPtr; - rechdr->xl_crc = rdata_crc; + (unsigned long long) payload_len, XLogRecordMaxSize, rmid, info))); return &hdr_rdt; } diff --git a/src/backend/access/transam/xlogreader.c b/src/backend/access/transam/xlogreader.c index 9f8687ae45..3f0143642d 100644 --- a/src/backend/access/transam/xlogreader.c +++ b/src/backend/access/transam/xlogreader.c @@ -49,8 +49,9 @@ static int ReadPageInternal(XLogReaderState *state, XLogRecPtr pageptr, static void XLogReaderInvalReadState(XLogReaderState *state); static XLogPageReadResult XLogDecodeNextRecord(XLogReaderState *state, bool nonblocking); static bool ValidXLogRecordHeader(XLogReaderState *state, XLogRecPtr RecPtr, - XLogRecPtr PrevRecPtr, XLogRecord *record, bool randAccess); -static bool ValidXLogRecord(XLogReaderState *state, XLogRecord *record, + XLogRecPtr PrevRecPtr, XLogRecHdr rechdr, + bool randAccess); +static bool ValidXLogRecord(XLogReaderState *state, XLogRecHdr record, XLogRecPtr recptr); static void ResetDecoder(XLogReaderState *state); static void WALOpenSegmentInit(WALOpenSegment *seg, WALSegmentContext *segcxt, @@ -535,7 +536,7 @@ static XLogPageReadResult XLogDecodeNextRecord(XLogReaderState *state, bool nonblocking) { XLogRecPtr RecPtr; - XLogRecord *record; + XLogRecHdr record; XLogRecPtr targetPagePtr; bool randAccess; uint32 len, @@ -602,7 +603,7 @@ restart: * fits on the same page. */ readOff = ReadPageInternal(state, targetPagePtr, - Min(targetRecOff + SizeOfXLogRecord, XLOG_BLCKSZ)); + Min(targetRecOff + XLogRecordMaxHdrSize, XLOG_BLCKSZ)); if (readOff == XLREAD_WOULDBLOCK) return XLREAD_WOULDBLOCK; else if (readOff < 0) @@ -644,13 +645,13 @@ restart: * Read the record length. * * NB: Even though we use an XLogRecord pointer here, the whole record - * header might not fit on this page. xl_tot_len is the first field of the - * struct, so it must be on this page (the records are MAXALIGNed), but we - * cannot access any other fields until we've verified that we got the - * whole header. + * header might not fit on this page. xl_payload_len is the third field of + * the data and starts at the third byte, so it must be on this page (the + * records are aligned to 8 bytes), but we cannot access most other fields + * until we've verified that we got the whole header. */ - record = (XLogRecord *) (state->readBuf + RecPtr % XLOG_BLCKSZ); - total_len = record->xl_tot_len; + record = (XLogRecHdr) (state->readBuf + (RecPtr % XLOG_BLCKSZ)); + total_len = XLogRecordTotalLength(record); /* * If the whole record header is on this page, validate it immediately. @@ -660,7 +661,7 @@ restart: * record" code path below; otherwise we might fail to apply * ValidXLogRecordHeader at all. */ - if (targetRecOff <= XLOG_BLCKSZ - SizeOfXLogRecord) + if (targetRecOff <= XLOG_BLCKSZ - XLogRecordHdrLen(record->xl_info)) { if (!ValidXLogRecordHeader(state, RecPtr, state->DecodeRecPtr, record, randAccess)) @@ -683,12 +684,12 @@ restart: LSN_FORMAT_ARGS(RecPtr)); goto err; } - if (total_len < SizeOfXLogRecord) + if (total_len < XLogRecordMinHdrSize) { report_invalid_record(state, "invalid record length at %X/%X: expected at least %u, got %u", LSN_FORMAT_ARGS(RecPtr), - (uint32) SizeOfXLogRecord, total_len); + (uint32) XLogRecordMinHdrSize, total_len); goto err; } gotheader = false; @@ -831,7 +832,7 @@ restart: /* If we just reassembled the record header, validate it. */ if (!gotheader) { - record = (XLogRecord *) state->readRecordBuf; + record = (XLogRecHdr) state->readRecordBuf; if (!ValidXLogRecordHeader(state, RecPtr, state->DecodeRecPtr, record, randAccess)) goto err; @@ -841,14 +842,14 @@ restart: Assert(gotheader); - record = (XLogRecord *) state->readRecordBuf; + record = (XLogRecHdr) state->readRecordBuf; if (!ValidXLogRecord(state, record, RecPtr)) goto err; pageHeaderSize = XLogPageHeaderSize((XLogPageHeader) state->readBuf); state->DecodeRecPtr = RecPtr; state->NextRecPtr = targetPagePtr + pageHeaderSize - + MAXALIGN(pageHeader->xlp_rem_len); + + XLP_ALIGN(pageHeader->xlp_rem_len); } else { @@ -864,7 +865,7 @@ restart: if (!ValidXLogRecord(state, record, RecPtr)) goto err; - state->NextRecPtr = RecPtr + MAXALIGN(total_len); + state->NextRecPtr = RecPtr + XLP_ALIGN(total_len); state->DecodeRecPtr = RecPtr; } @@ -873,7 +874,7 @@ restart: * Special processing if it's an XLOG SWITCH record */ if (record->xl_rmid == RM_XLOG_ID && - record->xl_rmgrinfo == XLOG_SWITCH) + XLRHdrGetRmgrInfo(record) == XLOG_SWITCH) { /* Pretend it extends to end of segment */ state->NextRecPtr += state->segcxt.ws_segsize - 1; @@ -892,7 +893,7 @@ restart: if (!decoded->oversized) { /* The new decode buffer head must be MAXALIGNed. */ - Assert(decoded->size == MAXALIGN(decoded->size)); + Assert(decoded->size == XLP_ALIGN(decoded->size)); if ((char *) decoded == state->decode_buffer) state->decode_buffer_tail = state->decode_buffer + decoded->size; else @@ -1123,22 +1124,25 @@ XLogReaderInvalReadState(XLogReaderState *state) */ static bool ValidXLogRecordHeader(XLogReaderState *state, XLogRecPtr RecPtr, - XLogRecPtr PrevRecPtr, XLogRecord *record, + XLogRecPtr PrevRecPtr, XLogRecHdr rechdr, bool randAccess) { - if (record->xl_tot_len < SizeOfXLogRecord) + XLogRecord record = {0}; + XLogReadRecHdrInto(rechdr, XLogRecordHdrLen(rechdr->xl_info), &record); + + if (XLogRecordTotalLength(rechdr) < XLogRecordMinHdrSize) { report_invalid_record(state, "invalid record length at %X/%X: expected at least %u, got %u", LSN_FORMAT_ARGS(RecPtr), - (uint32) SizeOfXLogRecord, record->xl_tot_len); + (uint32) XLogRecordMinHdrSize, record.xl_payload_len); return false; } - if (!RmgrIdIsValid(record->xl_rmid)) + if (!RmgrIdIsValid(rechdr->xl_rmid)) { report_invalid_record(state, "invalid resource manager ID %u at %X/%X", - record->xl_rmid, LSN_FORMAT_ARGS(RecPtr)); + rechdr->xl_rmid, LSN_FORMAT_ARGS(RecPtr)); return false; } if (randAccess) @@ -1147,12 +1151,14 @@ ValidXLogRecordHeader(XLogReaderState *state, XLogRecPtr RecPtr, * We can't exactly verify the prev-link, but surely it should be less * than the record's own address. */ - if (!(record->xl_prev < RecPtr)) + if (!(record.xl_prev < RecPtr)) { report_invalid_record(state, - "record with incorrect prev-link %X/%X at %X/%X", - LSN_FORMAT_ARGS(record->xl_prev), - LSN_FORMAT_ARGS(RecPtr)); + "record with incorrect prev-link %X/%X at %X/%X, info %u, rmgr %u", + LSN_FORMAT_ARGS(record.xl_prev), + LSN_FORMAT_ARGS(RecPtr), + (uint32) record.xl_info, + (uint32) record.xl_rmid); return false; } } @@ -1163,12 +1169,13 @@ ValidXLogRecordHeader(XLogReaderState *state, XLogRecPtr RecPtr, * check guards against torn WAL pages where a stale but valid-looking * WAL record starts on a sector boundary. */ - if (record->xl_prev != PrevRecPtr) + if (record.xl_prev != PrevRecPtr) { report_invalid_record(state, - "record with incorrect prev-link %X/%X at %X/%X", - LSN_FORMAT_ARGS(record->xl_prev), - LSN_FORMAT_ARGS(RecPtr)); + "record with incorrect prev-link %X/%X at %X/%X, expected %X/%X", + LSN_FORMAT_ARGS(record.xl_prev), + LSN_FORMAT_ARGS(RecPtr), + LSN_FORMAT_ARGS(PrevRecPtr)); return false; } } @@ -1188,19 +1195,26 @@ ValidXLogRecordHeader(XLogReaderState *state, XLogRecPtr RecPtr, * SizeOfXLogRecord. */ static bool -ValidXLogRecord(XLogReaderState *state, XLogRecord *record, XLogRecPtr recptr) +ValidXLogRecord(XLogReaderState *state, XLogRecHdr record, XLogRecPtr recptr) { - pg_crc32c crc; + pg_crc32c crc, + hdr_crc; + Size hdr_len = XLogRecordHdrLen(record->xl_info); + Size rec_length = XLogRecordTotalLength(record); /* Calculate the CRC */ INIT_CRC32C(crc); - COMP_CRC32C(crc, ((char *) record) + SizeOfXLogRecord, record->xl_tot_len - SizeOfXLogRecord); + COMP_CRC32C(crc, ((char *) record) + hdr_len, rec_length - hdr_len); /* include the record header last */ - COMP_CRC32C(crc, (char *) record, offsetof(XLogRecord, xl_crc)); + COMP_CRC32C(crc, (char *) record, hdr_len - sizeof(pg_crc32c)); FIN_CRC32C(crc); - if (!EQ_CRC32C(record->xl_crc, crc)) + memcpy(&hdr_crc, ((char *) record) + hdr_len - sizeof(pg_crc32c), + sizeof(pg_crc32c)); + + if (!EQ_CRC32C(hdr_crc, crc)) { + Assert(false); report_invalid_record(state, "incorrect resource manager data checksum in record at %X/%X", LSN_FORMAT_ARGS(recptr)); @@ -1440,7 +1454,7 @@ XLogFindNextRecord(XLogReaderState *state, XLogRecPtr RecPtr) * * Note that record headers are MAXALIGN'ed */ - if (MAXALIGN(header->xlp_rem_len) >= (XLOG_BLCKSZ - pageHeaderSize)) + if (XLP_ALIGN(header->xlp_rem_len) >= (XLOG_BLCKSZ - pageHeaderSize)) tmpRecPtr = targetPagePtr + XLOG_BLCKSZ; else { @@ -1449,7 +1463,7 @@ XLogFindNextRecord(XLogReaderState *state, XLogRecPtr RecPtr) * tmpRecPtr to point to the first valid record */ tmpRecPtr = targetPagePtr + pageHeaderSize - + MAXALIGN(header->xlp_rem_len); + + XLP_ALIGN(header->xlp_rem_len); break; } } @@ -1660,7 +1674,7 @@ DecodeXLogRecordRequiredSpace(size_t xl_tot_len) bool DecodeXLogRecord(XLogReaderState *state, DecodedXLogRecord *decoded, - XLogRecord *record, + XLogRecHdr record, XLogRecPtr lsn, char **errormsg) { @@ -1683,7 +1697,7 @@ DecodeXLogRecord(XLogReaderState *state, RelFileLocator *rlocator = NULL; uint8 block_id; - decoded->header = *record; + XLogReadRecHdrInto(record, XLogRecordHdrLen(record->xl_info), &decoded->header); decoded->lsn = lsn; decoded->next = NULL; decoded->record_origin = InvalidRepOriginId; @@ -1692,8 +1706,8 @@ DecodeXLogRecord(XLogReaderState *state, decoded->main_data_len = 0; decoded->max_block_id = -1; ptr = (char *) record; - ptr += SizeOfXLogRecord; - remaining = record->xl_tot_len - SizeOfXLogRecord; + ptr += XLogRecordHdrLen(record->xl_info); + remaining = decoded->header.xl_payload_len; /* Decode the headers */ datatotal = 0; @@ -1968,8 +1982,8 @@ DecodeXLogRecord(XLogReaderState *state, } /* Report the actual size we used. */ - decoded->size = MAXALIGN(out - (char *) decoded); - Assert(DecodeXLogRecordRequiredSpace(record->xl_tot_len) >= + decoded->size = XLP_ALIGN(out - (char *) decoded); + Assert(DecodeXLogRecordRequiredSpace(decoded->header.xl_payload_len) >= decoded->size); return true; @@ -1998,6 +2012,7 @@ XLogRecGetBlockTag(XLogReaderState *record, uint8 block_id, if (!XLogRecGetBlockTagExtended(record, block_id, rlocator, forknum, blknum, NULL)) { + Assert(false); #ifndef FRONTEND elog(ERROR, "could not locate backup block with ID %d in WAL record", block_id); diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index 56fa2f74a2..0362ba747f 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -4000,10 +4000,10 @@ ReadCheckpointRecord(XLogPrefetcher *xlogprefetcher, XLogRecPtr RecPtr, (errmsg("invalid xl_rmgrinfo in checkpoint record"))); return NULL; } - if (record->xl_tot_len != SizeOfXLogRecord + SizeOfXLogRecordDataHeaderShort + sizeof(CheckPoint)) + if (record->xl_payload_len != SizeOfXLogRecordDataHeaderShort + sizeof(CheckPoint)) { ereport(LOG, - (errmsg("invalid length of checkpoint record"))); + (errmsg("invalid payload length of checkpoint record"))); return NULL; } return record; diff --git a/src/bin/pg_resetwal/pg_resetwal.c b/src/bin/pg_resetwal/pg_resetwal.c index 80cb9a5bc4..4e171eed1f 100644 --- a/src/bin/pg_resetwal/pg_resetwal.c +++ b/src/bin/pg_resetwal/pg_resetwal.c @@ -1040,12 +1040,15 @@ WriteEmptyXLOG(void) PGAlignedXLogBlock buffer; XLogPageHeader page; XLogLongPageHeader longpage; - XLogRecord *record; pg_crc32c crc; char path[MAXPGPATH]; int fd; int nbytes; char *recptr; + char *baserecptr; + const int rec_hdr_len = offsetof(XLogRecHdrData, xl_hdrdata) + + sizeof(uint8) + sizeof(XLogRecPtr) + sizeof(pg_crc32c); + const int rec_payload_len = sizeof(uint8) * 2 + sizeof(CheckPoint); memset(buffer.data, 0, XLOG_BLCKSZ); @@ -1061,26 +1064,33 @@ WriteEmptyXLOG(void) longpage->xlp_xlog_blcksz = XLOG_BLCKSZ; /* Insert the initial checkpoint record */ - recptr = (char *) page + SizeOfXLogLongPHD; - record = (XLogRecord *) recptr; - record->xl_prev = 0; - record->xl_xid = InvalidTransactionId; - record->xl_tot_len = SizeOfXLogRecord + SizeOfXLogRecordDataHeaderShort + sizeof(CheckPoint); - record->xl_info = 0; - record->xl_rmgrinfo = XLOG_CHECKPOINT_SHUTDOWN; - record->xl_rmid = RM_XLOG_ID; - - recptr += SizeOfXLogRecord; + baserecptr = recptr = (char *) page + SizeOfXLogLongPHD; + *(recptr++) = (char) XLS_UINT8; + *(recptr++) = (char) RM_XLOG_ID; + recptr += XLogWriteLength(SizeOfXLogRecordDataHeaderShort + sizeof(CheckPoint), + XLS_UINT8, XLS_UINT8, recptr); + + /* include prevptr */ + { + XLogRecPtr prevptr = 0; + memcpy(recptr, &prevptr, sizeof(XLogRecPtr)); + recptr += sizeof(XLogRecPtr); + } + /* reserve location of crc */ + recptr += sizeof(pg_crc32c); + *(recptr++) = (char) XLR_BLOCK_ID_DATA_SHORT; *(recptr++) = sizeof(CheckPoint); memcpy(recptr, &ControlFile.checkPointCopy, sizeof(CheckPoint)); INIT_CRC32C(crc); - COMP_CRC32C(crc, ((char *) record) + SizeOfXLogRecord, record->xl_tot_len - SizeOfXLogRecord); - COMP_CRC32C(crc, (char *) record, offsetof(XLogRecord, xl_crc)); + COMP_CRC32C(crc, baserecptr + rec_hdr_len, rec_payload_len); + COMP_CRC32C(crc, baserecptr, rec_hdr_len - sizeof(pg_crc32c)); FIN_CRC32C(crc); - record->xl_crc = crc; + + memcpy(baserecptr + rec_hdr_len - sizeof(pg_crc32c), + &crc, sizeof(pg_crc32c)); /* Write the first page */ XLogFilePath(path, ControlFile.checkPointCopy.ThisTimeLineID, diff --git a/src/include/access/xlog_internal.h b/src/include/access/xlog_internal.h index 8b3c9b9598..ea32a9ba1b 100644 --- a/src/include/access/xlog_internal.h +++ b/src/include/access/xlog_internal.h @@ -26,7 +26,15 @@ #include "pgtime.h" #include "storage/block.h" #include "storage/relfilelocator.h" +#include "transam.h" +/* + * WAL records (and other XLogPage page-level content) are aligned to 8 bytes. + * + * Note that the contents of the records is not aligned (!) + */ +#define ALIGNOF_XLP_CONTENT 8 +#define XLP_ALIGN(LEN) TYPEALIGN64(ALIGNOF_XLP_CONTENT, LEN) /* * Each page of XLOG file has a header like this: @@ -49,7 +57,7 @@ typedef struct XLogPageHeaderData uint32 xlp_rem_len; /* total len of remaining data for record */ } XLogPageHeaderData; -#define SizeOfXLogShortPHD MAXALIGN(sizeof(XLogPageHeaderData)) +#define SizeOfXLogShortPHD XLP_ALIGN(sizeof(XLogPageHeaderData)) typedef XLogPageHeaderData *XLogPageHeader; @@ -66,7 +74,7 @@ typedef struct XLogLongPageHeaderData uint32 xlp_xlog_blcksz; /* just as a cross-check */ } XLogLongPageHeaderData; -#define SizeOfXLogLongPHD MAXALIGN(sizeof(XLogLongPageHeaderData)) +#define SizeOfXLogLongPHD XLP_ALIGN(sizeof(XLogLongPageHeaderData)) typedef XLogLongPageHeaderData *XLogLongPageHeader; @@ -381,7 +389,7 @@ static inline Size XLogWriteLength(uint32 length, XLogSizeClass sizeClass, case caseSizeClass: \ if ((caseSizeClass) <= maxSizeClass) \ { \ - field_type typedLength = length; \ + field_type typedLength = (field_type) length; \ memcpy(output, &typedLength, sizeof(field_type)); \ written = sizeof(field_type); \ } \ @@ -394,6 +402,9 @@ static inline Size XLogWriteLength(uint32 length, XLogSizeClass sizeClass, WRITE_OP(XLS_UINT8, uint8); WRITE_OP(XLS_UINT16, uint16); WRITE_OP(XLS_UINT32, uint32); + default: + Assert(false); + pg_unreachable(); } #undef WRITE_OP @@ -436,12 +447,95 @@ static inline Size XLogReadLength(uint32 *length, XLogSizeClass sizeClass, READ_OP(XLS_UINT8, uint8); READ_OP(XLS_UINT16, uint16); READ_OP(XLS_UINT32, uint32); + default: + Assert(false); + pg_unreachable(); } #undef READ_OP return readSize; } + +inline static uint8 XLRHdrGetRmgrInfo(XLogRecHdr record) +{ + XLogSizeClass recSizeClass; + int offset; + + if (!(record->xl_info & XLR_HAS_RMGRINFO)) + return 0; + + recSizeClass = XLR_SIZECLASS(record->xl_info); + /* xl_rmgrinfo is located immediately behind the xl_payload_len field */ + offset = XLogSizeClassToByteLength(recSizeClass); + + return (uint8) record->xl_hdrdata[offset]; +} + + +/* Works on any partial record */ +inline static Size XLogRecordTotalLength(XLogRecHdr record) +{ + uint8 xl_info = record->xl_info; + XLogSizeClass sizeClass; + uint32 length = 0; + sizeClass = XLR_SIZECLASS(xl_info); + + XLogReadLength(&length, sizeClass, XLS_UINT32, + &record->xl_hdrdata[0], 6); + + return (Size) length + XLogRecordHdrLen(xl_info); +} + +inline static void XLogReadRecHdrInto(XLogRecHdr recdata, Size length, + XLogRecord *record) +{ + Size offset = 0; + Size hdr_size PG_USED_FOR_ASSERTS_ONLY = 0; + XLogSizeClass sizeClass; + + Assert(length >= XLogRecordMinHdrSize); + + record->xl_info = recdata->xl_info; + + hdr_size = XLogRecordHdrLen(record->xl_info); + Assert(length >= hdr_size); + + record->xl_rmid = recdata->xl_rmid; + + sizeClass = XLR_SIZECLASS(record->xl_info); + offset += XLogReadLength(&record->xl_payload_len, sizeClass, + XLS_UINT32, &recdata->xl_hdrdata[offset], length - offset); + + if (record->xl_info & XLR_HAS_RMGRINFO) + { + record->xl_rmgrinfo = recdata->xl_hdrdata[offset]; + offset += sizeof(uint8); + } + else + { + record->xl_rmgrinfo = 0; + } + + if (record->xl_info & XLR_HAS_XID) + { + memcpy(&record->xl_xid, &recdata->xl_hdrdata[offset], sizeof(TransactionId)); + offset += sizeof(TransactionId); + } + else + { + record->xl_xid = InvalidTransactionId; + } + + memcpy(&record->xl_prev, &recdata->xl_hdrdata[offset], sizeof(XLogRecPtr)); + offset += sizeof(XLogRecPtr); + + memcpy(&record->xl_crc, &recdata->xl_hdrdata[offset], sizeof(pg_crc32c)); + offset += sizeof(pg_crc32c); + + Assert(hdr_size - 2 == offset); +} + /* * Method table for resource managers. * diff --git a/src/include/access/xlogreader.h b/src/include/access/xlogreader.h index a1d0216404..bb64f89dba 100644 --- a/src/include/access/xlogreader.h +++ b/src/include/access/xlogreader.h @@ -397,7 +397,7 @@ extern bool WALRead(XLogReaderState *state, extern size_t DecodeXLogRecordRequiredSpace(size_t xl_tot_len); extern bool DecodeXLogRecord(XLogReaderState *state, DecodedXLogRecord *decoded, - XLogRecord *record, + XLogRecHdr record, XLogRecPtr lsn, char **errormsg); @@ -405,7 +405,7 @@ extern bool DecodeXLogRecord(XLogReaderState *state, * Macros that provide access to parts of the record most recently returned by * XLogReadRecord() or XLogNextRecord(). */ -#define XLogRecGetTotalLen(decoder) ((decoder)->record->header.xl_tot_len) +#define XLogRecGetTotalLen(decoder) ((decoder)->record->header.xl_payload_len) #define XLogRecGetPrev(decoder) ((decoder)->record->header.xl_prev) #define XLogRecGetInfo(decoder) ((decoder)->record->header.xl_info) #define XLogRecGetRmgrInfo(decoder) ((decoder)->record->header.xl_rmgrinfo) diff --git a/src/include/access/xlogrecord.h b/src/include/access/xlogrecord.h index 42b06f163e..af615e2866 100644 --- a/src/include/access/xlogrecord.h +++ b/src/include/access/xlogrecord.h @@ -38,9 +38,38 @@ typedef enum XLogSizeClass { XLS_UINT32 = 3 /* length <= UINT32_MAX; stored in uint32 (4B) */ } XLogSizeClass; +static inline int XLogSizeClassToByteLength(XLogSizeClass sz) +{ + switch (sz) { + case XLS_EMPTY: + return 0; + case XLS_UINT8: + return sizeof(uint8); + case XLS_UINT16: + return sizeof(uint16); + case XLS_UINT32: + return sizeof(uint32); + default: + pg_unreachable(); + } +} + +typedef struct XLogRecord +{ + uint8 xl_info; /* flag bits, see below */ + RmgrId xl_rmid; /* resource manager for this record */ + uint32 xl_payload_len; /* total len of entire record */ + uint8 xl_rmgrinfo; /* rmgr flag bits, see below */ + TransactionId xl_xid; /* xact id */ + XLogRecPtr xl_prev; /* ptr to previous record in log */ + pg_crc32c xl_crc; /* CRC for this record */ +} XLogRecord; + +#define SizeOfXLogRecord (offsetof(XLogRecord, xl_crc) + sizeof(pg_crc32c)) + /* * The overall layout of an XLOG record is: - * Fixed-size header (XLogRecord struct) + * Variable-size header (containing the data of the XLogRecord struct) * XLogRecordBlockHeader struct * XLogRecordBlockHeader struct * ... @@ -50,6 +79,15 @@ typedef enum XLogSizeClass { * ... * main data * + * Different xlog record headers need to store different header fields, so + * depending on flags in the xl_info field the layout may change. + * + * Note that for records with a payload larger than 0xFFFFFFF, the size field + * will be 4 bytes long, which can be larger than the minimum MAXALIGN quantum. + * To still be able to read the record correctly, we always align WAL to 8 + * bytes, so that the length bytes are always present in the first aligned + * quantum of data. + * * There can be zero or more XLogRecordBlockHeaders, and 0 or more bytes of * rmgr-specific data not associated with a block. XLogRecord structs * always start on MAXALIGN boundaries in the WAL files, but the rest of @@ -59,22 +97,25 @@ typedef enum XLogSizeClass { * XLogRecordDataHeaderLong structs all begin with a single 'id' byte. It's * used to distinguish between block references, and the main data structs. */ -typedef struct XLogRecord +typedef struct XLogRecHdrData { - uint32 xl_tot_len; /* total len of entire record */ - TransactionId xl_xid; /* xact id */ - XLogRecPtr xl_prev; /* ptr to previous record in log */ uint8 xl_info; /* flag bits, see below */ RmgrId xl_rmid; /* resource manager for this record */ - uint8 xl_rmgrinfo; /* rmgr flag bits, see below */ - /* 1 byte of padding here, initialize to zero */ - pg_crc32c xl_crc; /* CRC for this record */ - - /* XLogRecordBlockHeaders and XLogRecordDataHeader follow, no padding */ + char xl_hdrdata[FLEXIBLE_ARRAY_MEMBER]; /* variable length record data */ -} XLogRecord; + /* + * Payload is as follows, in order: + * XLogRecord data encoded + * xl_payload_len; 0, 1, 2 or 4 bytes + * xl_rmgrinfo; 0 or 1 byte + * xl_xid; 0 or 4 bytes + * xl_prev; 8 bytes + * xl_prev follows; 8 bytes + * record payload, if present. + */ +} XLogRecHdrData; -#define SizeOfXLogRecord (offsetof(XLogRecord, xl_crc) + sizeof(pg_crc32c)) +typedef XLogRecHdrData *XLogRecHdr; /* * XLogReader needs to allocate all the data of a WAL record in a single @@ -87,13 +128,28 @@ typedef struct XLogRecord */ #define XLogRecordMaxSize (1020 * 1024 * 1024) +#define XLR_SIZECLASS_MASK 0x03 +#define XLR_SIZECLASS(xl_info) ((XLogSizeClass) ((xl_info) & XLR_SIZECLASS_MASK)) + +/* + * There are several rmgrs which don't use info bits, so we omit that + * byte whenever possible. + */ +#define XLR_HAS_RMGRINFO 0x04 + +/* + * If a WAL record uses the current transaction ID, that will be included + * in the record header. In all other cases we omit the XID to save bytes. + */ +#define XLR_HAS_XID 0x08 + /* * If a WAL record modifies any relation files, in ways not covered by the * usual block references, this flag is set. This is not used for anything * by PostgreSQL itself, but it allows external tools that read WAL and keep * track of modified blocks to recognize such special record types. */ -#define XLR_SPECIAL_REL_UPDATE 0x01 +#define XLR_SPECIAL_REL_UPDATE 0x10 /* * Enforces consistency checks of replayed WAL at recovery. If enabled, @@ -102,7 +158,48 @@ typedef struct XLogRecord * of XLogInsert can use this value if necessary, but if * wal_consistency_checking is enabled for a rmgr this is set unconditionally. */ -#define XLR_CHECK_CONSISTENCY 0x02 +#define XLR_CHECK_CONSISTENCY (0x20) + +#define XLogRecordMaxHdrSize ( \ + sizeof(uint8) /* xl_info */ + \ + sizeof(uint8) /* xl_rmid */ + \ + sizeof(uint32) /* xl_payload_len */ + \ + sizeof(uint8) /* xl_rmgrinfo */ + \ + sizeof(TransactionId) /* xl_xid */ + \ + sizeof(XLogRecPtr) /* xl_prev */ + \ + sizeof(pg_crc32c) /* xl_crc */ \ +) + +#define XLogRecordMinHdrSize ( \ + sizeof(uint8) /* xl_info */ + \ + sizeof(uint8) /* xl_rmid */ + \ + 0 /* xl_payload_len */ + \ + 0 /* xl_rmgrinfo */ + \ + 0 /* xl_xid */ + \ + sizeof(XLogRecPtr) /* xl_prev */ + \ + sizeof(pg_crc32c) /* xl_crc */ \ +) + +static inline Size XLogRecordHdrLen(uint8 info) +{ + Size size = 0; + /* xl_info */ + size += sizeof(uint8); + /* xl_rmid */ + size += sizeof(uint8); + /* xl_payload_len */ + size += XLogSizeClassToByteLength(XLR_SIZECLASS(info)); + /* xl_rmgrinfo */ + size += ((info) & XLR_HAS_RMGRINFO) ? sizeof(uint8) : 0; + /* xl_xid */ + size += ((info) & XLR_HAS_XID) ? sizeof(TransactionId) : 0; + /* xl_prev */ + size += sizeof(XLogRecPtr); + /* xl_crc */ + size += sizeof(pg_crc32c); + + return size; +} /* * Header info for block data appended to an XLOG record. -- 2.40.1 ^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: XLog size reductions: Reduced XLog record header size for PG17 @ 2023-09-19 10:07 Matthias van de Meent <[email protected]> parent: Matthias van de Meent <[email protected]> 0 siblings, 0 replies; 35+ messages in thread From: Matthias van de Meent @ 2023-09-19 10:07 UTC (permalink / raw) To: ; +Cc: pgsql-hackers; Heikki Linnakangas <[email protected]>; Robert Haas <[email protected]>; Dilip Kumar <[email protected]>; Andres Freund <[email protected]> On Wed, 12 Jul 2023 at 14:50, Matthias van de Meent <[email protected]> wrote: > > On Mon, 3 Jul 2023 at 13:08, Matthias van de Meent > <[email protected]> wrote: > > > > On Fri, 30 Jun 2023 at 17:36, Matthias van de Meent > > <[email protected]> wrote: > > > > > > Hi, > > > > > > The attached v2 patchset contains some small fixes for the failing > > > cfbot 32-bit tests - at least locally it does so. > > > > > > I'd overlooked one remaining use of MAXALIGN64() in xlog.c in the last > > > patch of the set, which has now been updated to XLP_ALIGN as well. > > > Additionally, XLP_ALIGN has been updated to use TYPEALIGN64 instead of > > > TYPEALIGN so that we don't lose bits of the aligned value in 32-bit > > > systems. > > > > Apparently there was some usage of MAXALIGN() in xlogreader that I'd > > missed, and which only shows up in TAP tests. In v3 I've fixed that, > > together with some improved early detection of invalid record headers. > > Another fix for CFBot - pg_waldump tests which were added in 96063e28 > exposed an issue in my patchset related to RM_INVALID_ID. > > v4 splits former patch 0006 into two: now 0006 adds RM_INVALID and > does the rmgr-related changes in the code, and 0007 does the WAL disk > format overhaul. V5 is a rebased version of v4, and includes the latest patch from "smaller XLRec block header" [0] as 0001. Kind regards, Matthias van de Meent [0] https://www.postgresql.org/message-id/CAEze2WhG_qvs0_HPCKyGLjFSSeiLZJcFhT%3DrzEUd7AzyxnSfKw%40mail.g... Attachments: [application/octet-stream] v5-0003-Rename-rmgr_identify-info-bits-argument-to-rmgrin.patch (21.6K, ../../CAEze2WgQ9d_aa6tGVDqXgdy2owvrrJtY6TaCZQ6r+BA7F1fSCA@mail.gmail.com/2-v5-0003-Rename-rmgr_identify-info-bits-argument-to-rmgrin.patch) download | inline diff: From 401745de6b7ea23796346dc1c0fd0d0f7caf6467 Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Mon, 23 Jan 2023 17:31:18 +0100 Subject: [PATCH v5 3/7] Rename rmgr_identify 'info'-bits argument to rmgrinfo This indicates the provided bits are fully owned by the rmgr, and none of the xlog record's owned bits are shared with the rmgr anymore. --- src/backend/access/rmgrdesc/brindesc.c | 4 ++-- src/backend/access/rmgrdesc/clogdesc.c | 4 ++-- src/backend/access/rmgrdesc/dbasedesc.c | 4 ++-- src/backend/access/rmgrdesc/gindesc.c | 4 ++-- src/backend/access/rmgrdesc/gistdesc.c | 4 ++-- src/backend/access/rmgrdesc/hashdesc.c | 4 ++-- src/backend/access/rmgrdesc/heapdesc.c | 8 ++++---- src/backend/access/rmgrdesc/logicalmsgdesc.c | 4 ++-- src/backend/access/rmgrdesc/mxactdesc.c | 4 ++-- src/backend/access/rmgrdesc/nbtdesc.c | 4 ++-- src/backend/access/rmgrdesc/relmapdesc.c | 4 ++-- src/backend/access/rmgrdesc/seqdesc.c | 4 ++-- src/backend/access/rmgrdesc/smgrdesc.c | 4 ++-- src/backend/access/rmgrdesc/spgdesc.c | 4 ++-- src/backend/access/rmgrdesc/standbydesc.c | 4 ++-- src/backend/access/rmgrdesc/tblspcdesc.c | 4 ++-- src/backend/access/rmgrdesc/xlogdesc.c | 4 ++-- src/include/access/brin_xlog.h | 2 +- src/include/access/clog.h | 2 +- src/include/access/ginxlog.h | 2 +- src/include/access/gistxlog.h | 2 +- src/include/access/hash_xlog.h | 2 +- src/include/access/heapam_xlog.h | 4 ++-- src/include/access/multixact.h | 2 +- src/include/access/nbtxlog.h | 2 +- src/include/access/spgxlog.h | 2 +- src/include/access/xlog.h | 2 +- src/include/catalog/storage_xlog.h | 2 +- src/include/commands/dbcommands_xlog.h | 2 +- src/include/commands/sequence.h | 2 +- src/include/commands/tablespace.h | 2 +- src/include/replication/message.h | 2 +- src/include/storage/standbydefs.h | 2 +- src/include/utils/relmapper.h | 2 +- src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c | 4 ++-- 35 files changed, 56 insertions(+), 56 deletions(-) diff --git a/src/backend/access/rmgrdesc/brindesc.c b/src/backend/access/rmgrdesc/brindesc.c index 1d3e5ee03e..fa272f0720 100644 --- a/src/backend/access/rmgrdesc/brindesc.c +++ b/src/backend/access/rmgrdesc/brindesc.c @@ -71,11 +71,11 @@ brin_desc(StringInfo buf, XLogReaderState *record) } const char * -brin_identify(uint8 info) +brin_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_BRIN_CREATE_INDEX: id = "CREATE_INDEX"; diff --git a/src/backend/access/rmgrdesc/clogdesc.c b/src/backend/access/rmgrdesc/clogdesc.c index 915f9945d0..b368af595e 100644 --- a/src/backend/access/rmgrdesc/clogdesc.c +++ b/src/backend/access/rmgrdesc/clogdesc.c @@ -41,11 +41,11 @@ clog_desc(StringInfo buf, XLogReaderState *record) } const char * -clog_identify(uint8 info) +clog_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case CLOG_ZEROPAGE: id = "ZEROPAGE"; diff --git a/src/backend/access/rmgrdesc/dbasedesc.c b/src/backend/access/rmgrdesc/dbasedesc.c index a9529bde05..a405c0738a 100644 --- a/src/backend/access/rmgrdesc/dbasedesc.c +++ b/src/backend/access/rmgrdesc/dbasedesc.c @@ -54,11 +54,11 @@ dbase_desc(StringInfo buf, XLogReaderState *record) } const char * -dbase_identify(uint8 info) +dbase_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_DBASE_CREATE_FILE_COPY: id = "CREATE_FILE_COPY"; diff --git a/src/backend/access/rmgrdesc/gindesc.c b/src/backend/access/rmgrdesc/gindesc.c index 08fc7ba49b..e7ed7a6094 100644 --- a/src/backend/access/rmgrdesc/gindesc.c +++ b/src/backend/access/rmgrdesc/gindesc.c @@ -179,11 +179,11 @@ gin_desc(StringInfo buf, XLogReaderState *record) } const char * -gin_identify(uint8 info) +gin_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_GIN_CREATE_PTREE: id = "CREATE_PTREE"; diff --git a/src/backend/access/rmgrdesc/gistdesc.c b/src/backend/access/rmgrdesc/gistdesc.c index 6473069c1a..899e12bcc3 100644 --- a/src/backend/access/rmgrdesc/gistdesc.c +++ b/src/backend/access/rmgrdesc/gistdesc.c @@ -86,11 +86,11 @@ gist_desc(StringInfo buf, XLogReaderState *record) } const char * -gist_identify(uint8 info) +gist_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_GIST_PAGE_UPDATE: id = "PAGE_UPDATE"; diff --git a/src/backend/access/rmgrdesc/hashdesc.c b/src/backend/access/rmgrdesc/hashdesc.c index bcb2013048..451251caca 100644 --- a/src/backend/access/rmgrdesc/hashdesc.c +++ b/src/backend/access/rmgrdesc/hashdesc.c @@ -122,11 +122,11 @@ hash_desc(StringInfo buf, XLogReaderState *record) } const char * -hash_identify(uint8 info) +hash_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_HASH_INIT_META_PAGE: id = "INIT_META_PAGE"; diff --git a/src/backend/access/rmgrdesc/heapdesc.c b/src/backend/access/rmgrdesc/heapdesc.c index 15f943051a..5224869af1 100644 --- a/src/backend/access/rmgrdesc/heapdesc.c +++ b/src/backend/access/rmgrdesc/heapdesc.c @@ -302,11 +302,11 @@ heap2_desc(StringInfo buf, XLogReaderState *record) } const char * -heap_identify(uint8 info) +heap_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_HEAP_INSERT: id = "INSERT"; @@ -347,11 +347,11 @@ heap_identify(uint8 info) } const char * -heap2_identify(uint8 info) +heap2_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_HEAP2_PRUNE: id = "PRUNE"; diff --git a/src/backend/access/rmgrdesc/logicalmsgdesc.c b/src/backend/access/rmgrdesc/logicalmsgdesc.c index 2bb6cb06c3..7566630961 100644 --- a/src/backend/access/rmgrdesc/logicalmsgdesc.c +++ b/src/backend/access/rmgrdesc/logicalmsgdesc.c @@ -43,9 +43,9 @@ logicalmsg_desc(StringInfo buf, XLogReaderState *record) } const char * -logicalmsg_identify(uint8 info) +logicalmsg_identify(uint8 rmgrinfo) { - if ((info & ~XLR_INFO_MASK) == XLOG_LOGICAL_MESSAGE) + if (rmgrinfo == XLOG_LOGICAL_MESSAGE) return "MESSAGE"; return NULL; diff --git a/src/backend/access/rmgrdesc/mxactdesc.c b/src/backend/access/rmgrdesc/mxactdesc.c index cdcf8a4f45..7676778239 100644 --- a/src/backend/access/rmgrdesc/mxactdesc.c +++ b/src/backend/access/rmgrdesc/mxactdesc.c @@ -81,11 +81,11 @@ multixact_desc(StringInfo buf, XLogReaderState *record) } const char * -multixact_identify(uint8 info) +multixact_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_MULTIXACT_ZERO_OFF_PAGE: id = "ZERO_OFF_PAGE"; diff --git a/src/backend/access/rmgrdesc/nbtdesc.c b/src/backend/access/rmgrdesc/nbtdesc.c index ede8df72c0..ff9e28ca55 100644 --- a/src/backend/access/rmgrdesc/nbtdesc.c +++ b/src/backend/access/rmgrdesc/nbtdesc.c @@ -134,11 +134,11 @@ btree_desc(StringInfo buf, XLogReaderState *record) } const char * -btree_identify(uint8 info) +btree_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_BTREE_INSERT_LEAF: id = "INSERT_LEAF"; diff --git a/src/backend/access/rmgrdesc/relmapdesc.c b/src/backend/access/rmgrdesc/relmapdesc.c index f4f0e5ce8b..9c9f191700 100644 --- a/src/backend/access/rmgrdesc/relmapdesc.c +++ b/src/backend/access/rmgrdesc/relmapdesc.c @@ -32,11 +32,11 @@ relmap_desc(StringInfo buf, XLogReaderState *record) } const char * -relmap_identify(uint8 info) +relmap_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_RELMAP_UPDATE: id = "UPDATE"; diff --git a/src/backend/access/rmgrdesc/seqdesc.c b/src/backend/access/rmgrdesc/seqdesc.c index 6def7b653b..eed631b5d8 100644 --- a/src/backend/access/rmgrdesc/seqdesc.c +++ b/src/backend/access/rmgrdesc/seqdesc.c @@ -31,11 +31,11 @@ seq_desc(StringInfo buf, XLogReaderState *record) } const char * -seq_identify(uint8 info) +seq_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_SEQ_LOG: id = "LOG"; diff --git a/src/backend/access/rmgrdesc/smgrdesc.c b/src/backend/access/rmgrdesc/smgrdesc.c index 8604bee0c0..653caf58e4 100644 --- a/src/backend/access/rmgrdesc/smgrdesc.c +++ b/src/backend/access/rmgrdesc/smgrdesc.c @@ -43,11 +43,11 @@ smgr_desc(StringInfo buf, XLogReaderState *record) } const char * -smgr_identify(uint8 info) +smgr_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_SMGR_CREATE: id = "CREATE"; diff --git a/src/backend/access/rmgrdesc/spgdesc.c b/src/backend/access/rmgrdesc/spgdesc.c index 9e1e48240b..0d1af9a5b6 100644 --- a/src/backend/access/rmgrdesc/spgdesc.c +++ b/src/backend/access/rmgrdesc/spgdesc.c @@ -128,11 +128,11 @@ spg_desc(StringInfo buf, XLogReaderState *record) } const char * -spg_identify(uint8 info) +spg_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_SPGIST_ADD_LEAF: id = "ADD_LEAF"; diff --git a/src/backend/access/rmgrdesc/standbydesc.c b/src/backend/access/rmgrdesc/standbydesc.c index 9a08e10fe1..56acea1407 100644 --- a/src/backend/access/rmgrdesc/standbydesc.c +++ b/src/backend/access/rmgrdesc/standbydesc.c @@ -76,11 +76,11 @@ standby_desc(StringInfo buf, XLogReaderState *record) } const char * -standby_identify(uint8 info) +standby_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_STANDBY_LOCK: id = "LOCK"; diff --git a/src/backend/access/rmgrdesc/tblspcdesc.c b/src/backend/access/rmgrdesc/tblspcdesc.c index c1feb4fa05..c5849ccf7c 100644 --- a/src/backend/access/rmgrdesc/tblspcdesc.c +++ b/src/backend/access/rmgrdesc/tblspcdesc.c @@ -38,11 +38,11 @@ tblspc_desc(StringInfo buf, XLogReaderState *record) } const char * -tblspc_identify(uint8 info) +tblspc_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_TBLSPC_CREATE: id = "CREATE"; diff --git a/src/backend/access/rmgrdesc/xlogdesc.c b/src/backend/access/rmgrdesc/xlogdesc.c index 3692dfb96a..4db4f3331f 100644 --- a/src/backend/access/rmgrdesc/xlogdesc.c +++ b/src/backend/access/rmgrdesc/xlogdesc.c @@ -151,11 +151,11 @@ xlog_desc(StringInfo buf, XLogReaderState *record) } const char * -xlog_identify(uint8 info) +xlog_identify(uint8 rmgrinfo) { const char *id = NULL; - switch (info & ~XLR_INFO_MASK) + switch (rmgrinfo) { case XLOG_CHECKPOINT_SHUTDOWN: id = "CHECKPOINT_SHUTDOWN"; diff --git a/src/include/access/brin_xlog.h b/src/include/access/brin_xlog.h index ae263a371f..45a688d09b 100644 --- a/src/include/access/brin_xlog.h +++ b/src/include/access/brin_xlog.h @@ -145,7 +145,7 @@ typedef struct xl_brin_desummarize extern void brin_redo(XLogReaderState *record); extern void brin_desc(StringInfo buf, XLogReaderState *record); -extern const char *brin_identify(uint8 info); +extern const char *brin_identify(uint8 rmgrinfo); extern void brin_mask(char *pagedata, BlockNumber blkno); #endif /* BRIN_XLOG_H */ diff --git a/src/include/access/clog.h b/src/include/access/clog.h index d99444f073..224cad4415 100644 --- a/src/include/access/clog.h +++ b/src/include/access/clog.h @@ -58,6 +58,6 @@ extern int clogsyncfiletag(const FileTag *ftag, char *path); extern void clog_redo(XLogReaderState *record); extern void clog_desc(StringInfo buf, XLogReaderState *record); -extern const char *clog_identify(uint8 info); +extern const char *clog_identify(uint8 rmgrinfo); #endif /* CLOG_H */ diff --git a/src/include/access/ginxlog.h b/src/include/access/ginxlog.h index 37095e5f17..adf1f6b6d8 100644 --- a/src/include/access/ginxlog.h +++ b/src/include/access/ginxlog.h @@ -208,7 +208,7 @@ typedef struct ginxlogDeleteListPages extern void gin_redo(XLogReaderState *record); extern void gin_desc(StringInfo buf, XLogReaderState *record); -extern const char *gin_identify(uint8 info); +extern const char *gin_identify(uint8 rmgrinfo); extern void gin_xlog_startup(void); extern void gin_xlog_cleanup(void); extern void gin_mask(char *pagedata, BlockNumber blkno); diff --git a/src/include/access/gistxlog.h b/src/include/access/gistxlog.h index aff2ffbdcc..0ca37f3a07 100644 --- a/src/include/access/gistxlog.h +++ b/src/include/access/gistxlog.h @@ -109,7 +109,7 @@ typedef struct gistxlogPageReuse extern void gist_redo(XLogReaderState *record); extern void gist_desc(StringInfo buf, XLogReaderState *record); -extern const char *gist_identify(uint8 info); +extern const char *gist_identify(uint8 rmgrinfo); extern void gist_xlog_startup(void); extern void gist_xlog_cleanup(void); extern void gist_mask(char *pagedata, BlockNumber blkno); diff --git a/src/include/access/hash_xlog.h b/src/include/access/hash_xlog.h index b93619d1a8..74859a2a47 100644 --- a/src/include/access/hash_xlog.h +++ b/src/include/access/hash_xlog.h @@ -263,7 +263,7 @@ typedef struct xl_hash_vacuum_one_page extern void hash_redo(XLogReaderState *record); extern void hash_desc(StringInfo buf, XLogReaderState *record); -extern const char *hash_identify(uint8 info); +extern const char *hash_identify(uint8 rmgrinfo); extern void hash_mask(char *pagedata, BlockNumber blkno); #endif /* HASH_XLOG_H */ diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h index a038450787..70e9acd350 100644 --- a/src/include/access/heapam_xlog.h +++ b/src/include/access/heapam_xlog.h @@ -406,11 +406,11 @@ extern void HeapTupleHeaderAdvanceConflictHorizon(HeapTupleHeader tuple, extern void heap_redo(XLogReaderState *record); extern void heap_desc(StringInfo buf, XLogReaderState *record); -extern const char *heap_identify(uint8 info); +extern const char *heap_identify(uint8 rmgrinfo); extern void heap_mask(char *pagedata, BlockNumber blkno); extern void heap2_redo(XLogReaderState *record); extern void heap2_desc(StringInfo buf, XLogReaderState *record); -extern const char *heap2_identify(uint8 info); +extern const char *heap2_identify(uint8 rmgrinfo); extern void heap_xlog_logical_rewrite(XLogReaderState *r); extern XLogRecPtr log_heap_visible(Relation rel, Buffer heap_buffer, diff --git a/src/include/access/multixact.h b/src/include/access/multixact.h index 246f757f6a..d52e69aa05 100644 --- a/src/include/access/multixact.h +++ b/src/include/access/multixact.h @@ -158,7 +158,7 @@ extern void multixact_twophase_postabort(TransactionId xid, uint16 info, extern void multixact_redo(XLogReaderState *record); extern void multixact_desc(StringInfo buf, XLogReaderState *record); -extern const char *multixact_identify(uint8 info); +extern const char *multixact_identify(uint8 rmgrinfo); extern char *mxid_to_string(MultiXactId multi, int nmembers, MultiXactMember *members); diff --git a/src/include/access/nbtxlog.h b/src/include/access/nbtxlog.h index 7dd9fd0206..422ac13d56 100644 --- a/src/include/access/nbtxlog.h +++ b/src/include/access/nbtxlog.h @@ -362,6 +362,6 @@ extern void btree_mask(char *pagedata, BlockNumber blkno); * prototypes for functions in nbtdesc.c */ extern void btree_desc(StringInfo buf, XLogReaderState *record); -extern const char *btree_identify(uint8 info); +extern const char *btree_identify(uint8 rmgrinfo); #endif /* NBTXLOG_H */ diff --git a/src/include/access/spgxlog.h b/src/include/access/spgxlog.h index a7f10bf2d3..3533894baf 100644 --- a/src/include/access/spgxlog.h +++ b/src/include/access/spgxlog.h @@ -251,7 +251,7 @@ typedef struct spgxlogVacuumRedirect extern void spg_redo(XLogReaderState *record); extern void spg_desc(StringInfo buf, XLogReaderState *record); -extern const char *spg_identify(uint8 info); +extern const char *spg_identify(uint8 rmgrinfo); extern void spg_xlog_startup(void); extern void spg_xlog_cleanup(void); extern void spg_mask(char *pagedata, BlockNumber blkno); diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h index 48ca852381..9007f559f3 100644 --- a/src/include/access/xlog.h +++ b/src/include/access/xlog.h @@ -211,7 +211,7 @@ extern void XLogSetReplicationSlotMinimumLSN(XLogRecPtr lsn); extern void xlog_redo(struct XLogReaderState *record); extern void xlog_desc(StringInfo buf, struct XLogReaderState *record); -extern const char *xlog_identify(uint8 info); +extern const char *xlog_identify(uint8 rmgrinfo); extern void issue_xlog_fsync(int fd, XLogSegNo segno, TimeLineID tli); diff --git a/src/include/catalog/storage_xlog.h b/src/include/catalog/storage_xlog.h index 6b0a7aa3df..27514163be 100644 --- a/src/include/catalog/storage_xlog.h +++ b/src/include/catalog/storage_xlog.h @@ -54,6 +54,6 @@ extern void log_smgrcreate(const RelFileLocator *rlocator, ForkNumber forkNum); extern void smgr_redo(XLogReaderState *record); extern void smgr_desc(StringInfo buf, XLogReaderState *record); -extern const char *smgr_identify(uint8 info); +extern const char *smgr_identify(uint8 rmgrinfo); #endif /* STORAGE_XLOG_H */ diff --git a/src/include/commands/dbcommands_xlog.h b/src/include/commands/dbcommands_xlog.h index a1433275f5..f3c11060b6 100644 --- a/src/include/commands/dbcommands_xlog.h +++ b/src/include/commands/dbcommands_xlog.h @@ -55,6 +55,6 @@ typedef struct xl_dbase_drop_rec extern void dbase_redo(XLogReaderState *record); extern void dbase_desc(StringInfo buf, XLogReaderState *record); -extern const char *dbase_identify(uint8 info); +extern const char *dbase_identify(uint8 rmgrinfo); #endif /* DBCOMMANDS_XLOG_H */ diff --git a/src/include/commands/sequence.h b/src/include/commands/sequence.h index 7db7b3da7b..fb7059b8f6 100644 --- a/src/include/commands/sequence.h +++ b/src/include/commands/sequence.h @@ -64,7 +64,7 @@ extern void ResetSequenceCaches(void); extern void seq_redo(XLogReaderState *record); extern void seq_desc(StringInfo buf, XLogReaderState *record); -extern const char *seq_identify(uint8 info); +extern const char *seq_identify(uint8 rmgrinfo); extern void seq_mask(char *page, BlockNumber blkno); #endif /* SEQUENCE_H */ diff --git a/src/include/commands/tablespace.h b/src/include/commands/tablespace.h index f1961c1813..db21eb59ee 100644 --- a/src/include/commands/tablespace.h +++ b/src/include/commands/tablespace.h @@ -64,6 +64,6 @@ extern void remove_tablespace_symlink(const char *linkloc); extern void tblspc_redo(XLogReaderState *record); extern void tblspc_desc(StringInfo buf, XLogReaderState *record); -extern const char *tblspc_identify(uint8 info); +extern const char *tblspc_identify(uint8 rmgrinfo); #endif /* TABLESPACE_H */ diff --git a/src/include/replication/message.h b/src/include/replication/message.h index 6ce7f2038b..2fe985a6fe 100644 --- a/src/include/replication/message.h +++ b/src/include/replication/message.h @@ -36,6 +36,6 @@ extern XLogRecPtr LogLogicalMessage(const char *prefix, const char *message, #define XLOG_LOGICAL_MESSAGE 0x00 extern void logicalmsg_redo(XLogReaderState *record); extern void logicalmsg_desc(StringInfo buf, XLogReaderState *record); -extern const char *logicalmsg_identify(uint8 info); +extern const char *logicalmsg_identify(uint8 rmgrinfo); #endif /* PG_LOGICAL_MESSAGE_H */ diff --git a/src/include/storage/standbydefs.h b/src/include/storage/standbydefs.h index 188e348618..84e1f03c8f 100644 --- a/src/include/storage/standbydefs.h +++ b/src/include/storage/standbydefs.h @@ -22,7 +22,7 @@ /* Recovery handlers for the Standby Rmgr (RM_STANDBY_ID) */ extern void standby_redo(XLogReaderState *record); extern void standby_desc(StringInfo buf, XLogReaderState *record); -extern const char *standby_identify(uint8 info); +extern const char *standby_identify(uint8 rmgrinfo); extern void standby_desc_invalidations(StringInfo buf, int nmsgs, SharedInvalidationMessage *msgs, Oid dbId, Oid tsId, diff --git a/src/include/utils/relmapper.h b/src/include/utils/relmapper.h index 5c173bdbb3..e3c41ff093 100644 --- a/src/include/utils/relmapper.h +++ b/src/include/utils/relmapper.h @@ -68,6 +68,6 @@ extern void RestoreRelationMap(char *startAddress); extern void relmap_redo(XLogReaderState *record); extern void relmap_desc(StringInfo buf, XLogReaderState *record); -extern const char *relmap_identify(uint8 info); +extern const char *relmap_identify(uint8 rmgrinfo); #endif /* RELMAPPER_H */ diff --git a/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c b/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c index b5537a5f5b..21514f8e33 100644 --- a/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c +++ b/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c @@ -103,9 +103,9 @@ testcustomrmgrs_desc(StringInfo buf, XLogReaderState *record) } const char * -testcustomrmgrs_identify(uint8 info) +testcustomrmgrs_identify(uint8 rmgrinfo) { - if ((info & ~XLR_INFO_MASK) == XLOG_TEST_CUSTOM_RMGRS_MESSAGE) + if (rmgrinfo == XLOG_TEST_CUSTOM_RMGRS_MESSAGE) return "TEST_CUSTOM_RMGRS_MESSAGE"; return NULL; -- 2.40.1 [application/octet-stream] v5-0004-Move-rmgr-info-bits-into-their-own-field-in-the-x.patch (26.1K, ../../CAEze2WgQ9d_aa6tGVDqXgdy2owvrrJtY6TaCZQ6r+BA7F1fSCA@mail.gmail.com/3-v5-0004-Move-rmgr-info-bits-into-their-own-field-in-the-x.patch) download | inline diff: From 3ca3cd6a7178e7f8d81d7205f362ac67911fc2bb Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Mon, 23 Jan 2023 17:21:12 +0100 Subject: [PATCH v5 4/7] Move rmgr info bits into their own field in the xlog record header. This takes one byte of the 2 previously lost to alignment, and allows each rmgr to use more info bits (useful for e.g. merging the heap rmgrs back into one). --- contrib/pg_walinspect/pg_walinspect.c | 7 +++--- src/backend/access/transam/xact.c | 22 +++++++++-------- src/backend/access/transam/xlog.c | 7 +++--- src/backend/access/transam/xloginsert.c | 27 ++++++++++++--------- src/backend/access/transam/xlogprefetcher.c | 2 +- src/backend/access/transam/xlogreader.c | 2 +- src/backend/access/transam/xlogrecovery.c | 10 ++++---- src/backend/access/transam/xlogstats.c | 12 ++++----- src/backend/catalog/storage.c | 7 +++--- src/backend/commands/dbcommands.c | 20 +++++++++------ src/backend/utils/probes.d | 2 +- src/bin/pg_resetwal/pg_resetwal.c | 3 ++- src/bin/pg_rewind/parsexlog.c | 4 +-- src/bin/pg_waldump/pg_waldump.c | 7 +++--- src/include/access/brin_xlog.h | 3 +-- src/include/access/heapam_xlog.h | 4 +-- src/include/access/nbtxlog.h | 3 +-- src/include/access/xact.h | 6 ++--- src/include/access/xlog_internal.h | 2 +- src/include/access/xloginsert.h | 3 ++- src/include/access/xlogreader.h | 4 +-- src/include/access/xlogrecord.h | 11 ++------- src/include/access/xlogstats.h | 5 ++-- 23 files changed, 89 insertions(+), 84 deletions(-) diff --git a/contrib/pg_walinspect/pg_walinspect.c b/contrib/pg_walinspect/pg_walinspect.c index 1b6f4cb178..909700f9a1 100644 --- a/contrib/pg_walinspect/pg_walinspect.c +++ b/contrib/pg_walinspect/pg_walinspect.c @@ -697,7 +697,7 @@ GetXLogSummaryStats(XLogStats *stats, ReturnSetInfo *rsinfo, { int rj; - for (rj = 0; rj < MAX_XLINFO_TYPES; rj++) + for (rj = 0; rj < MAX_XLRMGRINFO_TYPES; rj++) { const char *id; @@ -712,10 +712,9 @@ GetXLogSummaryStats(XLogStats *stats, ReturnSetInfo *rsinfo, old_cxt = MemoryContextSwitchTo(tmp_cxt); - /* the upper four bits in xl_info are the rmgr's */ - id = desc.rm_identify(rj << 4); + id = desc.rm_identify(rj); if (id == NULL) - id = psprintf("UNKNOWN (%x)", rj << 4); + id = psprintf("UNKNOWN (%x)", rj); FillXLogStatsRow(psprintf("%s/%s", desc.rm_name, id), count, total_count, rec_len, total_rec_len, fpi_len, diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index c560f07390..da0ac6c8d3 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -5650,7 +5650,8 @@ XactLogCommitRecord(TimestampTz commit_time, xl_xact_invals xl_invals; xl_xact_twophase xl_twophase; xl_xact_origin xl_origin; - uint8 info; + uint8 info = 0; + uint8 rmgr_info; Assert(CritSectionCount > 0); @@ -5658,9 +5659,9 @@ XactLogCommitRecord(TimestampTz commit_time, /* decide between a plain and 2pc commit */ if (!TransactionIdIsValid(twophase_xid)) - info = XLOG_XACT_COMMIT; + rmgr_info = XLOG_XACT_COMMIT; else - info = XLOG_XACT_COMMIT_PREPARED; + rmgr_info = XLOG_XACT_COMMIT_PREPARED; /* First figure out and collect all the information needed */ @@ -5736,7 +5737,7 @@ XactLogCommitRecord(TimestampTz commit_time, } if (xl_xinfo.xinfo != 0) - info |= XLOG_XACT_HAS_INFO; + rmgr_info |= XLOG_XACT_HAS_INFO; /* Then include all the collected data into the commit record. */ @@ -5794,7 +5795,7 @@ XactLogCommitRecord(TimestampTz commit_time, /* we allow filtering by xacts */ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); - return XLogInsert(RM_XACT_ID, info); + return XLogInsertExtended(RM_XACT_ID, info, rmgr_info); } /* @@ -5820,7 +5821,8 @@ XactLogAbortRecord(TimestampTz abort_time, xl_xact_dbinfo xl_dbinfo; xl_xact_origin xl_origin; - uint8 info; + uint8 info = 0; + uint8 rmgr_info; Assert(CritSectionCount > 0); @@ -5828,9 +5830,9 @@ XactLogAbortRecord(TimestampTz abort_time, /* decide between a plain and 2pc abort */ if (!TransactionIdIsValid(twophase_xid)) - info = XLOG_XACT_ABORT; + rmgr_info = XLOG_XACT_ABORT; else - info = XLOG_XACT_ABORT_PREPARED; + rmgr_info = XLOG_XACT_ABORT_PREPARED; /* First figure out and collect all the information needed */ @@ -5889,7 +5891,7 @@ XactLogAbortRecord(TimestampTz abort_time, } if (xl_xinfo.xinfo != 0) - info |= XLOG_XACT_HAS_INFO; + rmgr_info |= XLOG_XACT_HAS_INFO; /* Then include all the collected data into the abort record. */ @@ -5940,7 +5942,7 @@ XactLogAbortRecord(TimestampTz abort_time, /* Include the replication origin */ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); - return XLogInsert(RM_XACT_ID, info); + return XLogInsertExtended(RM_XACT_ID, info, rmgr_info); } /* diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 08c78bf57b..eded22ab1d 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -738,9 +738,9 @@ XLogInsertRecord(XLogRecData *rdata, pg_crc32c rdata_crc; bool inserted; XLogRecord *rechdr = (XLogRecord *) rdata->data; - uint8 info = rechdr->xl_info & ~XLR_INFO_MASK; + uint8 rmgrinfo = rechdr->xl_rmgrinfo; bool isLogSwitch = (rechdr->xl_rmid == RM_XLOG_ID && - info == XLOG_SWITCH); + rmgrinfo == XLOG_SWITCH); XLogRecPtr StartPos; XLogRecPtr EndPos; bool prevDoPageWrites = doPageWrites; @@ -4753,7 +4753,8 @@ BootStrapXLOG(void) record->xl_prev = 0; record->xl_xid = InvalidTransactionId; record->xl_tot_len = SizeOfXLogRecord + SizeOfXLogRecordDataHeaderShort + sizeof(checkPoint); - record->xl_info = XLOG_CHECKPOINT_SHUTDOWN; + record->xl_info = 0; + record->xl_rmgrinfo = XLOG_CHECKPOINT_SHUTDOWN; record->xl_rmid = RM_XLOG_ID; recptr += SizeOfXLogRecord; /* fill the XLogRecordDataHeaderShort struct */ diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c index 51b487887b..9596a37cb2 100644 --- a/src/backend/access/transam/xloginsert.c +++ b/src/backend/access/transam/xloginsert.c @@ -135,7 +135,7 @@ static bool begininsert_called = false; /* Memory context to hold the registered buffer and data references. */ static MemoryContext xloginsert_cxt; -static XLogRecData *XLogRecordAssemble(RmgrId rmid, uint8 info, +static XLogRecData *XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecPtr, bool doPageWrites, XLogRecPtr *fpw_lsn, int *num_fpi, bool *topxid_included); @@ -447,9 +447,9 @@ XLogSetRecordFlags(uint8 flags) } /* - * Insert an XLOG record having the specified RMID and info bytes, with the - * body of the record being the data and buffer references registered earlier - * with XLogRegister* calls. + * Insert an XLOG record having the specified RMID and rmgr_info bytes, with + * the body of the record being the data and buffer references registered + * earlier with XLogRegister* calls. * * Returns XLOG pointer to end of record (beginning of next record). * This can be used as LSN for data pages affected by the logged action. @@ -458,7 +458,13 @@ XLogSetRecordFlags(uint8 flags) * WAL rule "write the log before the data".) */ XLogRecPtr -XLogInsert(RmgrId rmid, uint8 info) +XLogInsert(RmgrId rmid, uint8 rmgr_info) +{ + return XLogInsertExtended(rmid, 0, rmgr_info); +} + +XLogRecPtr +XLogInsertExtended(RmgrId rmid, uint8 info, uint8 rmgr_info) { XLogRecPtr EndPos; @@ -470,12 +476,11 @@ XLogInsert(RmgrId rmid, uint8 info) * The caller can set rmgr bits, XLR_SPECIAL_REL_UPDATE and * XLR_CHECK_CONSISTENCY; the rest are reserved for use by me. */ - if ((info & ~(XLR_RMGR_INFO_MASK | - XLR_SPECIAL_REL_UPDATE | + if ((info & ~(XLR_SPECIAL_REL_UPDATE | XLR_CHECK_CONSISTENCY)) != 0) elog(PANIC, "invalid xlog info mask %02X", info); - TRACE_POSTGRESQL_WAL_INSERT(rmid, info); + TRACE_POSTGRESQL_WAL_INSERT(rmid, info, rmgr_info); /* * In bootstrap mode, we don't actually log anything but XLOG resources; @@ -504,7 +509,7 @@ XLogInsert(RmgrId rmid, uint8 info) */ GetFullPageWriteInfo(&RedoRecPtr, &doPageWrites); - rdt = XLogRecordAssemble(rmid, info, RedoRecPtr, doPageWrites, + rdt = XLogRecordAssemble(rmid, info, rmgr_info, RedoRecPtr, doPageWrites, &fpw_lsn, &num_fpi, &topxid_included); EndPos = XLogInsertRecord(rdt, fpw_lsn, curinsert_flags, num_fpi, @@ -532,8 +537,7 @@ XLogInsert(RmgrId rmid, uint8 info) * current subtransaction. */ static XLogRecData * -XLogRecordAssemble(RmgrId rmid, uint8 info, - XLogRecPtr RedoRecPtr, bool doPageWrites, +XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecPtr, bool doPageWrites, XLogRecPtr *fpw_lsn, int *num_fpi, bool *topxid_included) { XLogRecData *rdt; @@ -924,6 +928,7 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, rechdr->xl_tot_len = (uint32) total_len; rechdr->xl_info = info; rechdr->xl_rmid = rmid; + rechdr->xl_rmgrinfo = rmgr_info; rechdr->xl_prev = InvalidXLogRecPtr; rechdr->xl_crc = rdata_crc; diff --git a/src/backend/access/transam/xlogprefetcher.c b/src/backend/access/transam/xlogprefetcher.c index 539928cb85..dd4ca866a4 100644 --- a/src/backend/access/transam/xlogprefetcher.c +++ b/src/backend/access/transam/xlogprefetcher.c @@ -536,7 +536,7 @@ XLogPrefetcherNextBlock(uintptr_t pgsr_private, XLogRecPtr *lsn) if (replaying_lsn < record->lsn) { uint8 rmid = record->header.xl_rmid; - uint8 record_type = record->header.xl_info & ~XLR_INFO_MASK; + uint8 record_type = record->header.xl_rmgrinfo; if (rmid == RM_XLOG_ID) { diff --git a/src/backend/access/transam/xlogreader.c b/src/backend/access/transam/xlogreader.c index becb207152..bebdf1759a 100644 --- a/src/backend/access/transam/xlogreader.c +++ b/src/backend/access/transam/xlogreader.c @@ -860,7 +860,7 @@ restart: * Special processing if it's an XLOG SWITCH record */ if (record->xl_rmid == RM_XLOG_ID && - (record->xl_info & ~XLR_INFO_MASK) == XLOG_SWITCH) + record->xl_rmgrinfo == XLOG_SWITCH) { /* Pretend it extends to end of segment */ state->NextRecPtr += state->segcxt.ws_segsize - 1; diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index 6508246999..56fa2f74a2 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -632,7 +632,7 @@ InitWalRecovery(ControlFileData *ControlFile, bool *wasShutdown_ptr, if (record != NULL) { memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint)); - wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN); + wasShutdown = (record->xl_rmgrinfo == XLOG_CHECKPOINT_SHUTDOWN); ereport(DEBUG1, (errmsg_internal("checkpoint record is at %X/%X", LSN_FORMAT_ARGS(CheckPointLoc)))); @@ -786,7 +786,7 @@ InitWalRecovery(ControlFileData *ControlFile, bool *wasShutdown_ptr, (errmsg("could not locate a valid checkpoint record"))); } memcpy(&checkPoint, XLogRecGetData(xlogreader), sizeof(CheckPoint)); - wasShutdown = ((record->xl_info & ~XLR_INFO_MASK) == XLOG_CHECKPOINT_SHUTDOWN); + wasShutdown = (record->xl_rmgrinfo == XLOG_CHECKPOINT_SHUTDOWN); } /* @@ -1857,7 +1857,7 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl { TimeLineID newReplayTLI = *replayTLI; TimeLineID prevReplayTLI = *replayTLI; - uint8 info = record->xl_info & ~XLR_INFO_MASK; + uint8 info = record->xl_rmgrinfo; if (info == XLOG_CHECKPOINT_SHUTDOWN) { @@ -3992,12 +3992,12 @@ ReadCheckpointRecord(XLogPrefetcher *xlogprefetcher, XLogRecPtr RecPtr, (errmsg("invalid resource manager ID in checkpoint record"))); return NULL; } - info = record->xl_info & ~XLR_INFO_MASK; + info = record->xl_rmgrinfo; if (info != XLOG_CHECKPOINT_SHUTDOWN && info != XLOG_CHECKPOINT_ONLINE) { ereport(LOG, - (errmsg("invalid xl_info in checkpoint record"))); + (errmsg("invalid xl_rmgrinfo in checkpoint record"))); return NULL; } if (record->xl_tot_len != SizeOfXLogRecord + SizeOfXLogRecordDataHeaderShort + sizeof(CheckPoint)) diff --git a/src/backend/access/transam/xlogstats.c b/src/backend/access/transam/xlogstats.c index 267b108943..28d6d971ac 100644 --- a/src/backend/access/transam/xlogstats.c +++ b/src/backend/access/transam/xlogstats.c @@ -74,21 +74,21 @@ XLogRecStoreStats(XLogStats *stats, XLogReaderState *record) /* * Update per-record statistics, where the record is identified by a - * combination of the RmgrId and the four bits of the xl_info field that - * are the rmgr's domain (resulting in sixteen possible entries per + * combination of the RmgrId and the eight bits of the xl_rmgrinfo field + * that are the rmgr's domain (resulting in 256 possible entries per * RmgrId). */ - recid = XLogRecGetRmgrInfo(record) >> 4; + recid = XLogRecGetRmgrInfo(record); /* * XACT records need to be handled differently. Those records use the * first bit of those four bits for an optional flag variable and the - * following three bits for the opcode. We filter opcode out of xl_info - * and use it as the identifier of the record. + * following three bits for the opcode. We filter opcode out of + * xl_rmgrinfo and use it as the identifier of the record. */ if (rmid == RM_XACT_ID) - recid &= 0x07; + recid &= 0x70; stats->record_stats[rmid][recid].count++; stats->record_stats[rmid][recid].rec_len += rec_len; diff --git a/src/backend/catalog/storage.c b/src/backend/catalog/storage.c index 9ce4d9a920..0048ce81d7 100644 --- a/src/backend/catalog/storage.c +++ b/src/backend/catalog/storage.c @@ -194,7 +194,7 @@ log_smgrcreate(const RelFileLocator *rlocator, ForkNumber forkNum) XLogBeginInsert(); XLogRegisterData((char *) &xlrec, sizeof(xlrec)); - XLogInsert(RM_SMGR_ID, XLOG_SMGR_CREATE | XLR_SPECIAL_REL_UPDATE); + XLogInsertExtended(RM_SMGR_ID, XLR_SPECIAL_REL_UPDATE, XLOG_SMGR_CREATE); } /* @@ -375,8 +375,9 @@ RelationTruncate(Relation rel, BlockNumber nblocks) XLogBeginInsert(); XLogRegisterData((char *) &xlrec, sizeof(xlrec)); - lsn = XLogInsert(RM_SMGR_ID, - XLOG_SMGR_TRUNCATE | XLR_SPECIAL_REL_UPDATE); + lsn = XLogInsertExtended(RM_SMGR_ID, + XLR_SPECIAL_REL_UPDATE, + XLOG_SMGR_TRUNCATE); /* * Flush, because otherwise the truncation of the main relation might diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c index bc061a697d..1f6d670fa6 100644 --- a/src/backend/commands/dbcommands.c +++ b/src/backend/commands/dbcommands.c @@ -624,8 +624,9 @@ CreateDatabaseUsingFileCopy(Oid src_dboid, Oid dst_dboid, Oid src_tsid, XLogRegisterData((char *) &xlrec, sizeof(xl_dbase_create_file_copy_rec)); - (void) XLogInsert(RM_DBASE_ID, - XLOG_DBASE_CREATE_FILE_COPY | XLR_SPECIAL_REL_UPDATE); + (void) XLogInsertExtended(RM_DBASE_ID, + XLR_SPECIAL_REL_UPDATE, + XLOG_DBASE_CREATE_FILE_COPY); } pfree(srcpath); pfree(dstpath); @@ -2112,8 +2113,9 @@ movedb(const char *dbname, const char *tblspcname) XLogRegisterData((char *) &xlrec, sizeof(xl_dbase_create_file_copy_rec)); - (void) XLogInsert(RM_DBASE_ID, - XLOG_DBASE_CREATE_FILE_COPY | XLR_SPECIAL_REL_UPDATE); + (void) XLogInsertExtended(RM_DBASE_ID, + XLR_SPECIAL_REL_UPDATE, + XLOG_DBASE_CREATE_FILE_COPY); } /* @@ -2206,8 +2208,9 @@ movedb(const char *dbname, const char *tblspcname) XLogRegisterData((char *) &xlrec, sizeof(xl_dbase_drop_rec)); XLogRegisterData((char *) &src_tblspcoid, sizeof(Oid)); - (void) XLogInsert(RM_DBASE_ID, - XLOG_DBASE_DROP | XLR_SPECIAL_REL_UPDATE); + (void) XLogInsertExtended(RM_DBASE_ID, + XLR_SPECIAL_REL_UPDATE, + XLOG_DBASE_DROP); } /* Now it's safe to release the database lock */ @@ -2937,8 +2940,9 @@ remove_dbtablespaces(Oid db_id) XLogRegisterData((char *) &xlrec, MinSizeOfDbaseDropRec); XLogRegisterData((char *) tablespace_ids, ntblspc * sizeof(Oid)); - (void) XLogInsert(RM_DBASE_ID, - XLOG_DBASE_DROP | XLR_SPECIAL_REL_UPDATE); + (void) XLogInsertExtended(RM_DBASE_ID, + XLR_SPECIAL_REL_UPDATE, + XLOG_DBASE_DROP); } list_free(ltblspc); diff --git a/src/backend/utils/probes.d b/src/backend/utils/probes.d index 0af275587b..131067b5ac 100644 --- a/src/backend/utils/probes.d +++ b/src/backend/utils/probes.d @@ -87,7 +87,7 @@ provider postgresql { probe smgr__md__write__start(ForkNumber, BlockNumber, Oid, Oid, Oid, int); probe smgr__md__write__done(ForkNumber, BlockNumber, Oid, Oid, Oid, int, int, int); - probe wal__insert(unsigned char, unsigned char); + probe wal__insert(unsigned char, unsigned char, unsigned char); probe wal__switch(); probe wal__buffer__write__dirty__start(); probe wal__buffer__write__dirty__done(); diff --git a/src/bin/pg_resetwal/pg_resetwal.c b/src/bin/pg_resetwal/pg_resetwal.c index 25ecdaaa15..8e61a7117e 100644 --- a/src/bin/pg_resetwal/pg_resetwal.c +++ b/src/bin/pg_resetwal/pg_resetwal.c @@ -1070,7 +1070,8 @@ WriteEmptyXLOG(void) record->xl_prev = 0; record->xl_xid = InvalidTransactionId; record->xl_tot_len = SizeOfXLogRecord + SizeOfXLogRecordDataHeaderShort + sizeof(CheckPoint); - record->xl_info = XLOG_CHECKPOINT_SHUTDOWN; + record->xl_info = 0; + record->xl_rmgrinfo = XLOG_CHECKPOINT_SHUTDOWN; record->xl_rmid = RM_XLOG_ID; recptr += SizeOfXLogRecord; diff --git a/src/bin/pg_rewind/parsexlog.c b/src/bin/pg_rewind/parsexlog.c index 2b6b46f0e3..aa03fe8031 100644 --- a/src/bin/pg_rewind/parsexlog.c +++ b/src/bin/pg_rewind/parsexlog.c @@ -438,9 +438,9 @@ extractPageInfo(XLogReaderState *record) * track that change. */ pg_fatal("WAL record modifies a relation, but record type is not recognized: " - "lsn: %X/%X, rmid: %d, rmgr: %s, info: %02X", + "lsn: %X/%X, rmid: %d, rmgr: %s, info: %02X, rmgrinfo: %02X", LSN_FORMAT_ARGS(record->ReadRecPtr), - rmid, RmgrName(rmid), info | rminfo); + rmid, RmgrName(rmid), info, rminfo); } for (block_id = 0; block_id <= XLogRecMaxBlockId(record); block_id++) diff --git a/src/bin/pg_waldump/pg_waldump.c b/src/bin/pg_waldump/pg_waldump.c index 48b733b767..fb6a5e76b1 100644 --- a/src/bin/pg_waldump/pg_waldump.c +++ b/src/bin/pg_waldump/pg_waldump.c @@ -698,7 +698,7 @@ XLogDumpDisplayStats(XLogDumpConfig *config, XLogStats *stats) } else { - for (rj = 0; rj < MAX_XLINFO_TYPES; rj++) + for (rj = 0; rj < MAX_XLRMGRINFO_TYPES; rj++) { const char *id; @@ -711,10 +711,9 @@ XLogDumpDisplayStats(XLogDumpConfig *config, XLogStats *stats) if (count == 0) continue; - /* the upper four bits in xl_info are the rmgr's */ - id = desc->rm_identify(rj << 4); + id = desc->rm_identify(rj); if (id == NULL) - id = psprintf("UNKNOWN (%x)", rj << 4); + id = psprintf("UNKNOWN (%x)", rj); XLogDumpStatsRow(psprintf("%s/%s", desc->rm_name, id), count, total_count, rec_len, total_rec_len, diff --git a/src/include/access/brin_xlog.h b/src/include/access/brin_xlog.h index 45a688d09b..92190688c9 100644 --- a/src/include/access/brin_xlog.h +++ b/src/include/access/brin_xlog.h @@ -25,8 +25,7 @@ /* * WAL record definitions for BRIN's WAL operations * - * XLOG allows to store some information in high 4 bits of log - * record xl_info field. + * XLOG allows to store some information in the 8-bit xl_rmgrinfo field. */ #define XLOG_BRIN_CREATE_INDEX 0x00 #define XLOG_BRIN_INSERT 0x10 diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h index 70e9acd350..6705c8a316 100644 --- a/src/include/access/heapam_xlog.h +++ b/src/include/access/heapam_xlog.h @@ -26,8 +26,8 @@ /* * WAL record definitions for heapam.c's WAL operations * - * XLOG allows to store some information in high 4 bits of log - * record xl_info field. We use 3 for opcode and one for init bit. + * XLOG allows to store some information in the 8-bit xl_rmgrinfo field. + * We use 3 for opcode and one for init bit. */ #define XLOG_HEAP_INSERT 0x00 #define XLOG_HEAP_DELETE 0x10 diff --git a/src/include/access/nbtxlog.h b/src/include/access/nbtxlog.h index 422ac13d56..6d1231a01e 100644 --- a/src/include/access/nbtxlog.h +++ b/src/include/access/nbtxlog.h @@ -21,8 +21,7 @@ /* * XLOG records for btree operations * - * XLOG allows to store some information in high 4 bits of log - * record xl_info field + * XLOG allows to store some information in the 8-bit xl_rmgrinfo field. */ #define XLOG_BTREE_INSERT_LEAF 0x00 /* add index tuple without split */ #define XLOG_BTREE_INSERT_UPPER 0x10 /* same, on a non-leaf page */ diff --git a/src/include/access/xact.h b/src/include/access/xact.h index b94b264402..2a9a708279 100644 --- a/src/include/access/xact.h +++ b/src/include/access/xact.h @@ -163,8 +163,8 @@ typedef struct SavedTransactionCharacteristics */ /* - * XLOG allows to store some information in high 4 bits of log record xl_info - * field. We use 3 for the opcode, and one about an optional flag variable. + * XLOG allows to store some information in the 8-bit xl_rmgrinfo field. + * We use 3 for the opcode, and one about an optional flag variable. */ #define XLOG_XACT_COMMIT 0x00 #define XLOG_XACT_PREPARE 0x10 @@ -232,7 +232,7 @@ typedef struct xl_xact_assignment * A minimal commit/abort record only consists of a xl_xact_commit/abort * struct. The presence of additional information is indicated by bits set in * 'xl_xact_xinfo->xinfo'. The presence of the xinfo field itself is signaled - * by a set XLOG_XACT_HAS_INFO bit in the xl_info field. + * by a set XLOG_XACT_HAS_INFO bit in the xl_rmgrinfo field. * * NB: All the individual data chunks should be sized to multiples of * sizeof(int) and only require int32 alignment. If they require bigger diff --git a/src/include/access/xlog_internal.h b/src/include/access/xlog_internal.h index af822b471b..9413836a6a 100644 --- a/src/include/access/xlog_internal.h +++ b/src/include/access/xlog_internal.h @@ -463,7 +463,7 @@ static inline int XLogReadLength(uint32 *length, XLogSizeClass sizeClass, * This struct must be kept in sync with the PG_RMGR definition in * rmgr.c. * - * rm_identify must return a name for the record based on xl_info (without + * rm_identify must return a name for the record based on xl_rmgrinfo (without * reference to the rmid). For example, XLOG_BTREE_VACUUM would be named * "VACUUM". rm_desc can then be called to obtain additional detail for the * record, if available (e.g. the last block). diff --git a/src/include/access/xloginsert.h b/src/include/access/xloginsert.h index 31785dc578..c6c46e5409 100644 --- a/src/include/access/xloginsert.h +++ b/src/include/access/xloginsert.h @@ -41,7 +41,8 @@ /* prototypes for public functions in xloginsert.c: */ extern void XLogBeginInsert(void); extern void XLogSetRecordFlags(uint8 flags); -extern XLogRecPtr XLogInsert(RmgrId rmid, uint8 info); +extern XLogRecPtr XLogInsertExtended(RmgrId rmid, uint8 info, uint8 rmgr_info); +extern XLogRecPtr XLogInsert(RmgrId rmid, uint8 rmgr_info); extern void XLogEnsureRecordSpace(int max_block_id, int ndatas); extern void XLogRegisterData(char *data, uint32 len); extern void XLogRegisterBuffer(uint8 block_id, Buffer buffer, uint8 flags); diff --git a/src/include/access/xlogreader.h b/src/include/access/xlogreader.h index 2b976dcc03..a1d0216404 100644 --- a/src/include/access/xlogreader.h +++ b/src/include/access/xlogreader.h @@ -407,8 +407,8 @@ extern bool DecodeXLogRecord(XLogReaderState *state, */ #define XLogRecGetTotalLen(decoder) ((decoder)->record->header.xl_tot_len) #define XLogRecGetPrev(decoder) ((decoder)->record->header.xl_prev) -#define XLogRecGetInfo(decoder) ((decoder)->record->header.xl_info & XLR_INFO_MASK) -#define XLogRecGetRmgrInfo(decoder) (((decoder)->record->header.xl_info) & XLR_RMGR_INFO_MASK) +#define XLogRecGetInfo(decoder) ((decoder)->record->header.xl_info) +#define XLogRecGetRmgrInfo(decoder) ((decoder)->record->header.xl_rmgrinfo) #define XLogRecGetRmid(decoder) ((decoder)->record->header.xl_rmid) #define XLogRecGetXid(decoder) ((decoder)->record->header.xl_xid) #define XLogRecGetOrigin(decoder) ((decoder)->record->record_origin) diff --git a/src/include/access/xlogrecord.h b/src/include/access/xlogrecord.h index c1c4f04a06..0f32f20591 100644 --- a/src/include/access/xlogrecord.h +++ b/src/include/access/xlogrecord.h @@ -67,7 +67,8 @@ typedef struct XLogRecord XLogRecPtr xl_prev; /* ptr to previous record in log */ uint8 xl_info; /* flag bits, see below */ RmgrId xl_rmid; /* resource manager for this record */ - /* 2 bytes of padding here, initialize to zero */ + uint8 xl_rmgrinfo; /* rmgr flag bits, see below */ + /* 1 byte of padding here, initialize to zero */ pg_crc32c xl_crc; /* CRC for this record */ /* XLogRecordBlockHeaders and XLogRecordDataHeader follow, no padding */ @@ -76,14 +77,6 @@ typedef struct XLogRecord #define SizeOfXLogRecord (offsetof(XLogRecord, xl_crc) + sizeof(pg_crc32c)) -/* - * The high 4 bits in xl_info may be used freely by rmgr. The - * XLR_SPECIAL_REL_UPDATE and XLR_CHECK_CONSISTENCY bits can be passed by - * XLogInsert caller. The rest are set internally by XLogInsert. - */ -#define XLR_INFO_MASK 0x0F -#define XLR_RMGR_INFO_MASK 0xF0 - /* * XLogReader needs to allocate all the data of a WAL record in a single * chunk. This means that a single XLogRecord cannot exceed MaxAllocSize diff --git a/src/include/access/xlogstats.h b/src/include/access/xlogstats.h index 89410ce92b..30974f5ba7 100644 --- a/src/include/access/xlogstats.h +++ b/src/include/access/xlogstats.h @@ -16,7 +16,8 @@ #include "access/rmgr.h" #include "access/xlogreader.h" -#define MAX_XLINFO_TYPES 16 +/* all 8 bits of xl_rmgrinfo are available */ +#define MAX_XLRMGRINFO_TYPES 256 typedef struct XLogRecStats { @@ -33,7 +34,7 @@ typedef struct XLogStats XLogRecPtr endptr; #endif XLogRecStats rmgr_stats[RM_MAX_ID + 1]; - XLogRecStats record_stats[RM_MAX_ID + 1][MAX_XLINFO_TYPES]; + XLogRecStats record_stats[RM_MAX_ID + 1][MAX_XLRMGRINFO_TYPES]; } XLogStats; extern void XLogRecGetLen(XLogReaderState *record, uint32 *rec_len, -- 2.40.1 [application/octet-stream] v5-0002-Replace-accesses-to-rmgr-s-XLogRecord-info-bit-ac.patch (44.3K, ../../CAEze2WgQ9d_aa6tGVDqXgdy2owvrrJtY6TaCZQ6r+BA7F1fSCA@mail.gmail.com/4-v5-0002-Replace-accesses-to-rmgr-s-XLogRecord-info-bit-ac.patch) download | inline diff: From c4a93d778478c8140ff7d57f26e0dafe20aba3a0 Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Mon, 23 Jan 2023 16:24:01 +0100 Subject: [PATCH v5 2/7] Replace accesses to rmgr's XLogRecord info bit accesses with XLogRecGetRmgrInfo This allows refactoring of the info bits into its own struct field, instead of sharing the field with the records own info bits. --- contrib/pg_walinspect/pg_walinspect.c | 8 ++--- src/backend/access/brin/brin_xlog.c | 4 +-- src/backend/access/gin/ginxlog.c | 2 +- src/backend/access/gist/gistxlog.c | 2 +- src/backend/access/hash/hash_xlog.c | 2 +- src/backend/access/heap/heapam.c | 10 +++---- src/backend/access/nbtree/nbtxlog.c | 2 +- src/backend/access/rmgrdesc/brindesc.c | 2 +- src/backend/access/rmgrdesc/clogdesc.c | 2 +- src/backend/access/rmgrdesc/committsdesc.c | 2 +- src/backend/access/rmgrdesc/dbasedesc.c | 2 +- src/backend/access/rmgrdesc/gindesc.c | 2 +- src/backend/access/rmgrdesc/gistdesc.c | 2 +- src/backend/access/rmgrdesc/hashdesc.c | 2 +- src/backend/access/rmgrdesc/heapdesc.c | 6 ++-- src/backend/access/rmgrdesc/logicalmsgdesc.c | 2 +- src/backend/access/rmgrdesc/mxactdesc.c | 2 +- src/backend/access/rmgrdesc/nbtdesc.c | 2 +- src/backend/access/rmgrdesc/relmapdesc.c | 2 +- src/backend/access/rmgrdesc/replorigindesc.c | 2 +- src/backend/access/rmgrdesc/seqdesc.c | 2 +- src/backend/access/rmgrdesc/smgrdesc.c | 2 +- src/backend/access/rmgrdesc/spgdesc.c | 2 +- src/backend/access/rmgrdesc/standbydesc.c | 2 +- src/backend/access/rmgrdesc/tblspcdesc.c | 2 +- src/backend/access/rmgrdesc/xactdesc.c | 30 +++++++++---------- src/backend/access/rmgrdesc/xlogdesc.c | 2 +- src/backend/access/spgist/spgxlog.c | 2 +- src/backend/access/transam/clog.c | 2 +- src/backend/access/transam/commit_ts.c | 2 +- src/backend/access/transam/multixact.c | 2 +- src/backend/access/transam/twophase.c | 2 +- src/backend/access/transam/xact.c | 10 +++---- src/backend/access/transam/xlog.c | 2 +- src/backend/access/transam/xlogrecovery.c | 22 +++++++------- src/backend/access/transam/xlogstats.c | 2 +- src/backend/catalog/storage.c | 2 +- src/backend/commands/dbcommands.c | 2 +- src/backend/commands/sequence.c | 2 +- src/backend/commands/tablespace.c | 2 +- src/backend/replication/logical/decode.c | 20 +++++++------ src/backend/replication/logical/message.c | 2 +- src/backend/replication/logical/origin.c | 2 +- src/backend/storage/ipc/standby.c | 2 +- src/backend/utils/cache/relmapper.c | 2 +- src/bin/pg_rewind/parsexlog.c | 6 ++-- src/bin/pg_waldump/pg_waldump.c | 4 +-- src/include/access/xact.h | 6 ++-- src/include/access/xlogreader.h | 3 +- .../test_custom_rmgrs/test_custom_rmgrs.c | 4 +-- 50 files changed, 105 insertions(+), 102 deletions(-) diff --git a/contrib/pg_walinspect/pg_walinspect.c b/contrib/pg_walinspect/pg_walinspect.c index 796a74f322..1b6f4cb178 100644 --- a/contrib/pg_walinspect/pg_walinspect.c +++ b/contrib/pg_walinspect/pg_walinspect.c @@ -196,10 +196,10 @@ GetWALRecordInfo(XLogReaderState *record, Datum *values, int i = 0; desc = GetRmgr(XLogRecGetRmid(record)); - record_type = desc.rm_identify(XLogRecGetInfo(record)); + record_type = desc.rm_identify(XLogRecGetRmgrInfo(record)); if (record_type == NULL) - record_type = psprintf("UNKNOWN (%x)", XLogRecGetInfo(record) & ~XLR_INFO_MASK); + record_type = psprintf("UNKNOWN (%x)", XLogRecGetRmgrInfo(record)); initStringInfo(&rec_desc); desc.rm_desc(&rec_desc, record); @@ -258,11 +258,11 @@ GetWALBlockInfo(FunctionCallInfo fcinfo, XLogReaderState *record, Assert(XLogRecHasAnyBlockRefs(record)); desc = GetRmgr(XLogRecGetRmid(record)); - record_type = desc.rm_identify(XLogRecGetInfo(record)); + record_type = desc.rm_identify(XLogRecGetRmgrInfo(record)); if (record_type == NULL) record_type = psprintf("UNKNOWN (%x)", - XLogRecGetInfo(record) & ~XLR_INFO_MASK); + XLogRecGetRmgrInfo(record)); initStringInfo(&rec_desc); desc.rm_desc(&rec_desc, record); diff --git a/src/backend/access/brin/brin_xlog.c b/src/backend/access/brin/brin_xlog.c index 89145b68f6..00dd85bdc5 100644 --- a/src/backend/access/brin/brin_xlog.c +++ b/src/backend/access/brin/brin_xlog.c @@ -56,7 +56,7 @@ brin_xlog_insert_update(XLogReaderState *record, * If we inserted the first and only tuple on the page, re-initialize the * page from scratch. */ - if (XLogRecGetInfo(record) & XLOG_BRIN_INIT_PAGE) + if (XLogRecGetRmgrInfo(record) & XLOG_BRIN_INIT_PAGE) { buffer = XLogInitBufferForRedo(record, 0); page = BufferGetPage(buffer); @@ -308,7 +308,7 @@ brin_xlog_desummarize_page(XLogReaderState *record) void brin_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info & XLOG_BRIN_OPMASK) { diff --git a/src/backend/access/gin/ginxlog.c b/src/backend/access/gin/ginxlog.c index f7c84beef8..feb675ed19 100644 --- a/src/backend/access/gin/ginxlog.c +++ b/src/backend/access/gin/ginxlog.c @@ -725,7 +725,7 @@ ginRedoDeleteListPages(XLogReaderState *record) void gin_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); MemoryContext oldCtx; /* diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index 15249aa921..a49210b64f 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -398,7 +398,7 @@ gistRedoPageReuse(XLogReaderState *record) void gist_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); MemoryContext oldCxt; /* diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index e8e06c62a9..3a45b7ee21 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -1048,7 +1048,7 @@ hash_xlog_vacuum_one_page(XLogReaderState *record) void hash_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 88a123d38a..e61a6f67d5 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -9247,7 +9247,7 @@ heap_xlog_insert(XLogReaderState *record) * If we inserted the first and only tuple on the page, re-initialize the * page from scratch. */ - if (XLogRecGetInfo(record) & XLOG_HEAP_INIT_PAGE) + if (XLogRecGetRmgrInfo(record) & XLOG_HEAP_INIT_PAGE) { buffer = XLogInitBufferForRedo(record, 0); page = BufferGetPage(buffer); @@ -9341,7 +9341,7 @@ heap_xlog_multi_insert(XLogReaderState *record) uint32 newlen; Size freespace = 0; int i; - bool isinit = (XLogRecGetInfo(record) & XLOG_HEAP_INIT_PAGE) != 0; + bool isinit = (XLogRecGetRmgrInfo(record) & XLOG_HEAP_INIT_PAGE) != 0; XLogRedoAction action; /* @@ -9589,7 +9589,7 @@ heap_xlog_update(XLogReaderState *record, bool hot_update) nbuffer = obuffer; newaction = oldaction; } - else if (XLogRecGetInfo(record) & XLOG_HEAP_INIT_PAGE) + else if (XLogRecGetRmgrInfo(record) & XLOG_HEAP_INIT_PAGE) { nbuffer = XLogInitBufferForRedo(record, 0); page = (Page) BufferGetPage(nbuffer); @@ -9953,7 +9953,7 @@ heap_xlog_inplace(XLogReaderState *record) void heap_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* * These operations don't overwrite MVCC data so no conflict processing is @@ -9999,7 +9999,7 @@ heap_redo(XLogReaderState *record) void heap2_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info & XLOG_HEAP_OPMASK) { diff --git a/src/backend/access/nbtree/nbtxlog.c b/src/backend/access/nbtree/nbtxlog.c index c87e46ed66..97d8d068c6 100644 --- a/src/backend/access/nbtree/nbtxlog.c +++ b/src/backend/access/nbtree/nbtxlog.c @@ -1015,7 +1015,7 @@ btree_xlog_reuse_page(XLogReaderState *record) void btree_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); MemoryContext oldCtx; oldCtx = MemoryContextSwitchTo(opCtx); diff --git a/src/backend/access/rmgrdesc/brindesc.c b/src/backend/access/rmgrdesc/brindesc.c index 1466a31bbb..1d3e5ee03e 100644 --- a/src/backend/access/rmgrdesc/brindesc.c +++ b/src/backend/access/rmgrdesc/brindesc.c @@ -20,7 +20,7 @@ void brin_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); info &= XLOG_BRIN_OPMASK; if (info == XLOG_BRIN_CREATE_INDEX) diff --git a/src/backend/access/rmgrdesc/clogdesc.c b/src/backend/access/rmgrdesc/clogdesc.c index e60b76f9da..915f9945d0 100644 --- a/src/backend/access/rmgrdesc/clogdesc.c +++ b/src/backend/access/rmgrdesc/clogdesc.c @@ -21,7 +21,7 @@ void clog_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == CLOG_ZEROPAGE) { diff --git a/src/backend/access/rmgrdesc/committsdesc.c b/src/backend/access/rmgrdesc/committsdesc.c index e7155cd507..3d673c2302 100644 --- a/src/backend/access/rmgrdesc/committsdesc.c +++ b/src/backend/access/rmgrdesc/committsdesc.c @@ -22,7 +22,7 @@ void commit_ts_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == COMMIT_TS_ZEROPAGE) { diff --git a/src/backend/access/rmgrdesc/dbasedesc.c b/src/backend/access/rmgrdesc/dbasedesc.c index 3922120d64..a9529bde05 100644 --- a/src/backend/access/rmgrdesc/dbasedesc.c +++ b/src/backend/access/rmgrdesc/dbasedesc.c @@ -22,7 +22,7 @@ void dbase_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_DBASE_CREATE_FILE_COPY) { diff --git a/src/backend/access/rmgrdesc/gindesc.c b/src/backend/access/rmgrdesc/gindesc.c index 246a6a6b85..08fc7ba49b 100644 --- a/src/backend/access/rmgrdesc/gindesc.c +++ b/src/backend/access/rmgrdesc/gindesc.c @@ -74,7 +74,7 @@ void gin_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/access/rmgrdesc/gistdesc.c b/src/backend/access/rmgrdesc/gistdesc.c index 5dc6e1dcee..6473069c1a 100644 --- a/src/backend/access/rmgrdesc/gistdesc.c +++ b/src/backend/access/rmgrdesc/gistdesc.c @@ -60,7 +60,7 @@ void gist_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/access/rmgrdesc/hashdesc.c b/src/backend/access/rmgrdesc/hashdesc.c index b6810a9320..bcb2013048 100644 --- a/src/backend/access/rmgrdesc/hashdesc.c +++ b/src/backend/access/rmgrdesc/hashdesc.c @@ -20,7 +20,7 @@ void hash_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/access/rmgrdesc/heapdesc.c b/src/backend/access/rmgrdesc/heapdesc.c index d73248abdd..15f943051a 100644 --- a/src/backend/access/rmgrdesc/heapdesc.c +++ b/src/backend/access/rmgrdesc/heapdesc.c @@ -95,7 +95,7 @@ void heap_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); info &= XLOG_HEAP_OPMASK; if (info == XLOG_HEAP_INSERT) @@ -172,7 +172,7 @@ void heap2_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); info &= XLOG_HEAP_OPMASK; if (info == XLOG_HEAP2_PRUNE) @@ -265,7 +265,7 @@ heap2_desc(StringInfo buf, XLogReaderState *record) else if (info == XLOG_HEAP2_MULTI_INSERT) { xl_heap_multi_insert *xlrec = (xl_heap_multi_insert *) rec; - bool isinit = (XLogRecGetInfo(record) & XLOG_HEAP_INIT_PAGE) != 0; + bool isinit = (XLogRecGetRmgrInfo(record) & XLOG_HEAP_INIT_PAGE) != 0; appendStringInfo(buf, "ntuples: %d, flags: 0x%02X", xlrec->ntuples, xlrec->flags); diff --git a/src/backend/access/rmgrdesc/logicalmsgdesc.c b/src/backend/access/rmgrdesc/logicalmsgdesc.c index 0578ec5535..2bb6cb06c3 100644 --- a/src/backend/access/rmgrdesc/logicalmsgdesc.c +++ b/src/backend/access/rmgrdesc/logicalmsgdesc.c @@ -19,7 +19,7 @@ void logicalmsg_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_LOGICAL_MESSAGE) { diff --git a/src/backend/access/rmgrdesc/mxactdesc.c b/src/backend/access/rmgrdesc/mxactdesc.c index a2fa1eca18..cdcf8a4f45 100644 --- a/src/backend/access/rmgrdesc/mxactdesc.c +++ b/src/backend/access/rmgrdesc/mxactdesc.c @@ -50,7 +50,7 @@ void multixact_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_MULTIXACT_ZERO_OFF_PAGE || info == XLOG_MULTIXACT_ZERO_MEM_PAGE) diff --git a/src/backend/access/rmgrdesc/nbtdesc.c b/src/backend/access/rmgrdesc/nbtdesc.c index e4fbaa4d5d..ede8df72c0 100644 --- a/src/backend/access/rmgrdesc/nbtdesc.c +++ b/src/backend/access/rmgrdesc/nbtdesc.c @@ -24,7 +24,7 @@ void btree_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/access/rmgrdesc/relmapdesc.c b/src/backend/access/rmgrdesc/relmapdesc.c index 82ad51eb9a..f4f0e5ce8b 100644 --- a/src/backend/access/rmgrdesc/relmapdesc.c +++ b/src/backend/access/rmgrdesc/relmapdesc.c @@ -20,7 +20,7 @@ void relmap_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_RELMAP_UPDATE) { diff --git a/src/backend/access/rmgrdesc/replorigindesc.c b/src/backend/access/rmgrdesc/replorigindesc.c index 8ce4588fda..539dc57939 100644 --- a/src/backend/access/rmgrdesc/replorigindesc.c +++ b/src/backend/access/rmgrdesc/replorigindesc.c @@ -19,7 +19,7 @@ void replorigin_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/access/rmgrdesc/seqdesc.c b/src/backend/access/rmgrdesc/seqdesc.c index ba60544085..6def7b653b 100644 --- a/src/backend/access/rmgrdesc/seqdesc.c +++ b/src/backend/access/rmgrdesc/seqdesc.c @@ -21,7 +21,7 @@ void seq_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); xl_seq_rec *xlrec = (xl_seq_rec *) rec; if (info == XLOG_SEQ_LOG) diff --git a/src/backend/access/rmgrdesc/smgrdesc.c b/src/backend/access/rmgrdesc/smgrdesc.c index bd841b96e8..8604bee0c0 100644 --- a/src/backend/access/rmgrdesc/smgrdesc.c +++ b/src/backend/access/rmgrdesc/smgrdesc.c @@ -21,7 +21,7 @@ void smgr_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_SMGR_CREATE) { diff --git a/src/backend/access/rmgrdesc/spgdesc.c b/src/backend/access/rmgrdesc/spgdesc.c index 87f62f0fb4..9e1e48240b 100644 --- a/src/backend/access/rmgrdesc/spgdesc.c +++ b/src/backend/access/rmgrdesc/spgdesc.c @@ -20,7 +20,7 @@ void spg_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/access/rmgrdesc/standbydesc.c b/src/backend/access/rmgrdesc/standbydesc.c index f2bce9a37a..9a08e10fe1 100644 --- a/src/backend/access/rmgrdesc/standbydesc.c +++ b/src/backend/access/rmgrdesc/standbydesc.c @@ -47,7 +47,7 @@ void standby_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_STANDBY_LOCK) { diff --git a/src/backend/access/rmgrdesc/tblspcdesc.c b/src/backend/access/rmgrdesc/tblspcdesc.c index b8c89f8c54..c1feb4fa05 100644 --- a/src/backend/access/rmgrdesc/tblspcdesc.c +++ b/src/backend/access/rmgrdesc/tblspcdesc.c @@ -21,7 +21,7 @@ void tblspc_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_TBLSPC_CREATE) { diff --git a/src/backend/access/rmgrdesc/xactdesc.c b/src/backend/access/rmgrdesc/xactdesc.c index 01610c5ddc..8903bb32de 100644 --- a/src/backend/access/rmgrdesc/xactdesc.c +++ b/src/backend/access/rmgrdesc/xactdesc.c @@ -32,7 +32,7 @@ */ void -ParseCommitRecord(uint8 info, xl_xact_commit *xlrec, xl_xact_parsed_commit *parsed) +ParseCommitRecord(uint8 rmgrinfo, xl_xact_commit *xlrec, xl_xact_parsed_commit *parsed) { char *data = ((char *) xlrec) + MinSizeOfXactCommit; @@ -43,7 +43,7 @@ ParseCommitRecord(uint8 info, xl_xact_commit *xlrec, xl_xact_parsed_commit *pars parsed->xact_time = xlrec->xact_time; - if (info & XLOG_XACT_HAS_INFO) + if (rmgrinfo & XLOG_XACT_HAS_INFO) { xl_xact_xinfo *xl_xinfo = (xl_xact_xinfo *) data; @@ -138,7 +138,7 @@ ParseCommitRecord(uint8 info, xl_xact_commit *xlrec, xl_xact_parsed_commit *pars } void -ParseAbortRecord(uint8 info, xl_xact_abort *xlrec, xl_xact_parsed_abort *parsed) +ParseAbortRecord(uint8 rmgrinfo, xl_xact_abort *xlrec, xl_xact_parsed_abort *parsed) { char *data = ((char *) xlrec) + MinSizeOfXactAbort; @@ -149,7 +149,7 @@ ParseAbortRecord(uint8 info, xl_xact_abort *xlrec, xl_xact_parsed_abort *parsed) parsed->xact_time = xlrec->xact_time; - if (info & XLOG_XACT_HAS_INFO) + if (rmgrinfo & XLOG_XACT_HAS_INFO) { xl_xact_xinfo *xl_xinfo = (xl_xact_xinfo *) data; @@ -236,7 +236,7 @@ ParseAbortRecord(uint8 info, xl_xact_abort *xlrec, xl_xact_parsed_abort *parsed) * ParsePrepareRecord */ void -ParsePrepareRecord(uint8 info, xl_xact_prepare *xlrec, xl_xact_parsed_prepare *parsed) +ParsePrepareRecord(uint8 rmgrinfo, xl_xact_prepare *xlrec, xl_xact_parsed_prepare *parsed) { char *bufptr; @@ -328,11 +328,11 @@ xact_desc_stats(StringInfo buf, const char *label, } static void -xact_desc_commit(StringInfo buf, uint8 info, xl_xact_commit *xlrec, RepOriginId origin_id) +xact_desc_commit(StringInfo buf, uint8 rmgrinfo, xl_xact_commit *xlrec, RepOriginId origin_id) { xl_xact_parsed_commit parsed; - ParseCommitRecord(info, xlrec, &parsed); + ParseCommitRecord(rmgrinfo, xlrec, &parsed); /* If this is a prepared xact, show the xid of the original xact */ if (TransactionIdIsValid(parsed.twophase_xid)) @@ -364,11 +364,11 @@ xact_desc_commit(StringInfo buf, uint8 info, xl_xact_commit *xlrec, RepOriginId } static void -xact_desc_abort(StringInfo buf, uint8 info, xl_xact_abort *xlrec, RepOriginId origin_id) +xact_desc_abort(StringInfo buf, uint8 rmgrinfo, xl_xact_abort *xlrec, RepOriginId origin_id) { xl_xact_parsed_abort parsed; - ParseAbortRecord(info, xlrec, &parsed); + ParseAbortRecord(rmgrinfo, xlrec, &parsed); /* If this is a prepared xact, show the xid of the original xact */ if (TransactionIdIsValid(parsed.twophase_xid)) @@ -391,11 +391,11 @@ xact_desc_abort(StringInfo buf, uint8 info, xl_xact_abort *xlrec, RepOriginId or } static void -xact_desc_prepare(StringInfo buf, uint8 info, xl_xact_prepare *xlrec, RepOriginId origin_id) +xact_desc_prepare(StringInfo buf, uint8 rmgrinfo, xl_xact_prepare *xlrec, RepOriginId origin_id) { xl_xact_parsed_prepare parsed; - ParsePrepareRecord(info, xlrec, &parsed); + ParsePrepareRecord(rmgrinfo, xlrec, &parsed); appendStringInfo(buf, "gid %s: ", parsed.twophase_gid); appendStringInfoString(buf, timestamptz_to_str(parsed.xact_time)); @@ -436,27 +436,27 @@ void xact_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & XLOG_XACT_OPMASK; + uint8 info = XLogRecGetRmgrInfo(record) & XLOG_XACT_OPMASK; if (info == XLOG_XACT_COMMIT || info == XLOG_XACT_COMMIT_PREPARED) { xl_xact_commit *xlrec = (xl_xact_commit *) rec; - xact_desc_commit(buf, XLogRecGetInfo(record), xlrec, + xact_desc_commit(buf, XLogRecGetRmgrInfo(record), xlrec, XLogRecGetOrigin(record)); } else if (info == XLOG_XACT_ABORT || info == XLOG_XACT_ABORT_PREPARED) { xl_xact_abort *xlrec = (xl_xact_abort *) rec; - xact_desc_abort(buf, XLogRecGetInfo(record), xlrec, + xact_desc_abort(buf, XLogRecGetRmgrInfo(record), xlrec, XLogRecGetOrigin(record)); } else if (info == XLOG_XACT_PREPARE) { xl_xact_prepare *xlrec = (xl_xact_prepare *) rec; - xact_desc_prepare(buf, XLogRecGetInfo(record), xlrec, + xact_desc_prepare(buf, XLogRecGetRmgrInfo(record), xlrec, XLogRecGetOrigin(record)); } else if (info == XLOG_XACT_ASSIGNMENT) diff --git a/src/backend/access/rmgrdesc/xlogdesc.c b/src/backend/access/rmgrdesc/xlogdesc.c index f390c177e4..3692dfb96a 100644 --- a/src/backend/access/rmgrdesc/xlogdesc.c +++ b/src/backend/access/rmgrdesc/xlogdesc.c @@ -37,7 +37,7 @@ void xlog_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_CHECKPOINT_SHUTDOWN || info == XLOG_CHECKPOINT_ONLINE) diff --git a/src/backend/access/spgist/spgxlog.c b/src/backend/access/spgist/spgxlog.c index 459ac929ba..53190a67f9 100644 --- a/src/backend/access/spgist/spgxlog.c +++ b/src/backend/access/spgist/spgxlog.c @@ -936,7 +936,7 @@ spgRedoVacuumRedirect(XLogReaderState *record) void spg_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); MemoryContext oldCxt; oldCxt = MemoryContextSwitchTo(opCtx); diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c index 4a431d5876..11c10b936d 100644 --- a/src/backend/access/transam/clog.c +++ b/src/backend/access/transam/clog.c @@ -984,7 +984,7 @@ WriteTruncateXlogRec(int pageno, TransactionId oldestXact, Oid oldestXactDb) void clog_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in clog records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/backend/access/transam/commit_ts.c b/src/backend/access/transam/commit_ts.c index b897fabc70..a351ee84ce 100644 --- a/src/backend/access/transam/commit_ts.c +++ b/src/backend/access/transam/commit_ts.c @@ -966,7 +966,7 @@ WriteTruncateXlogRec(int pageno, TransactionId oldestXid) void commit_ts_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in commit_ts records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c index abb022e067..c70b6b7358 100644 --- a/src/backend/access/transam/multixact.c +++ b/src/backend/access/transam/multixact.c @@ -3232,7 +3232,7 @@ WriteMTruncateXlogRec(Oid oldestMultiDB, void multixact_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in multixact records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c index c6af8cfd7e..cf852a2668 100644 --- a/src/backend/access/transam/twophase.c +++ b/src/backend/access/transam/twophase.c @@ -1430,7 +1430,7 @@ XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len) } if (XLogRecGetRmid(xlogreader) != RM_XACT_ID || - (XLogRecGetInfo(xlogreader) & XLOG_XACT_OPMASK) != XLOG_XACT_PREPARE) + (XLogRecGetRmgrInfo(xlogreader) & XLOG_XACT_OPMASK) != XLOG_XACT_PREPARE) ereport(ERROR, (errcode_for_file_access(), errmsg("expected two-phase state data is not present in WAL at %X/%X", diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index 8daaa535ed..c560f07390 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -6183,7 +6183,7 @@ xact_redo_abort(xl_xact_parsed_abort *parsed, TransactionId xid, void xact_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & XLOG_XACT_OPMASK; + uint8 info = XLogRecGetRmgrInfo(record) & XLOG_XACT_OPMASK; /* Backup blocks are not used in xact records */ Assert(!XLogRecHasAnyBlockRefs(record)); @@ -6193,7 +6193,7 @@ xact_redo(XLogReaderState *record) xl_xact_commit *xlrec = (xl_xact_commit *) XLogRecGetData(record); xl_xact_parsed_commit parsed; - ParseCommitRecord(XLogRecGetInfo(record), xlrec, &parsed); + ParseCommitRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); xact_redo_commit(&parsed, XLogRecGetXid(record), record->EndRecPtr, XLogRecGetOrigin(record)); } @@ -6202,7 +6202,7 @@ xact_redo(XLogReaderState *record) xl_xact_commit *xlrec = (xl_xact_commit *) XLogRecGetData(record); xl_xact_parsed_commit parsed; - ParseCommitRecord(XLogRecGetInfo(record), xlrec, &parsed); + ParseCommitRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); xact_redo_commit(&parsed, parsed.twophase_xid, record->EndRecPtr, XLogRecGetOrigin(record)); @@ -6216,7 +6216,7 @@ xact_redo(XLogReaderState *record) xl_xact_abort *xlrec = (xl_xact_abort *) XLogRecGetData(record); xl_xact_parsed_abort parsed; - ParseAbortRecord(XLogRecGetInfo(record), xlrec, &parsed); + ParseAbortRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); xact_redo_abort(&parsed, XLogRecGetXid(record), record->EndRecPtr, XLogRecGetOrigin(record)); } @@ -6225,7 +6225,7 @@ xact_redo(XLogReaderState *record) xl_xact_abort *xlrec = (xl_xact_abort *) XLogRecGetData(record); xl_xact_parsed_abort parsed; - ParseAbortRecord(XLogRecGetInfo(record), xlrec, &parsed); + ParseAbortRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); xact_redo_abort(&parsed, parsed.twophase_xid, record->EndRecPtr, XLogRecGetOrigin(record)); diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index fcbde10529..08c78bf57b 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -7742,7 +7742,7 @@ UpdateFullPageWrites(void) void xlog_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); XLogRecPtr lsn = record->EndRecPtr; /* diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index becc2bda62..6508246999 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -1995,7 +1995,7 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl static void xlogrecovery_redo(XLogReaderState *record, TimeLineID replayTLI) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); XLogRecPtr lsn = record->EndRecPtr; Assert(XLogRecGetRmid(record) == RM_XLOG_ID); @@ -2213,7 +2213,7 @@ void xlog_outdesc(StringInfo buf, XLogReaderState *record) { RmgrData rmgr = GetRmgr(XLogRecGetRmid(record)); - uint8 info = XLogRecGetInfo(record); + uint8 info = XLogRecGetRmgrInfo(record); const char *id; appendStringInfoString(buf, rmgr.rm_name); @@ -2221,7 +2221,7 @@ xlog_outdesc(StringInfo buf, XLogReaderState *record) id = rmgr.rm_identify(info); if (id == NULL) - appendStringInfo(buf, "UNKNOWN (%X): ", info & ~XLR_INFO_MASK); + appendStringInfo(buf, "UNKNOWN (%X): ", info); else appendStringInfo(buf, "%s: ", id); @@ -2341,7 +2341,7 @@ checkTimeLineSwitch(XLogRecPtr lsn, TimeLineID newTLI, TimeLineID prevTLI, static bool getRecordTimestamp(XLogReaderState *record, TimestampTz *recordXtime) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); uint8 xact_info = info & XLOG_XACT_OPMASK; uint8 rmid = XLogRecGetRmid(record); @@ -2535,7 +2535,7 @@ recoveryStopsBefore(XLogReaderState *record) if (XLogRecGetRmid(record) != RM_XACT_ID) return false; - xact_info = XLogRecGetInfo(record) & XLOG_XACT_OPMASK; + xact_info = XLogRecGetRmgrInfo(record); if (xact_info == XLOG_XACT_COMMIT) { @@ -2548,7 +2548,7 @@ recoveryStopsBefore(XLogReaderState *record) xl_xact_parsed_commit parsed; isCommit = true; - ParseCommitRecord(XLogRecGetInfo(record), + ParseCommitRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); recordXid = parsed.twophase_xid; @@ -2564,7 +2564,7 @@ recoveryStopsBefore(XLogReaderState *record) xl_xact_parsed_abort parsed; isCommit = false; - ParseAbortRecord(XLogRecGetInfo(record), + ParseAbortRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); recordXid = parsed.twophase_xid; @@ -2653,7 +2653,7 @@ recoveryStopsAfter(XLogReaderState *record) if (!ArchiveRecoveryRequested) return false; - info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + info = XLogRecGetRmgrInfo(record); rmid = XLogRecGetRmid(record); /* @@ -2721,7 +2721,7 @@ recoveryStopsAfter(XLogReaderState *record) xl_xact_commit *xlrec = (xl_xact_commit *) XLogRecGetData(record); xl_xact_parsed_commit parsed; - ParseCommitRecord(XLogRecGetInfo(record), + ParseCommitRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); recordXid = parsed.twophase_xid; @@ -2731,7 +2731,7 @@ recoveryStopsAfter(XLogReaderState *record) xl_xact_abort *xlrec = (xl_xact_abort *) XLogRecGetData(record); xl_xact_parsed_abort parsed; - ParseAbortRecord(XLogRecGetInfo(record), + ParseAbortRecord(XLogRecGetRmgrInfo(record), xlrec, &parsed); recordXid = parsed.twophase_xid; @@ -2925,7 +2925,7 @@ recoveryApplyDelay(XLogReaderState *record) if (XLogRecGetRmid(record) != RM_XACT_ID) return false; - xact_info = XLogRecGetInfo(record) & XLOG_XACT_OPMASK; + xact_info = XLogRecGetRmgrInfo(record) & XLOG_XACT_OPMASK; if (xact_info != XLOG_XACT_COMMIT && xact_info != XLOG_XACT_COMMIT_PREPARED) diff --git a/src/backend/access/transam/xlogstats.c b/src/backend/access/transam/xlogstats.c index 2d315df67a..267b108943 100644 --- a/src/backend/access/transam/xlogstats.c +++ b/src/backend/access/transam/xlogstats.c @@ -79,7 +79,7 @@ XLogRecStoreStats(XLogStats *stats, XLogReaderState *record) * RmgrId). */ - recid = XLogRecGetInfo(record) >> 4; + recid = XLogRecGetRmgrInfo(record) >> 4; /* * XACT records need to be handled differently. Those records use the diff --git a/src/backend/catalog/storage.c b/src/backend/catalog/storage.c index 93f07e49b7..9ce4d9a920 100644 --- a/src/backend/catalog/storage.c +++ b/src/backend/catalog/storage.c @@ -958,7 +958,7 @@ void smgr_redo(XLogReaderState *record) { XLogRecPtr lsn = record->EndRecPtr; - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in smgr records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c index 307729ab7e..bc061a697d 100644 --- a/src/backend/commands/dbcommands.c +++ b/src/backend/commands/dbcommands.c @@ -3179,7 +3179,7 @@ recovery_create_dbdir(char *path, bool only_tblspc) void dbase_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in dbase records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c index 47acdf5166..efd0bd4f02 100644 --- a/src/backend/commands/sequence.c +++ b/src/backend/commands/sequence.c @@ -1816,7 +1816,7 @@ void seq_redo(XLogReaderState *record) { XLogRecPtr lsn = record->EndRecPtr; - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); Buffer buffer; Page page; Page localpage; diff --git a/src/backend/commands/tablespace.c b/src/backend/commands/tablespace.c index 13b0dee146..1de6803df7 100644 --- a/src/backend/commands/tablespace.c +++ b/src/backend/commands/tablespace.c @@ -1516,7 +1516,7 @@ get_tablespace_name(Oid spc_oid) void tblspc_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in tblspc records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c index 730061c9da..1878dad9c8 100644 --- a/src/backend/replication/logical/decode.c +++ b/src/backend/replication/logical/decode.c @@ -132,7 +132,7 @@ void xlog_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) { SnapBuild *builder = ctx->snapshot_builder; - uint8 info = XLogRecGetInfo(buf->record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(buf->record); ReorderBufferProcessXid(ctx->reorder, XLogRecGetXid(buf->record), buf->origptr); @@ -205,7 +205,7 @@ xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) SnapBuild *builder = ctx->snapshot_builder; ReorderBuffer *reorder = ctx->reorder; XLogReaderState *r = buf->record; - uint8 info = XLogRecGetInfo(r) & XLOG_XACT_OPMASK; + uint8 info = XLogRecGetRmgrInfo(r) & XLOG_XACT_OPMASK; /* * If the snapshot isn't yet fully built, we cannot decode anything, so @@ -225,7 +225,9 @@ xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) bool two_phase = false; xlrec = (xl_xact_commit *) XLogRecGetData(r); - ParseCommitRecord(XLogRecGetInfo(buf->record), xlrec, &parsed); + ParseCommitRecord(XLogRecGetRmgrInfo(buf->record), + xlrec, + &parsed); if (!TransactionIdIsValid(parsed.twophase_xid)) xid = XLogRecGetXid(r); @@ -253,7 +255,7 @@ xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) bool two_phase = false; xlrec = (xl_xact_abort *) XLogRecGetData(r); - ParseAbortRecord(XLogRecGetInfo(buf->record), xlrec, &parsed); + ParseAbortRecord(XLogRecGetRmgrInfo(buf->record), xlrec, &parsed); if (!TransactionIdIsValid(parsed.twophase_xid)) xid = XLogRecGetXid(r); @@ -316,7 +318,7 @@ xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) /* ok, parse it */ xlrec = (xl_xact_prepare *) XLogRecGetData(r); - ParsePrepareRecord(XLogRecGetInfo(buf->record), + ParsePrepareRecord(XLogRecGetRmgrInfo(buf->record), xlrec, &parsed); /* @@ -361,7 +363,7 @@ standby_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) { SnapBuild *builder = ctx->snapshot_builder; XLogReaderState *r = buf->record; - uint8 info = XLogRecGetInfo(r) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(r); ReorderBufferProcessXid(ctx->reorder, XLogRecGetXid(r), buf->origptr); @@ -405,7 +407,7 @@ standby_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) void heap2_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) { - uint8 info = XLogRecGetInfo(buf->record) & XLOG_HEAP_OPMASK; + uint8 info = XLogRecGetRmgrInfo(buf->record) & XLOG_HEAP_OPMASK; TransactionId xid = XLogRecGetXid(buf->record); SnapBuild *builder = ctx->snapshot_builder; @@ -464,7 +466,7 @@ heap2_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) void heap_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) { - uint8 info = XLogRecGetInfo(buf->record) & XLOG_HEAP_OPMASK; + uint8 info = XLogRecGetRmgrInfo(buf->record) & XLOG_HEAP_OPMASK; TransactionId xid = XLogRecGetXid(buf->record); SnapBuild *builder = ctx->snapshot_builder; @@ -589,7 +591,7 @@ logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf) SnapBuild *builder = ctx->snapshot_builder; XLogReaderState *r = buf->record; TransactionId xid = XLogRecGetXid(r); - uint8 info = XLogRecGetInfo(r) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(r); RepOriginId origin_id = XLogRecGetOrigin(r); Snapshot snapshot = NULL; xl_logical_message *message; diff --git a/src/backend/replication/logical/message.c b/src/backend/replication/logical/message.c index c5de14afc6..c31dc1203f 100644 --- a/src/backend/replication/logical/message.c +++ b/src/backend/replication/logical/message.c @@ -80,7 +80,7 @@ LogLogicalMessage(const char *prefix, const char *message, size_t size, void logicalmsg_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info != XLOG_LOGICAL_MESSAGE) elog(PANIC, "logicalmsg_redo: unknown op code %u", info); diff --git a/src/backend/replication/logical/origin.c b/src/backend/replication/logical/origin.c index b0255ffd25..4977d2eb9a 100644 --- a/src/backend/replication/logical/origin.c +++ b/src/backend/replication/logical/origin.c @@ -826,7 +826,7 @@ StartupReplicationOrigin(void) void replorigin_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); switch (info) { diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c index cc22d2e87c..44338d6de3 100644 --- a/src/backend/storage/ipc/standby.c +++ b/src/backend/storage/ipc/standby.c @@ -1160,7 +1160,7 @@ StandbyReleaseOldLocks(TransactionId oldxid) void standby_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in standby records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/backend/utils/cache/relmapper.c b/src/backend/utils/cache/relmapper.c index 26575cae6c..b825dda353 100644 --- a/src/backend/utils/cache/relmapper.c +++ b/src/backend/utils/cache/relmapper.c @@ -1084,7 +1084,7 @@ perform_relmap_update(bool shared, const RelMapFile *updates) void relmap_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); /* Backup blocks are not used in relmap records */ Assert(!XLogRecHasAnyBlockRefs(record)); diff --git a/src/bin/pg_rewind/parsexlog.c b/src/bin/pg_rewind/parsexlog.c index 27782237d0..2b6b46f0e3 100644 --- a/src/bin/pg_rewind/parsexlog.c +++ b/src/bin/pg_rewind/parsexlog.c @@ -222,7 +222,7 @@ findLastCheckpoint(const char *datadir, XLogRecPtr forkptr, int tliIndex, * be the latest checkpoint before WAL forked and not the checkpoint * where the primary has been stopped to be rewound. */ - info = XLogRecGetInfo(xlogreader) & ~XLR_INFO_MASK; + info = XLogRecGetRmgrInfo(xlogreader); if (searchptr < forkptr && XLogRecGetRmid(xlogreader) == RM_XLOG_ID && (info == XLOG_CHECKPOINT_SHUTDOWN || @@ -370,7 +370,7 @@ extractPageInfo(XLogReaderState *record) int block_id; RmgrId rmid = XLogRecGetRmid(record); uint8 info = XLogRecGetInfo(record); - uint8 rminfo = info & ~XLR_INFO_MASK; + uint8 rminfo = XLogRecGetRmgrInfo(record); /* Is this a special record type that I recognize? */ @@ -440,7 +440,7 @@ extractPageInfo(XLogReaderState *record) pg_fatal("WAL record modifies a relation, but record type is not recognized: " "lsn: %X/%X, rmid: %d, rmgr: %s, info: %02X", LSN_FORMAT_ARGS(record->ReadRecPtr), - rmid, RmgrName(rmid), info); + rmid, RmgrName(rmid), info | rminfo); } for (block_id = 0; block_id <= XLogRecMaxBlockId(record); block_id++) diff --git a/src/bin/pg_waldump/pg_waldump.c b/src/bin/pg_waldump/pg_waldump.c index a3535bdfa9..48b733b767 100644 --- a/src/bin/pg_waldump/pg_waldump.c +++ b/src/bin/pg_waldump/pg_waldump.c @@ -549,7 +549,7 @@ XLogDumpDisplayRecord(XLogDumpConfig *config, XLogReaderState *record) const RmgrDescData *desc = GetRmgrDesc(XLogRecGetRmid(record)); uint32 rec_len; uint32 fpi_len; - uint8 info = XLogRecGetInfo(record); + uint8 info = XLogRecGetRmgrInfo(record); XLogRecPtr xl_prev = XLogRecGetPrev(record); StringInfoData s; @@ -564,7 +564,7 @@ XLogDumpDisplayRecord(XLogDumpConfig *config, XLogReaderState *record) id = desc->rm_identify(info); if (id == NULL) - printf("desc: UNKNOWN (%x) ", info & ~XLR_INFO_MASK); + printf("desc: UNKNOWN (%x) ", info); else printf("desc: %s ", id); diff --git a/src/include/access/xact.h b/src/include/access/xact.h index 7d3b9446e6..b94b264402 100644 --- a/src/include/access/xact.h +++ b/src/include/access/xact.h @@ -519,9 +519,9 @@ extern void xact_desc(StringInfo buf, XLogReaderState *record); extern const char *xact_identify(uint8 info); /* also in xactdesc.c, so they can be shared between front/backend code */ -extern void ParseCommitRecord(uint8 info, xl_xact_commit *xlrec, xl_xact_parsed_commit *parsed); -extern void ParseAbortRecord(uint8 info, xl_xact_abort *xlrec, xl_xact_parsed_abort *parsed); -extern void ParsePrepareRecord(uint8 info, xl_xact_prepare *xlrec, xl_xact_parsed_prepare *parsed); +extern void ParseCommitRecord(uint8 rmgrinfo, xl_xact_commit *xlrec, xl_xact_parsed_commit *parsed); +extern void ParseAbortRecord(uint8 rmgrinfo, xl_xact_abort *xlrec, xl_xact_parsed_abort *parsed); +extern void ParsePrepareRecord(uint8 rmgrinfo, xl_xact_prepare *xlrec, xl_xact_parsed_prepare *parsed); extern void EnterParallelMode(void); extern void ExitParallelMode(void); diff --git a/src/include/access/xlogreader.h b/src/include/access/xlogreader.h index da32c7db77..2b976dcc03 100644 --- a/src/include/access/xlogreader.h +++ b/src/include/access/xlogreader.h @@ -407,7 +407,8 @@ extern bool DecodeXLogRecord(XLogReaderState *state, */ #define XLogRecGetTotalLen(decoder) ((decoder)->record->header.xl_tot_len) #define XLogRecGetPrev(decoder) ((decoder)->record->header.xl_prev) -#define XLogRecGetInfo(decoder) ((decoder)->record->header.xl_info) +#define XLogRecGetInfo(decoder) ((decoder)->record->header.xl_info & XLR_INFO_MASK) +#define XLogRecGetRmgrInfo(decoder) (((decoder)->record->header.xl_info) & XLR_RMGR_INFO_MASK) #define XLogRecGetRmid(decoder) ((decoder)->record->header.xl_rmid) #define XLogRecGetXid(decoder) ((decoder)->record->header.xl_xid) #define XLogRecGetOrigin(decoder) ((decoder)->record->record_origin) diff --git a/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c b/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c index a304ba54bb..b5537a5f5b 100644 --- a/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c +++ b/src/test/modules/test_custom_rmgrs/test_custom_rmgrs.c @@ -81,7 +81,7 @@ _PG_init(void) void testcustomrmgrs_redo(XLogReaderState *record) { - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info != XLOG_TEST_CUSTOM_RMGRS_MESSAGE) elog(PANIC, "testcustomrmgrs_redo: unknown op code %u", info); @@ -91,7 +91,7 @@ void testcustomrmgrs_desc(StringInfo buf, XLogReaderState *record) { char *rec = XLogRecGetData(record); - uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK; + uint8 info = XLogRecGetRmgrInfo(record); if (info == XLOG_TEST_CUSTOM_RMGRS_MESSAGE) { -- 2.40.1 [application/octet-stream] v5-0001-Reduce-overhead-of-small-block-data-in-xlog-recor.patch (11.5K, ../../CAEze2WgQ9d_aa6tGVDqXgdy2owvrrJtY6TaCZQ6r+BA7F1fSCA@mail.gmail.com/5-v5-0001-Reduce-overhead-of-small-block-data-in-xlog-recor.patch) download | inline diff: From ab37830c8af766d34bfcafa2b5653562debe65a5 Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Tue, 24 Jan 2023 12:37:09 +0100 Subject: [PATCH v5 1/7] Reduce overhead of small block data in xlog records We reserve 2 bits in the block ID header for how large the block's registered data is: 0, 1 or 2 bytes, for 0, 1-255 and 256-UINT16_MAX bytes to the block, respectively. This reduces the minimum size of any block-referencing xlog record from 48 bytes to 46 bytes, and reduces the size of any non-hot heap update record by at least 2 bytes. It includes more infrastructure for future value-dependent encoding of length fields, even though it is currently only used in one place. --- src/backend/access/transam/xloginsert.c | 13 ++- src/backend/access/transam/xlogreader.c | 34 ++++++- src/include/access/xlog_internal.h | 129 ++++++++++++++++++++++++ src/include/access/xlogrecord.h | 44 ++++++-- 4 files changed, 204 insertions(+), 16 deletions(-) diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c index 258cbd7035..51b487887b 100644 --- a/src/backend/access/transam/xloginsert.c +++ b/src/backend/access/transam/xloginsert.c @@ -581,6 +581,8 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, XLogRecordBlockHeader bkpb; XLogRecordBlockImageHeader bimg; XLogRecordBlockCompressHeader cbimg = {0}; + int data_length = 0; + XLogSizeClass data_sizeclass = XLS_EMPTY; bool samerel; bool is_compressed = false; bool include_image; @@ -622,7 +624,7 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, bkpb.id = block_id; bkpb.fork_flags = regbuf->forkno; - bkpb.data_length = 0; + data_length = 0; if ((regbuf->flags & REGBUF_WILL_INIT) == REGBUF_WILL_INIT) bkpb.fork_flags |= BKPBLOCK_WILL_INIT; @@ -785,13 +787,16 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, * overall list. */ bkpb.fork_flags |= BKPBLOCK_HAS_DATA; - bkpb.data_length = (uint16) regbuf->rdata_len; + data_length = (uint16) regbuf->rdata_len; + data_sizeclass = XLogLengthToSizeClass(data_length, XLS_UINT16); total_len += regbuf->rdata_len; rdt_datas_last->next = regbuf->rdata_head; rdt_datas_last = regbuf->rdata_tail; } + bkpb.id |= data_sizeclass << XLR_BLOCKID_SZCLASS_SHIFT; + if (prev_regbuf && RelFileLocatorEquals(regbuf->rlocator, prev_regbuf->rlocator)) { samerel = true; @@ -804,6 +809,10 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, /* Ok, copy the header to the scratch buffer */ memcpy(scratch, &bkpb, SizeOfXLogRecordBlockHeader); scratch += SizeOfXLogRecordBlockHeader; + + scratch += XLogWriteLength(data_length, data_sizeclass, + XLS_UINT16, scratch); + if (include_image) { memcpy(scratch, &bimg, SizeOfXLogRecordBlockImageHeader); diff --git a/src/backend/access/transam/xlogreader.c b/src/backend/access/transam/xlogreader.c index c9f9f6e98f..becb207152 100644 --- a/src/backend/access/transam/xlogreader.c +++ b/src/backend/access/transam/xlogreader.c @@ -1656,11 +1656,11 @@ DecodeXLogRecord(XLogReaderState *state, */ #define COPY_HEADER_FIELD(_dst, _size) \ do { \ - if (remaining < _size) \ + if (remaining < (_size)) \ goto shortdata_err; \ - memcpy(_dst, ptr, _size); \ - ptr += _size; \ - remaining -= _size; \ + memcpy((_dst), ptr, (_size)); \ + ptr += (_size); \ + remaining -= (_size); \ } while(0) char *ptr; @@ -1686,8 +1686,13 @@ DecodeXLogRecord(XLogReaderState *state, datatotal = 0; while (remaining > datatotal) { + XLogSizeClass sizeClass; COPY_HEADER_FIELD(&block_id, sizeof(uint8)); + sizeClass = (block_id & XLR_BLOCKID_SZCLASS_MASK) >> + XLR_BLOCKID_SZCLASS_SHIFT; + block_id &= XLR_BLOCK_ID_MASK; + if (block_id == XLR_BLOCK_ID_DATA_SHORT) { /* XLogRecordDataHeaderShort */ @@ -1751,7 +1756,26 @@ DecodeXLogRecord(XLogReaderState *state, blk->prefetch_buffer = InvalidBuffer; - COPY_HEADER_FIELD(&blk->data_len, sizeof(uint16)); + { + int read; + uint32 length = 0; + read = XLogReadLength(&length, sizeClass, + XLS_UINT16, + ptr, remaining); + + if (read < 0) + { + report_invalid_record(state, + "Could not read length from record at %X/%X", + LSN_FORMAT_ARGS(state->ReadRecPtr)); + goto err; + } + + ptr += read; + remaining -= read; + blk->data_len = length; + } + /* cross-check that the HAS_DATA flag is set iff data_length > 0 */ if (blk->has_data && blk->data_len == 0) { diff --git a/src/include/access/xlog_internal.h b/src/include/access/xlog_internal.h index b0fd338a00..af822b471b 100644 --- a/src/include/access/xlog_internal.h +++ b/src/include/access/xlog_internal.h @@ -328,6 +328,135 @@ typedef enum struct LogicalDecodingContext; struct XLogRecordBuffer; +/* + * XLogSizeClass interactions + */ + +/* + * XLogLengthToSizeClass + * Turns a length into a size class + * + * The maxSizeClass argument is used for constant folding, to allow the + * compiler to remove any branches that will never be hit. + * + * Feeding it a length value that is larger than the largest permitted value + * in the given maxSizeClass will lead to undefined behavior. + */ +static inline XLogSizeClass XLogLengthToSizeClass(uint32 length, + const XLogSizeClass maxSizeClass) +{ + XLogSizeClass sizeClass; + + if (length == 0) + sizeClass = XLS_EMPTY; + else if (length <= UINT8_MAX && maxSizeClass >= XLS_UINT8) + sizeClass = XLS_UINT8; + else if (length <= UINT16_MAX && maxSizeClass >= XLS_UINT16) + sizeClass = XLS_UINT16; + else + { + Assert(maxSizeClass == XLS_UINT32); + sizeClass = XLS_UINT32; + } + + Assert(sizeClass <= maxSizeClass); + + return sizeClass; +} + +/* + * XLogWriteLength + * + * Write a length with size determined by sizeClass into the output. + * Returns the Size of bytes written. The user is responsible for making sure + * there is enough spare space at the pointer. This function writes at most 4 + * bytes. + * + * The maxSizeClass argument is used for constant folding, to allow the + * compiler to remove any branches that will never be hit. + */ +static inline int XLogWriteLength(uint32 length, XLogSizeClass sizeClass, + const XLogSizeClass maxSizeClass, + char *output) +{ + int written = -1; + + Assert(sizeClass <= maxSizeClass && + XLogLengthToSizeClass(length, maxSizeClass) == sizeClass); + + /* + * The writing version of the READ_OP macro in XLogReadLength, but without + * as many assertions and checks. + */ +#define WRITE_OP(caseSC, field_type) \ + case caseSC: \ + if ((caseSC) <= maxSizeClass) \ + { \ + field_type typedLength = length; \ + memcpy(output, &typedLength, sizeof(field_type)); \ + written = sizeof(field_type); \ + } \ + break + + switch (sizeClass) { + case XLS_EMPTY: + written = 0; + break; + WRITE_OP(XLS_UINT8, uint8); + WRITE_OP(XLS_UINT16, uint16); + WRITE_OP(XLS_UINT32, uint32); + } + +#undef WRITE_OP + return written; +} + +/* + * XLogReadLength + * + * Read a length with size determined by sizeClass from the input into + * *length. Returns the Size of bytes read, or -1 if the input size was + * too small to read the relevant length field. + */ +static inline int XLogReadLength(uint32 *length, XLogSizeClass sizeClass, + const XLogSizeClass maxSizeClass, + char *input, Size inSize) +{ + int readSize = -1; + + Assert(sizeClass <= maxSizeClass); + + /* + * A templated version of reading a length using the correct size class+ + * field type pair. Handles both + */ +#define READ_OP(caseSC, field_type) \ + case caseSC: \ + if ((caseSC) <= maxSizeClass) \ + { \ + field_type typedLength; \ + if (inSize < sizeof(field_type)) \ + return -1; \ + memcpy(&typedLength, input, sizeof(field_type)); \ + readSize = sizeof(field_type); \ + *length = typedLength; \ + } \ + break + + switch (sizeClass) { + case XLS_EMPTY: + readSize = 0; + *length = 0; + break; + READ_OP(XLS_UINT8, uint8); + READ_OP(XLS_UINT16, uint16); + READ_OP(XLS_UINT32, uint32); + } + +#undef READ_OP + return readSize; +} + /* * Method table for resource managers. * diff --git a/src/include/access/xlogrecord.h b/src/include/access/xlogrecord.h index f355e08e1d..c1c4f04a06 100644 --- a/src/include/access/xlogrecord.h +++ b/src/include/access/xlogrecord.h @@ -17,6 +17,28 @@ #include "storage/block.h" #include "storage/relfilelocator.h" +/* + * XLogSizeClass + * + * XLog data segments that are registered to e.g. blocks often are quite + * small. This XLogSizeClass infrastructure allows us to do variable length + * encoding on these fields, so that we may reduce the overhead of size + * administration of small record data sections. + * + * Note that it is possible to encode smaller sizes in the larger size classes, + * but the reverse is not true. For maximum efficiency, it is important to use + * the smallest size class that still supports the given length: Whilst larger + * size classes are supported, they use more bytes to store the same value. + * + * Values > UINT32_MAX are not supported in this scheme. + */ +typedef enum XLogSizeClass { + XLS_EMPTY = 0, /* length == 0 */ + XLS_UINT8 = 1, /* length <= UINT8_MAX; stored in uint8 (1B) */ + XLS_UINT16 = 2, /* length <= UINT16_MAX; stored in uint16 (2B) */ + XLS_UINT32 = 3 /* length <= UINT32_MAX; stored in uint32 (4B) */ +} XLogSizeClass; + /* * The overall layout of an XLOG record is: * Fixed-size header (XLogRecord struct) @@ -104,15 +126,14 @@ typedef struct XLogRecordBlockHeader { uint8 id; /* block reference ID */ uint8 fork_flags; /* fork within the relation, and flags */ - uint16 data_length; /* number of payload bytes (not including page - * image) */ + /* Depending on XLR_BLOCKID_SZCLASS; 0, 1 or 2 bytes of data_length follow */ /* If BKPBLOCK_HAS_IMAGE, an XLogRecordBlockImageHeader struct follows */ /* If BKPBLOCK_SAME_REL is not set, a RelFileLocator follows */ /* BlockNumber follows */ } XLogRecordBlockHeader; -#define SizeOfXLogRecordBlockHeader (offsetof(XLogRecordBlockHeader, data_length) + sizeof(uint16)) +#define SizeOfXLogRecordBlockHeader (offsetof(XLogRecordBlockHeader, fork_flags) + sizeof(uint8)) /* * Additional header information when a full-page image is included @@ -226,6 +247,11 @@ typedef struct XLogRecordDataHeaderLong #define SizeOfXLogRecordDataHeaderLong (sizeof(uint8) + sizeof(uint32)) +#define XLR_BLOCK_ID_MASK 0x3F + +#define XLR_BLOCKID_SZCLASS_SHIFT 6 +#define XLR_BLOCKID_SZCLASS_MASK (3 << XLR_BLOCKID_SZCLASS_SHIFT) + /* * Block IDs used to distinguish different kinds of record fragments. Block * references are numbered from 0 to XLR_MAX_BLOCK_ID. A rmgr is free to use @@ -234,15 +260,15 @@ typedef struct XLogRecordDataHeaderLong * numbers are reserved to denote the "main" data portion of the record, * as well as replication-supporting transaction metadata. * - * The maximum is currently set at 32, quite arbitrarily. Most records only + * The maximum is currently set at 0x20 (32), quite arbitrarily. Most records only * need a handful of block references, but there are a few exceptions that * need more. */ -#define XLR_MAX_BLOCK_ID 32 +#define XLR_MAX_BLOCK_ID 0x20 -#define XLR_BLOCK_ID_DATA_SHORT 255 -#define XLR_BLOCK_ID_DATA_LONG 254 -#define XLR_BLOCK_ID_ORIGIN 253 -#define XLR_BLOCK_ID_TOPLEVEL_XID 252 +#define XLR_BLOCK_ID_DATA_SHORT 0x3F +#define XLR_BLOCK_ID_DATA_LONG 0x3E +#define XLR_BLOCK_ID_ORIGIN 0x3D +#define XLR_BLOCK_ID_TOPLEVEL_XID 0x3C #endif /* XLOGRECORD_H */ -- 2.40.1 [application/octet-stream] v5-0005-Stop-the-logging-of-XIDs-in-records-where-the-XID.patch (10.4K, ../../CAEze2WgQ9d_aa6tGVDqXgdy2owvrrJtY6TaCZQ6r+BA7F1fSCA@mail.gmail.com/6-v5-0005-Stop-the-logging-of-XIDs-in-records-where-the-XID.patch) download | inline diff: From 036657ae71cea6f8040907f755b2b570d5245758 Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Wed, 25 Jan 2023 15:40:30 +0100 Subject: [PATCH v5 5/7] Stop the logging of XIDs in records where the XID is unused. Many WAL record types do not need to contain an XID. This change prepares for the removal of the xid from the main xlog record header, by requiring the user of the XLog insertion API to explicitly request the inclusion of the current xid in the log record - in all other cases we'll log, assume and track InvalidXid instead. --- src/backend/access/heap/heapam.c | 16 +++++++++------- src/backend/access/heap/pruneheap.c | 1 + src/backend/access/heap/rewriteheap.c | 1 + src/backend/access/transam/generic_xlog.c | 3 +++ src/backend/access/transam/multixact.c | 1 + src/backend/access/transam/xact.c | 4 ++-- src/backend/access/transam/xloginsert.c | 12 ++++++++++-- src/backend/commands/tablecmds.c | 2 +- src/backend/replication/logical/message.c | 2 +- src/backend/utils/cache/inval.c | 1 + src/include/access/xlog.h | 1 + 11 files changed, 31 insertions(+), 13 deletions(-) diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index e61a6f67d5..49421d48ed 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -1961,7 +1961,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid, heaptup->t_len - SizeofHeapTupleHeader); /* filtering by origin on a row level is much more efficient */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP_ID, info); @@ -2357,7 +2357,7 @@ heap_multi_insert(Relation relation, TupleTableSlot **slots, int ntuples, XLogRegisterBufData(0, tupledata, totaldatalen); /* filtering by origin on a row level is much more efficient */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP2_ID, info); @@ -2857,7 +2857,7 @@ l1: } /* filtering by origin on a row level is much more efficient */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP_ID, XLOG_HEAP_DELETE); @@ -5527,6 +5527,7 @@ l4: cleared_all_frozen ? XLH_LOCK_ALL_FROZEN_CLEARED : 0; XLogRegisterData((char *) &xlrec, SizeOfHeapLockUpdated); + XLogSetRecordFlags(XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP2_ID, XLOG_HEAP2_LOCK_UPDATED); @@ -5677,7 +5678,7 @@ heap_finish_speculative(Relation relation, ItemPointer tid) XLogBeginInsert(); /* We want the same filtering on this as on a plain insert */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); XLogRegisterData((char *) &xlrec, SizeOfHeapConfirm); XLogRegisterBuffer(0, buffer, REGBUF_STANDARD); @@ -5823,7 +5824,8 @@ heap_abort_speculative(Relation relation, ItemPointer tid) XLogRegisterData((char *) &xlrec, SizeOfHeapDelete); XLogRegisterBuffer(0, buffer, REGBUF_STANDARD); - /* No replica identity & replication origin logged */ + /* No replica identity & replication origin logged, but XID is required */ + XLogSetRecordFlags(XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP_ID, XLOG_HEAP_DELETE); @@ -8555,7 +8557,7 @@ log_heap_update(Relation reln, Buffer oldbuf, } /* filtering by origin on a row level is much more efficient */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP_ID, info); @@ -8630,7 +8632,7 @@ log_heap_new_cid(Relation relation, HeapTuple tup) XLogRegisterData((char *) &xlrec, SizeOfHeapNewCid); /* will be looked at irrespective of origin */ - + XLogSetRecordFlags(XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP2_ID, XLOG_HEAP2_NEW_CID); return recptr; diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index 18193efa23..8e3b242521 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -391,6 +391,7 @@ heap_page_prune(Relation relation, Buffer buffer, XLogRegisterBufData(0, (char *) prstate.nowunused, prstate.nunused * sizeof(OffsetNumber)); + XLogSetRecordFlags(XLOG_INCLUDE_XID); recptr = XLogInsert(RM_HEAP2_ID, XLOG_HEAP2_PRUNE); PageSetLSN(BufferGetPage(buffer), recptr); diff --git a/src/backend/access/heap/rewriteheap.c b/src/backend/access/heap/rewriteheap.c index 424958912c..9f77537335 100644 --- a/src/backend/access/heap/rewriteheap.c +++ b/src/backend/access/heap/rewriteheap.c @@ -921,6 +921,7 @@ logical_heap_rewrite_flush_mappings(RewriteState state) src->off += len; XLogBeginInsert(); + XLogSetRecordFlags(XLOG_INCLUDE_XID); XLogRegisterData((char *) (&xlrec), sizeof(xlrec)); XLogRegisterData(waldata_start, len); diff --git a/src/backend/access/transam/generic_xlog.c b/src/backend/access/transam/generic_xlog.c index 6c68191ca6..c8983b40ad 100644 --- a/src/backend/access/transam/generic_xlog.c +++ b/src/backend/access/transam/generic_xlog.c @@ -399,6 +399,9 @@ GenericXLogFinish(GenericXLogState *state) } } + /* Assume XID is required */ + XLogSetRecordFlags(XLOG_INCLUDE_XID); + /* Insert xlog record */ lsn = XLogInsert(RM_GENERIC_ID, 0); diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c index c70b6b7358..883da6a33b 100644 --- a/src/backend/access/transam/multixact.c +++ b/src/backend/access/transam/multixact.c @@ -834,6 +834,7 @@ MultiXactIdCreateFromMembers(int nmembers, MultiXactMember *members) * Not clear that it's worth the trouble though. */ XLogBeginInsert(); + XLogSetRecordFlags(XLOG_INCLUDE_XID); XLogRegisterData((char *) (&xlrec), SizeOfMultiXactCreate); XLogRegisterData((char *) members, nmembers * sizeof(MultiXactMember)); diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c index da0ac6c8d3..57504594b0 100644 --- a/src/backend/access/transam/xact.c +++ b/src/backend/access/transam/xact.c @@ -5793,7 +5793,7 @@ XactLogCommitRecord(TimestampTz commit_time, XLogRegisterData((char *) (&xl_origin), sizeof(xl_xact_origin)); /* we allow filtering by xacts */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); return XLogInsertExtended(RM_XACT_ID, info, rmgr_info); } @@ -5940,7 +5940,7 @@ XactLogAbortRecord(TimestampTz abort_time, XLogRegisterData((char *) (&xl_origin), sizeof(xl_xact_origin)); /* Include the replication origin */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); return XLogInsertExtended(RM_XACT_ID, info, rmgr_info); } diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c index 9596a37cb2..fbb081d95d 100644 --- a/src/backend/access/transam/xloginsert.c +++ b/src/backend/access/transam/xloginsert.c @@ -456,6 +456,10 @@ XLogSetRecordFlags(uint8 flags) * (LSN is the XLOG point up to which the XLOG must be flushed to disk * before the data page can be written out. This implements the basic * WAL rule "write the log before the data".) + * + * Note: To include the current backend's TransactionID in the record, + * you have to set the XLOG_INCLUDE_XID flag using XLogRecordSetFlags + * before calling XLogInsert. */ XLogRecPtr XLogInsert(RmgrId rmid, uint8 rmgr_info) @@ -847,7 +851,7 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP } /* followed by toplevel XID, if not already included in previous record */ - if (IsSubxactTopXidLogPending()) + if (curinsert_flags & XLOG_INCLUDE_XID && IsSubxactTopXidLogPending()) { TransactionId xid = GetTopTransactionIdIfAny(); @@ -924,7 +928,11 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP * once we know where in the WAL the record will be inserted. The CRC does * not include the record header yet. */ - rechdr->xl_xid = GetCurrentTransactionIdIfAny(); + if (curinsert_flags & XLOG_INCLUDE_XID) + rechdr->xl_xid = GetCurrentTransactionIdIfAny(); + else + rechdr->xl_xid = InvalidTransactionId; + rechdr->xl_tot_len = (uint32) total_len; rechdr->xl_info = info; rechdr->xl_rmid = rmid; diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 8a2c671b66..ff9b718250 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -2150,7 +2150,7 @@ ExecuteTruncateGuts(List *explicit_rels, XLogRegisterData((char *) &xlrec, SizeOfHeapTruncate); XLogRegisterData((char *) logrelids, list_length(relids_logged) * sizeof(Oid)); - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); (void) XLogInsert(RM_HEAP_ID, XLOG_HEAP_TRUNCATE); } diff --git a/src/backend/replication/logical/message.c b/src/backend/replication/logical/message.c index c31dc1203f..af86d18457 100644 --- a/src/backend/replication/logical/message.c +++ b/src/backend/replication/logical/message.c @@ -69,7 +69,7 @@ LogLogicalMessage(const char *prefix, const char *message, size_t size, XLogRegisterData(unconstify(char *, message), size); /* allow origin filtering */ - XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN); + XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN | XLOG_INCLUDE_XID); return XLogInsert(RM_LOGICALMSG_ID, XLOG_LOGICAL_MESSAGE); } diff --git a/src/backend/utils/cache/inval.c b/src/backend/utils/cache/inval.c index 0008826f67..b0afdd6f90 100644 --- a/src/backend/utils/cache/inval.c +++ b/src/backend/utils/cache/inval.c @@ -1632,6 +1632,7 @@ LogLogicalInvalidations(void) ProcessMessageSubGroupMulti(group, RelCacheMsgs, XLogRegisterData((char *) msgs, n * sizeof(SharedInvalidationMessage))); + XLogSetRecordFlags(XLOG_INCLUDE_XID); XLogInsert(RM_XACT_ID, XLOG_XACT_INVALIDATIONS); } } diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h index 9007f559f3..f4dfaeda29 100644 --- a/src/include/access/xlog.h +++ b/src/include/access/xlog.h @@ -150,6 +150,7 @@ extern PGDLLIMPORT bool XLOG_DEBUG; */ #define XLOG_INCLUDE_ORIGIN 0x01 /* include the replication origin */ #define XLOG_MARK_UNIMPORTANT 0x02 /* record not important for durability */ +#define XLOG_INCLUDE_XID 0x04 /* include the transaction ID */ /* Checkpoint statistics */ -- 2.40.1 [application/octet-stream] v5-0006-Add-RM_INVALID-an-invalid-resource-manager.patch (3.7K, ../../CAEze2WgQ9d_aa6tGVDqXgdy2owvrrJtY6TaCZQ6r+BA7F1fSCA@mail.gmail.com/7-v5-0006-Add-RM_INVALID-an-invalid-resource-manager.patch) download | inline diff: From 063c45667ef3ba3c8a23297b07d42edc8653904c Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Wed, 12 Jul 2023 14:08:36 +0200 Subject: [PATCH v5 6/7] Add RM_INVALID, an invalid resource manager The resource manager has ID = 0, thus requiring some special handling in other code. Apart from being generally useful, it is used in future patches to detect the end of wal in lieu of a zero-ed fixed-size xl_tot_len field. --- src/backend/access/transam/xlogreader.c | 13 +++++++++++++ src/bin/pg_waldump/pg_waldump.c | 4 ++-- src/include/access/rmgr.h | 2 +- src/include/access/rmgrlist.h | 1 + src/include/access/xlog_internal.h | 2 +- 5 files changed, 18 insertions(+), 4 deletions(-) diff --git a/src/backend/access/transam/xlogreader.c b/src/backend/access/transam/xlogreader.c index bebdf1759a..069289847d 100644 --- a/src/backend/access/transam/xlogreader.c +++ b/src/backend/access/transam/xlogreader.c @@ -670,6 +670,19 @@ restart: else { /* XXX: more validation should be done here */ + + /* + * RM_INVALID_ID can be an indication of end-of-WAL, or other signs of + * corruption - that byte should be non-0. + */ + if (record->xl_rmid == RM_INVALID_ID) + { + report_invalid_record(state, + "invalid resource manager id %u at %X/%X", + (uint32) record->xl_rmid, + LSN_FORMAT_ARGS(RecPtr)); + goto err; + } if (total_len < SizeOfXLogRecord) { report_invalid_record(state, diff --git a/src/bin/pg_waldump/pg_waldump.c b/src/bin/pg_waldump/pg_waldump.c index fb6a5e76b1..83058d3b47 100644 --- a/src/bin/pg_waldump/pg_waldump.c +++ b/src/bin/pg_waldump/pg_waldump.c @@ -99,7 +99,7 @@ print_rmgr_list(void) { int i; - for (i = 0; i <= RM_MAX_BUILTIN_ID; i++) + for (i = RM_INVALID_ID + 1; i <= RM_MAX_BUILTIN_ID; i++) { printf("%s\n", GetRmgrDesc(i)->rm_name); } @@ -967,7 +967,7 @@ main(int argc, char **argv) else { /* then look for builtin rmgrs */ - for (rmid = 0; rmid <= RM_MAX_BUILTIN_ID; rmid++) + for (rmid = RM_INVALID_ID + 1; rmid <= RM_MAX_BUILTIN_ID; rmid++) { if (pg_strcasecmp(optarg, GetRmgrDesc(rmid)->rm_name) == 0) { diff --git a/src/include/access/rmgr.h b/src/include/access/rmgr.h index 3b6a497e1b..7038ba0833 100644 --- a/src/include/access/rmgr.h +++ b/src/include/access/rmgr.h @@ -41,7 +41,7 @@ typedef enum RmgrIds static inline bool RmgrIdIsBuiltin(int rmid) { - return rmid <= RM_MAX_BUILTIN_ID; + return rmid <= RM_MAX_BUILTIN_ID && rmid != RM_INVALID_ID; } static inline bool diff --git a/src/include/access/rmgrlist.h b/src/include/access/rmgrlist.h index 463bcb67c5..b53d731f56 100644 --- a/src/include/access/rmgrlist.h +++ b/src/include/access/rmgrlist.h @@ -25,6 +25,7 @@ */ /* symbol name, textual name, redo, desc, identify, startup, cleanup, mask, decode */ +PG_RMGR(RM_INVALID_ID, "invalid", NULL, NULL, NULL, NULL, NULL, NULL, NULL) PG_RMGR(RM_XLOG_ID, "XLOG", xlog_redo, xlog_desc, xlog_identify, NULL, NULL, NULL, xlog_decode) PG_RMGR(RM_XACT_ID, "Transaction", xact_redo, xact_desc, xact_identify, NULL, NULL, NULL, xact_decode) PG_RMGR(RM_SMGR_ID, "Storage", smgr_redo, smgr_desc, smgr_identify, NULL, NULL, NULL, NULL) diff --git a/src/include/access/xlog_internal.h b/src/include/access/xlog_internal.h index 9413836a6a..2277b32d05 100644 --- a/src/include/access/xlog_internal.h +++ b/src/include/access/xlog_internal.h @@ -31,7 +31,7 @@ /* * Each page of XLOG file has a header like this: */ -#define XLOG_PAGE_MAGIC 0xD113 /* can be used as WAL version indicator */ +#define XLOG_PAGE_MAGIC 0xD114 /* can be used as WAL version indicator */ typedef struct XLogPageHeaderData { -- 2.40.1 [application/octet-stream] v5-0007-Reformat-the-XLog-record-header.patch (42.6K, ../../CAEze2WgQ9d_aa6tGVDqXgdy2owvrrJtY6TaCZQ6r+BA7F1fSCA@mail.gmail.com/8-v5-0007-Reformat-the-XLog-record-header.patch) download | inline diff: From 5fd93d55573d2360138862ca456d086761f18da4 Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Mon, 19 Jun 2023 17:46:45 +0200 Subject: [PATCH v5 7/7] Reformat the XLog record header Many WAL records don't utilize all bytes of the XLog header, resulting in a lot of wasted bytes. This new XLog header format saves bytes in several ways: - Records that don't need the XID won't include it This potentially saves 4 bytes /record - Records that are small enough don't need 4 bytes to describe their length By varint encoding the length, we save up to 3 bytes /record (up to 4 bytes /record if dataless records are considered) - Only include xl_rmgrinfo if it is non-0 Several RMGRs have a record that they use most often. We save 1 byte/record if that record id is 0. - Lose some alignment losses We save 1 byte/record by removing one byte of alignment losses --- src/backend/access/transam/xlog.c | 133 ++++++++++++++------- src/backend/access/transam/xloginsert.c | 135 ++++++++++++++++------ src/backend/access/transam/xlogreader.c | 105 +++++++++-------- src/backend/access/transam/xlogrecovery.c | 4 +- src/bin/pg_resetwal/pg_resetwal.c | 38 +++--- src/include/access/xlog_internal.h | 100 +++++++++++++++- src/include/access/xlogreader.h | 4 +- src/include/access/xlogrecord.h | 125 +++++++++++++++++--- 8 files changed, 486 insertions(+), 158 deletions(-) diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index eded22ab1d..6200f91445 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -737,17 +737,18 @@ XLogInsertRecord(XLogRecData *rdata, XLogCtlInsert *Insert = &XLogCtl->Insert; pg_crc32c rdata_crc; bool inserted; - XLogRecord *rechdr = (XLogRecord *) rdata->data; - uint8 rmgrinfo = rechdr->xl_rmgrinfo; - bool isLogSwitch = (rechdr->xl_rmid == RM_XLOG_ID && - rmgrinfo == XLOG_SWITCH); + XLogRecHdr rechdr; + bool isLogSwitch; XLogRecPtr StartPos; XLogRecPtr EndPos; + XLogRecPtr xl_prev; bool prevDoPageWrites = doPageWrites; TimeLineID insertTLI; + int rec_payload_len, + rec_hdr_len; - /* we assume that all of the record header is in the first chunk */ - Assert(rdata->len >= SizeOfXLogRecord); + /* we assume that all data of the record header is in the first chunk */ + Assert(rdata->len >= XLogRecordMinHdrSize); /* cross-check on whether we should be here or not */ if (!XLogInsertAllowed()) @@ -758,6 +759,17 @@ XLogInsertRecord(XLogRecData *rdata, * change, so we can read it without a lock. */ insertTLI = XLogCtl->InsertTimeLineID; + rechdr = (XLogRecHdr) rdata->data; + + { + XLogRecord record = {0}; + XLogReadRecHdrInto(rechdr, (Size) rdata->len, &record); + + isLogSwitch = (record.xl_rmid == RM_XLOG_ID && + record.xl_rmgrinfo == XLOG_SWITCH); + rec_hdr_len = (int) XLogRecordHdrLen(rechdr->xl_info); + rec_payload_len = (int) record.xl_payload_len; + } /*---------- * @@ -834,34 +846,44 @@ XLogInsertRecord(XLogRecData *rdata, } /* - * Reserve space for the record in the WAL. This also sets the xl_prev - * pointer. + * Reserve space for the record in the WAL. */ if (isLogSwitch) - inserted = ReserveXLogSwitch(&StartPos, &EndPos, &rechdr->xl_prev); + inserted = ReserveXLogSwitch(&StartPos, &EndPos, &xl_prev); else { - ReserveXLogInsertLocation(rechdr->xl_tot_len, &StartPos, &EndPos, - &rechdr->xl_prev); + ReserveXLogInsertLocation(rec_hdr_len + rec_payload_len, + &StartPos, &EndPos, &xl_prev); inserted = true; } if (inserted) { + char *rec = (char *) rechdr; + + /* fill in xl_prev */ + memcpy(rec + rec_hdr_len - sizeof(pg_crc32c) - sizeof(XLogRecPtr), + &xl_prev, sizeof(XLogRecPtr)); + /* * Now that xl_prev has been filled in, calculate CRC of the record * header. */ - rdata_crc = rechdr->xl_crc; - COMP_CRC32C(rdata_crc, rechdr, offsetof(XLogRecord, xl_crc)); + + memcpy(&rdata_crc, rec + rec_hdr_len - sizeof(pg_crc32c), + sizeof(pg_crc32c)); + + COMP_CRC32C(rdata_crc, rec, rec_hdr_len - sizeof(pg_crc32c)); FIN_CRC32C(rdata_crc); - rechdr->xl_crc = rdata_crc; + + memcpy(rec + rec_hdr_len - sizeof(pg_crc32c), + &rdata_crc, sizeof(pg_crc32c)); /* * All the record data, including the header, is now ready to be * inserted. Copy the record in the space reserved. */ - CopyXLogRecordToWAL(rechdr->xl_tot_len, isLogSwitch, rdata, + CopyXLogRecordToWAL(rec_hdr_len + rec_payload_len, isLogSwitch, rdata, StartPos, EndPos, insertTLI); /* @@ -932,7 +954,8 @@ XLogInsertRecord(XLogRecData *rdata, */ if (inserted) { - EndPos = StartPos + SizeOfXLogRecord; + /* xlog switch is minimal record header, plus a byte for rmgrinfo */ + EndPos = StartPos + XLogRecordMinHdrSize + sizeof(uint8); if (StartPos / XLOG_BLCKSZ != EndPos / XLOG_BLCKSZ) { uint64 offset = XLogSegmentOffset(EndPos, wal_segment_size); @@ -949,7 +972,7 @@ XLogInsertRecord(XLogRecData *rdata, if (XLOG_DEBUG) { static XLogReaderState *debug_reader = NULL; - XLogRecord *record; + XLogRecHdr record; DecodedXLogRecord *decoded; StringInfoData buf; StringInfoData recordBuf; @@ -971,9 +994,9 @@ XLogInsertRecord(XLogRecData *rdata, appendBinaryStringInfo(&recordBuf, rdata->data, rdata->len); /* We also need temporary space to decode the record. */ - record = (XLogRecord *) recordBuf.data; + record = (XLogRecHdr) recordBuf.data; decoded = (DecodedXLogRecord *) - palloc(DecodeXLogRecordRequiredSpace(record->xl_tot_len)); + palloc(DecodeXLogRecordRequiredSpace(recordBuf.len)); if (!debug_reader) debug_reader = XLogReaderAllocate(wal_segment_size, NULL, @@ -1018,7 +1041,7 @@ XLogInsertRecord(XLogRecData *rdata, /* Report WAL traffic to the instrumentation. */ if (inserted) { - pgWalUsage.wal_bytes += rechdr->xl_tot_len; + pgWalUsage.wal_bytes += rec_hdr_len + rec_payload_len; pgWalUsage.wal_records++; pgWalUsage.wal_fpi += num_fpi; } @@ -1049,10 +1072,10 @@ ReserveXLogInsertLocation(int size, XLogRecPtr *StartPos, XLogRecPtr *EndPos, uint64 endbytepos; uint64 prevbytepos; - size = MAXALIGN(size); + size = XLP_ALIGN(size); /* All (non xlog-switch) records should contain data. */ - Assert(size > SizeOfXLogRecord); + Assert(size > XLogRecordMinHdrSize); /* * The duration the spinlock needs to be held is minimized by minimizing @@ -1103,7 +1126,7 @@ ReserveXLogSwitch(XLogRecPtr *StartPos, XLogRecPtr *EndPos, XLogRecPtr *PrevPtr) uint64 startbytepos; uint64 endbytepos; uint64 prevbytepos; - uint32 size = MAXALIGN(SizeOfXLogRecord); + uint32 size = XLP_ALIGN(XLogRecordMinHdrSize + sizeof(uint8)); XLogRecPtr ptr; uint32 segleft; @@ -1176,10 +1199,10 @@ CopyXLogRecordToWAL(int write_len, bool isLogSwitch, XLogRecData *rdata, freespace = INSERT_FREESPACE(CurrPos); /* - * there should be enough space for at least the first field (xl_tot_len) + * there should be enough space for at least the first XL_ALIGN quantum * on this page. */ - Assert(freespace >= sizeof(uint32)); + Assert(freespace >= XLP_ALIGN(1)); /* Copy record data */ written = 0; @@ -1247,7 +1270,7 @@ CopyXLogRecordToWAL(int write_len, bool isLogSwitch, XLogRecData *rdata, if (isLogSwitch && XLogSegmentOffset(CurrPos, wal_segment_size) != 0) { /* An xlog-switch record doesn't contain any data besides the header */ - Assert(write_len == SizeOfXLogRecord); + Assert(write_len == XLogRecordMinHdrSize + sizeof(uint8)); /* Assert that we did reserve the right amount of space */ Assert(XLogSegmentOffset(EndPos, wal_segment_size) == 0); @@ -1291,7 +1314,7 @@ CopyXLogRecordToWAL(int write_len, bool isLogSwitch, XLogRecData *rdata, else { /* Align the end position, so that the next record starts aligned */ - CurrPos = MAXALIGN64(CurrPos); + CurrPos = XLP_ALIGN(CurrPos); } if (CurrPos != EndPos) @@ -4672,11 +4695,15 @@ BootStrapXLOG(void) char *buffer; XLogPageHeader page; XLogLongPageHeader longpage; - XLogRecord *record; + XLogRecHdr rec_hdr; + char *baserecptr; char *recptr; uint64 sysidentifier; struct timeval tv; pg_crc32c crc; + const int rec_hdr_len = offsetof(XLogRecHdrData, xl_hdrdata) + + sizeof(uint8) + sizeof(XLogRecPtr) + sizeof(pg_crc32c); + const int rec_payload_len = sizeof(uint8) * 2 + sizeof(CheckPoint); /* allow ordinary WAL segment creation, like StartupXLOG() would */ SetInstallXLogFileSegmentActive(); @@ -4747,28 +4774,52 @@ BootStrapXLOG(void) longpage->xlp_seg_size = wal_segment_size; longpage->xlp_xlog_blcksz = XLOG_BLCKSZ; + /* if this changes, we need to update the code below */ + Assert(XLogLengthToSizeClass(SizeOfXLogRecordDataHeaderShort + sizeof(checkPoint), XLS_UINT32) == XLS_UINT8); + /* if this changes, we need to add XLR_HAS_RMGRINFO */ + Assert(XLOG_CHECKPOINT_SHUTDOWN == 0); + /* Insert the initial checkpoint record */ - recptr = ((char *) page + SizeOfXLogLongPHD); - record = (XLogRecord *) recptr; - record->xl_prev = 0; - record->xl_xid = InvalidTransactionId; - record->xl_tot_len = SizeOfXLogRecord + SizeOfXLogRecordDataHeaderShort + sizeof(checkPoint); - record->xl_info = 0; - record->xl_rmgrinfo = XLOG_CHECKPOINT_SHUTDOWN; - record->xl_rmid = RM_XLOG_ID; - recptr += SizeOfXLogRecord; + baserecptr = recptr = (((char *) page) + SizeOfXLogLongPHD); + + rec_hdr = (XLogRecHdr) baserecptr; + rec_hdr->xl_info = (char) XLS_UINT8; + rec_hdr->xl_rmid = (char) RM_XLOG_ID; + recptr += offsetof(XLogRecHdrData, xl_hdrdata); + + recptr += XLogWriteLength(SizeOfXLogRecordDataHeaderShort + sizeof(checkPoint), + XLS_UINT8, XLS_UINT8, recptr); + + /* include prevptr */ + { + XLogRecPtr prevptr = 0; + memcpy(recptr, &prevptr, sizeof(XLogRecPtr)); + recptr += sizeof(XLogRecPtr); + } + + /* reserve location of crc */ + recptr += sizeof(pg_crc32c); + + Assert(recptr - baserecptr == XLogRecordHdrLen(rec_hdr->xl_info)); + Assert(recptr - baserecptr == rec_hdr_len); + /* fill the XLogRecordDataHeaderShort struct */ *(recptr++) = (char) XLR_BLOCK_ID_DATA_SHORT; *(recptr++) = sizeof(checkPoint); memcpy(recptr, &checkPoint, sizeof(checkPoint)); recptr += sizeof(checkPoint); - Assert(recptr - (char *) record == record->xl_tot_len); + + /* Assert length of record matches expectations */ + Assert(recptr - baserecptr == XLogRecordTotalLength(rec_hdr)); + Assert((rec_hdr)->xl_info == XLS_UINT8); INIT_CRC32C(crc); - COMP_CRC32C(crc, ((char *) record) + SizeOfXLogRecord, record->xl_tot_len - SizeOfXLogRecord); - COMP_CRC32C(crc, (char *) record, offsetof(XLogRecord, xl_crc)); + COMP_CRC32C(crc, baserecptr + rec_hdr_len, rec_payload_len); + COMP_CRC32C(crc, baserecptr, rec_hdr_len - sizeof(pg_crc32c)); FIN_CRC32C(crc); - record->xl_crc = crc; + + memcpy(baserecptr + rec_hdr_len - sizeof(pg_crc32c), + &crc, sizeof(pg_crc32c)); /* Create first XLOG segment file */ openLogTLI = BootstrapTimeLineID; diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c index fbb081d95d..dc6e22ef5f 100644 --- a/src/backend/access/transam/xloginsert.c +++ b/src/backend/access/transam/xloginsert.c @@ -545,12 +545,16 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP XLogRecPtr *fpw_lsn, int *num_fpi, bool *topxid_included) { XLogRecData *rdt; - uint64 total_len = 0; + uint64 payload_len = 0; + XLogSizeClass payload_sizeclass = XLS_EMPTY; int block_id; pg_crc32c rdata_crc; registered_buffer *prev_regbuf = NULL; XLogRecData *rdt_datas_last; - XLogRecord *rechdr; + TransactionId xid; + XLogRecHdr rechdr; + uint8 xlr_flags = 0; + uint32 rec_hdr_len; char *scratch = hdr_scratch; /* @@ -558,9 +562,10 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP * All the modifications we do to the rdata chains below must handle that. */ - /* The record begins with the fixed-size header */ - rechdr = (XLogRecord *) scratch; - scratch += SizeOfXLogRecord; + /* The record begins with the variable-size header */ + rechdr = (XLogRecHdr) scratch; + + scratch += XLogRecordMaxHdrSize; hdr_rdt.next = NULL; rdt_datas_last = &hdr_rdt; @@ -779,7 +784,7 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP } } - total_len += bimg.length; + payload_len += bimg.length; } if (needs_data) @@ -797,7 +802,7 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP bkpb.fork_flags |= BKPBLOCK_HAS_DATA; data_length = (uint16) regbuf->rdata_len; data_sizeclass = XLogLengthToSizeClass(data_length, XLS_UINT16); - total_len += regbuf->rdata_len; + payload_len += regbuf->rdata_len; rdt_datas_last->next = regbuf->rdata_head; rdt_datas_last = regbuf->rdata_tail; @@ -851,16 +856,33 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP } /* followed by toplevel XID, if not already included in previous record */ - if (curinsert_flags & XLOG_INCLUDE_XID && IsSubxactTopXidLogPending()) + if (curinsert_flags & XLOG_INCLUDE_XID) { - TransactionId xid = GetTopTransactionIdIfAny(); + xid = GetCurrentTransactionIdIfAny(); + + if (IsSubxactTopXidLogPending()) + { + TransactionId txid = GetTopTransactionIdIfAny(); + + xlr_flags |= XLR_HAS_XID; + Assert(TransactionIdIsValid(xid)); - /* Set the flag that the top xid is included in the WAL */ - *topxid_included = true; + /* Set the flag that the top xid is included in the WAL */ + *topxid_included = true; - *(scratch++) = (char) XLR_BLOCK_ID_TOPLEVEL_XID; - memcpy(scratch, &xid, sizeof(TransactionId)); - scratch += sizeof(TransactionId); + *(scratch++) = (char) XLR_BLOCK_ID_TOPLEVEL_XID; + memcpy(scratch, &txid, sizeof(TransactionId)); + scratch += sizeof(TransactionId); + } + else if (TransactionIdIsValid(xid)) + { + xlr_flags |= XLR_HAS_XID; + } + } + else + { + xid = InvalidTransactionId; + Assert((xlr_flags & XLR_HAS_XID) == 0); } /* followed by main data, if any */ @@ -889,12 +911,64 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP } rdt_datas_last->next = mainrdata_head; rdt_datas_last = mainrdata_last; - total_len += mainrdata_len; + payload_len += mainrdata_len; } rdt_datas_last->next = NULL; - hdr_rdt.len = (scratch - hdr_scratch); - total_len += hdr_rdt.len; + /* Add the block headers section's length to the payload */ + payload_len += scratch - (hdr_scratch + XLogRecordMaxHdrSize); + + /* + * Fill in the fields in the record header. Prev-link is filled in later, + * once we know where in the WAL the record will be inserted. The CRC does + * not include the record header yet. + */ + payload_sizeclass = XLogLengthToSizeClass(payload_len, XLS_UINT32); + + xlr_flags |= payload_sizeclass; + + if (rmgr_info != 0) + xlr_flags |= XLR_HAS_RMGRINFO; + + /* Set up the xlog header. and xl_rmgr */ + rechdr->xl_info = xlr_flags; + rechdr->xl_rmid = rmid; + + rec_hdr_len = 0; + + /* next, xl_payload_len */ + rec_hdr_len += XLogWriteLength(payload_len, payload_sizeclass, + XLS_UINT32, + &rechdr->xl_hdrdata[rec_hdr_len]); + + if (xlr_flags & XLR_HAS_RMGRINFO) + rechdr->xl_hdrdata[rec_hdr_len++] = (char) rmgr_info; + + if (xlr_flags & XLR_HAS_XID) + { + Assert(curinsert_flags & XLOG_INCLUDE_XID); + Assert(TransactionIdIsValid(xid)); + + memcpy(&rechdr->xl_hdrdata[rec_hdr_len], &xid, sizeof(TransactionId)); + rec_hdr_len += sizeof(TransactionId); + } + + /* reserve space for XLogRecPtr and checksum */ + rec_hdr_len += sizeof(XLogRecPtr); + rec_hdr_len += sizeof(pg_crc32c); + /* Add static header length */ + rec_hdr_len += offsetof(XLogRecHdrData, xl_hdrdata); + + Assert(rec_hdr_len == XLogRecordHdrLen(rechdr->xl_info)); + + /* move the record to be placed the rest of the payload */ + memmove(hdr_scratch + XLogRecordMaxHdrSize - rec_hdr_len, + hdr_scratch, rec_hdr_len); + + rechdr = (XLogRecHdr) (hdr_scratch + XLogRecordMaxHdrSize - rec_hdr_len); + + hdr_rdt.data = (char *) rechdr; + hdr_rdt.len = (scratch - hdr_rdt.data); /* * Calculate CRC of the data @@ -905,10 +979,14 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP * header. */ INIT_CRC32C(rdata_crc); - COMP_CRC32C(rdata_crc, hdr_scratch + SizeOfXLogRecord, hdr_rdt.len - SizeOfXLogRecord); + COMP_CRC32C(rdata_crc, hdr_scratch + XLogRecordMaxHdrSize, + hdr_rdt.len - rec_hdr_len); for (rdt = hdr_rdt.next; rdt != NULL; rdt = rdt->next) COMP_CRC32C(rdata_crc, rdt->data, rdt->len); + memcpy(hdr_rdt.data + rec_hdr_len - sizeof(pg_crc32c), + &rdata_crc, sizeof(pg_crc32c)); + /* * Ensure that the XLogRecord is not too large. * @@ -917,28 +995,11 @@ XLogRecordAssemble(RmgrId rmid, uint8 info, uint8 rmgr_info, XLogRecPtr RedoRecP * not emit records larger than the sizes advertised to be supported. This * cap is based on DecodeXLogRecordRequiredSpace(). */ - if (total_len > XLogRecordMaxSize) + if (payload_len + rec_hdr_len > XLogRecordMaxSize) ereport(ERROR, (errmsg_internal("oversized WAL record"), errdetail_internal("WAL record would be %llu bytes (of maximum %u bytes); rmid %u flags %u.", - (unsigned long long) total_len, XLogRecordMaxSize, rmid, info))); - - /* - * Fill in the fields in the record header. Prev-link is filled in later, - * once we know where in the WAL the record will be inserted. The CRC does - * not include the record header yet. - */ - if (curinsert_flags & XLOG_INCLUDE_XID) - rechdr->xl_xid = GetCurrentTransactionIdIfAny(); - else - rechdr->xl_xid = InvalidTransactionId; - - rechdr->xl_tot_len = (uint32) total_len; - rechdr->xl_info = info; - rechdr->xl_rmid = rmid; - rechdr->xl_rmgrinfo = rmgr_info; - rechdr->xl_prev = InvalidXLogRecPtr; - rechdr->xl_crc = rdata_crc; + (unsigned long long) payload_len, XLogRecordMaxSize, rmid, info))); return &hdr_rdt; } diff --git a/src/backend/access/transam/xlogreader.c b/src/backend/access/transam/xlogreader.c index 069289847d..2c032fc13c 100644 --- a/src/backend/access/transam/xlogreader.c +++ b/src/backend/access/transam/xlogreader.c @@ -49,8 +49,9 @@ static int ReadPageInternal(XLogReaderState *state, XLogRecPtr pageptr, static void XLogReaderInvalReadState(XLogReaderState *state); static XLogPageReadResult XLogDecodeNextRecord(XLogReaderState *state, bool nonblocking); static bool ValidXLogRecordHeader(XLogReaderState *state, XLogRecPtr RecPtr, - XLogRecPtr PrevRecPtr, XLogRecord *record, bool randAccess); -static bool ValidXLogRecord(XLogReaderState *state, XLogRecord *record, + XLogRecPtr PrevRecPtr, XLogRecHdr rechdr, + bool randAccess); +static bool ValidXLogRecord(XLogReaderState *state, XLogRecHdr record, XLogRecPtr recptr); static void ResetDecoder(XLogReaderState *state); static void WALOpenSegmentInit(WALOpenSegment *seg, WALSegmentContext *segcxt, @@ -535,7 +536,7 @@ static XLogPageReadResult XLogDecodeNextRecord(XLogReaderState *state, bool nonblocking) { XLogRecPtr RecPtr; - XLogRecord *record; + XLogRecHdr record; XLogRecPtr targetPagePtr; bool randAccess; uint32 len, @@ -602,7 +603,7 @@ restart: * fits on the same page. */ readOff = ReadPageInternal(state, targetPagePtr, - Min(targetRecOff + SizeOfXLogRecord, XLOG_BLCKSZ)); + Min(targetRecOff + XLogRecordMaxHdrSize, XLOG_BLCKSZ)); if (readOff == XLREAD_WOULDBLOCK) return XLREAD_WOULDBLOCK; else if (readOff < 0) @@ -644,13 +645,13 @@ restart: * Read the record length. * * NB: Even though we use an XLogRecord pointer here, the whole record - * header might not fit on this page. xl_tot_len is the first field of the - * struct, so it must be on this page (the records are MAXALIGNed), but we - * cannot access any other fields until we've verified that we got the - * whole header. + * header might not fit on this page. xl_payload_len is the third field of + * the data and starts at the third byte, so it must be on this page (the + * records are aligned to 8 bytes), but we cannot access most other fields + * until we've verified that we got the whole header. */ - record = (XLogRecord *) (state->readBuf + RecPtr % XLOG_BLCKSZ); - total_len = record->xl_tot_len; + record = (XLogRecHdr) (state->readBuf + (RecPtr % XLOG_BLCKSZ)); + total_len = XLogRecordTotalLength(record); /* * If the whole record header is on this page, validate it immediately. @@ -660,7 +661,7 @@ restart: * record" code path below; otherwise we might fail to apply * ValidXLogRecordHeader at all. */ - if (targetRecOff <= XLOG_BLCKSZ - SizeOfXLogRecord) + if (targetRecOff <= XLOG_BLCKSZ - XLogRecordHdrLen(record->xl_info)) { if (!ValidXLogRecordHeader(state, RecPtr, state->DecodeRecPtr, record, randAccess)) @@ -683,12 +684,12 @@ restart: LSN_FORMAT_ARGS(RecPtr)); goto err; } - if (total_len < SizeOfXLogRecord) + if (total_len < XLogRecordMinHdrSize) { report_invalid_record(state, "invalid record length at %X/%X: expected at least %u, got %u", LSN_FORMAT_ARGS(RecPtr), - (uint32) SizeOfXLogRecord, total_len); + (uint32) XLogRecordMinHdrSize, total_len); goto err; } gotheader = false; @@ -831,7 +832,7 @@ restart: /* If we just reassembled the record header, validate it. */ if (!gotheader) { - record = (XLogRecord *) state->readRecordBuf; + record = (XLogRecHdr) state->readRecordBuf; if (!ValidXLogRecordHeader(state, RecPtr, state->DecodeRecPtr, record, randAccess)) goto err; @@ -841,14 +842,14 @@ restart: Assert(gotheader); - record = (XLogRecord *) state->readRecordBuf; + record = (XLogRecHdr) state->readRecordBuf; if (!ValidXLogRecord(state, record, RecPtr)) goto err; pageHeaderSize = XLogPageHeaderSize((XLogPageHeader) state->readBuf); state->DecodeRecPtr = RecPtr; state->NextRecPtr = targetPagePtr + pageHeaderSize - + MAXALIGN(pageHeader->xlp_rem_len); + + XLP_ALIGN(pageHeader->xlp_rem_len); } else { @@ -864,7 +865,7 @@ restart: if (!ValidXLogRecord(state, record, RecPtr)) goto err; - state->NextRecPtr = RecPtr + MAXALIGN(total_len); + state->NextRecPtr = RecPtr + XLP_ALIGN(total_len); state->DecodeRecPtr = RecPtr; } @@ -873,7 +874,7 @@ restart: * Special processing if it's an XLOG SWITCH record */ if (record->xl_rmid == RM_XLOG_ID && - record->xl_rmgrinfo == XLOG_SWITCH) + XLRHdrGetRmgrInfo(record) == XLOG_SWITCH) { /* Pretend it extends to end of segment */ state->NextRecPtr += state->segcxt.ws_segsize - 1; @@ -892,7 +893,7 @@ restart: if (!decoded->oversized) { /* The new decode buffer head must be MAXALIGNed. */ - Assert(decoded->size == MAXALIGN(decoded->size)); + Assert(decoded->size == XLP_ALIGN(decoded->size)); if ((char *) decoded == state->decode_buffer) state->decode_buffer_tail = state->decode_buffer + decoded->size; else @@ -1123,22 +1124,25 @@ XLogReaderInvalReadState(XLogReaderState *state) */ static bool ValidXLogRecordHeader(XLogReaderState *state, XLogRecPtr RecPtr, - XLogRecPtr PrevRecPtr, XLogRecord *record, + XLogRecPtr PrevRecPtr, XLogRecHdr rechdr, bool randAccess) { - if (record->xl_tot_len < SizeOfXLogRecord) + XLogRecord record = {0}; + XLogReadRecHdrInto(rechdr, XLogRecordHdrLen(rechdr->xl_info), &record); + + if (XLogRecordTotalLength(rechdr) < XLogRecordMinHdrSize) { report_invalid_record(state, "invalid record length at %X/%X: expected at least %u, got %u", LSN_FORMAT_ARGS(RecPtr), - (uint32) SizeOfXLogRecord, record->xl_tot_len); + (uint32) XLogRecordMinHdrSize, record.xl_payload_len); return false; } - if (!RmgrIdIsValid(record->xl_rmid)) + if (!RmgrIdIsValid(rechdr->xl_rmid)) { report_invalid_record(state, "invalid resource manager ID %u at %X/%X", - record->xl_rmid, LSN_FORMAT_ARGS(RecPtr)); + rechdr->xl_rmid, LSN_FORMAT_ARGS(RecPtr)); return false; } if (randAccess) @@ -1147,12 +1151,14 @@ ValidXLogRecordHeader(XLogReaderState *state, XLogRecPtr RecPtr, * We can't exactly verify the prev-link, but surely it should be less * than the record's own address. */ - if (!(record->xl_prev < RecPtr)) + if (!(record.xl_prev < RecPtr)) { report_invalid_record(state, - "record with incorrect prev-link %X/%X at %X/%X", - LSN_FORMAT_ARGS(record->xl_prev), - LSN_FORMAT_ARGS(RecPtr)); + "record with incorrect prev-link %X/%X at %X/%X, info %u, rmgr %u", + LSN_FORMAT_ARGS(record.xl_prev), + LSN_FORMAT_ARGS(RecPtr), + (uint32) record.xl_info, + (uint32) record.xl_rmid); return false; } } @@ -1163,12 +1169,13 @@ ValidXLogRecordHeader(XLogReaderState *state, XLogRecPtr RecPtr, * check guards against torn WAL pages where a stale but valid-looking * WAL record starts on a sector boundary. */ - if (record->xl_prev != PrevRecPtr) + if (record.xl_prev != PrevRecPtr) { report_invalid_record(state, - "record with incorrect prev-link %X/%X at %X/%X", - LSN_FORMAT_ARGS(record->xl_prev), - LSN_FORMAT_ARGS(RecPtr)); + "record with incorrect prev-link %X/%X at %X/%X, expected %X/%X", + LSN_FORMAT_ARGS(record.xl_prev), + LSN_FORMAT_ARGS(RecPtr), + LSN_FORMAT_ARGS(PrevRecPtr)); return false; } } @@ -1188,19 +1195,26 @@ ValidXLogRecordHeader(XLogReaderState *state, XLogRecPtr RecPtr, * SizeOfXLogRecord. */ static bool -ValidXLogRecord(XLogReaderState *state, XLogRecord *record, XLogRecPtr recptr) +ValidXLogRecord(XLogReaderState *state, XLogRecHdr record, XLogRecPtr recptr) { - pg_crc32c crc; + pg_crc32c crc, + hdr_crc; + Size hdr_len = XLogRecordHdrLen(record->xl_info); + Size rec_length = XLogRecordTotalLength(record); /* Calculate the CRC */ INIT_CRC32C(crc); - COMP_CRC32C(crc, ((char *) record) + SizeOfXLogRecord, record->xl_tot_len - SizeOfXLogRecord); + COMP_CRC32C(crc, ((char *) record) + hdr_len, rec_length - hdr_len); /* include the record header last */ - COMP_CRC32C(crc, (char *) record, offsetof(XLogRecord, xl_crc)); + COMP_CRC32C(crc, (char *) record, hdr_len - sizeof(pg_crc32c)); FIN_CRC32C(crc); - if (!EQ_CRC32C(record->xl_crc, crc)) + memcpy(&hdr_crc, ((char *) record) + hdr_len - sizeof(pg_crc32c), + sizeof(pg_crc32c)); + + if (!EQ_CRC32C(hdr_crc, crc)) { + Assert(false); report_invalid_record(state, "incorrect resource manager data checksum in record at %X/%X", LSN_FORMAT_ARGS(recptr)); @@ -1440,7 +1454,7 @@ XLogFindNextRecord(XLogReaderState *state, XLogRecPtr RecPtr) * * Note that record headers are MAXALIGN'ed */ - if (MAXALIGN(header->xlp_rem_len) >= (XLOG_BLCKSZ - pageHeaderSize)) + if (XLP_ALIGN(header->xlp_rem_len) >= (XLOG_BLCKSZ - pageHeaderSize)) tmpRecPtr = targetPagePtr + XLOG_BLCKSZ; else { @@ -1449,7 +1463,7 @@ XLogFindNextRecord(XLogReaderState *state, XLogRecPtr RecPtr) * tmpRecPtr to point to the first valid record */ tmpRecPtr = targetPagePtr + pageHeaderSize - + MAXALIGN(header->xlp_rem_len); + + XLP_ALIGN(header->xlp_rem_len); break; } } @@ -1660,7 +1674,7 @@ DecodeXLogRecordRequiredSpace(size_t xl_tot_len) bool DecodeXLogRecord(XLogReaderState *state, DecodedXLogRecord *decoded, - XLogRecord *record, + XLogRecHdr record, XLogRecPtr lsn, char **errormsg) { @@ -1683,7 +1697,7 @@ DecodeXLogRecord(XLogReaderState *state, RelFileLocator *rlocator = NULL; uint8 block_id; - decoded->header = *record; + XLogReadRecHdrInto(record, XLogRecordHdrLen(record->xl_info), &decoded->header); decoded->lsn = lsn; decoded->next = NULL; decoded->record_origin = InvalidRepOriginId; @@ -1692,8 +1706,8 @@ DecodeXLogRecord(XLogReaderState *state, decoded->main_data_len = 0; decoded->max_block_id = -1; ptr = (char *) record; - ptr += SizeOfXLogRecord; - remaining = record->xl_tot_len - SizeOfXLogRecord; + ptr += XLogRecordHdrLen(record->xl_info); + remaining = decoded->header.xl_payload_len; /* Decode the headers */ datatotal = 0; @@ -1968,8 +1982,8 @@ DecodeXLogRecord(XLogReaderState *state, } /* Report the actual size we used. */ - decoded->size = MAXALIGN(out - (char *) decoded); - Assert(DecodeXLogRecordRequiredSpace(record->xl_tot_len) >= + decoded->size = XLP_ALIGN(out - (char *) decoded); + Assert(DecodeXLogRecordRequiredSpace(decoded->header.xl_payload_len) >= decoded->size); return true; @@ -1998,6 +2012,7 @@ XLogRecGetBlockTag(XLogReaderState *record, uint8 block_id, if (!XLogRecGetBlockTagExtended(record, block_id, rlocator, forknum, blknum, NULL)) { + Assert(false); #ifndef FRONTEND elog(ERROR, "could not locate backup block with ID %d in WAL record", block_id); diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index 56fa2f74a2..0362ba747f 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -4000,10 +4000,10 @@ ReadCheckpointRecord(XLogPrefetcher *xlogprefetcher, XLogRecPtr RecPtr, (errmsg("invalid xl_rmgrinfo in checkpoint record"))); return NULL; } - if (record->xl_tot_len != SizeOfXLogRecord + SizeOfXLogRecordDataHeaderShort + sizeof(CheckPoint)) + if (record->xl_payload_len != SizeOfXLogRecordDataHeaderShort + sizeof(CheckPoint)) { ereport(LOG, - (errmsg("invalid length of checkpoint record"))); + (errmsg("invalid payload length of checkpoint record"))); return NULL; } return record; diff --git a/src/bin/pg_resetwal/pg_resetwal.c b/src/bin/pg_resetwal/pg_resetwal.c index 8e61a7117e..6edc477c6c 100644 --- a/src/bin/pg_resetwal/pg_resetwal.c +++ b/src/bin/pg_resetwal/pg_resetwal.c @@ -1044,12 +1044,15 @@ WriteEmptyXLOG(void) PGAlignedXLogBlock buffer; XLogPageHeader page; XLogLongPageHeader longpage; - XLogRecord *record; pg_crc32c crc; char path[MAXPGPATH]; int fd; int nbytes; char *recptr; + char *baserecptr; + const int rec_hdr_len = offsetof(XLogRecHdrData, xl_hdrdata) + + sizeof(uint8) + sizeof(XLogRecPtr) + sizeof(pg_crc32c); + const int rec_payload_len = sizeof(uint8) * 2 + sizeof(CheckPoint); memset(buffer.data, 0, XLOG_BLCKSZ); @@ -1065,26 +1068,33 @@ WriteEmptyXLOG(void) longpage->xlp_xlog_blcksz = XLOG_BLCKSZ; /* Insert the initial checkpoint record */ - recptr = (char *) page + SizeOfXLogLongPHD; - record = (XLogRecord *) recptr; - record->xl_prev = 0; - record->xl_xid = InvalidTransactionId; - record->xl_tot_len = SizeOfXLogRecord + SizeOfXLogRecordDataHeaderShort + sizeof(CheckPoint); - record->xl_info = 0; - record->xl_rmgrinfo = XLOG_CHECKPOINT_SHUTDOWN; - record->xl_rmid = RM_XLOG_ID; - - recptr += SizeOfXLogRecord; + baserecptr = recptr = (char *) page + SizeOfXLogLongPHD; + *(recptr++) = (char) XLS_UINT8; + *(recptr++) = (char) RM_XLOG_ID; + recptr += XLogWriteLength(SizeOfXLogRecordDataHeaderShort + sizeof(CheckPoint), + XLS_UINT8, XLS_UINT8, recptr); + + /* include prevptr */ + { + XLogRecPtr prevptr = 0; + memcpy(recptr, &prevptr, sizeof(XLogRecPtr)); + recptr += sizeof(XLogRecPtr); + } + /* reserve location of crc */ + recptr += sizeof(pg_crc32c); + *(recptr++) = (char) XLR_BLOCK_ID_DATA_SHORT; *(recptr++) = sizeof(CheckPoint); memcpy(recptr, &ControlFile.checkPointCopy, sizeof(CheckPoint)); INIT_CRC32C(crc); - COMP_CRC32C(crc, ((char *) record) + SizeOfXLogRecord, record->xl_tot_len - SizeOfXLogRecord); - COMP_CRC32C(crc, (char *) record, offsetof(XLogRecord, xl_crc)); + COMP_CRC32C(crc, baserecptr + rec_hdr_len, rec_payload_len); + COMP_CRC32C(crc, baserecptr, rec_hdr_len - sizeof(pg_crc32c)); FIN_CRC32C(crc); - record->xl_crc = crc; + + memcpy(baserecptr + rec_hdr_len - sizeof(pg_crc32c), + &crc, sizeof(pg_crc32c)); /* Write the first page */ XLogFilePath(path, ControlFile.checkPointCopy.ThisTimeLineID, diff --git a/src/include/access/xlog_internal.h b/src/include/access/xlog_internal.h index 2277b32d05..8b72d947c2 100644 --- a/src/include/access/xlog_internal.h +++ b/src/include/access/xlog_internal.h @@ -26,7 +26,15 @@ #include "pgtime.h" #include "storage/block.h" #include "storage/relfilelocator.h" +#include "transam.h" +/* + * WAL records (and other XLogPage page-level content) are aligned to 8 bytes. + * + * Note that the contents of the records is not aligned (!) + */ +#define ALIGNOF_XLP_CONTENT 8 +#define XLP_ALIGN(LEN) TYPEALIGN64(ALIGNOF_XLP_CONTENT, LEN) /* * Each page of XLOG file has a header like this: @@ -49,7 +57,7 @@ typedef struct XLogPageHeaderData uint32 xlp_rem_len; /* total len of remaining data for record */ } XLogPageHeaderData; -#define SizeOfXLogShortPHD MAXALIGN(sizeof(XLogPageHeaderData)) +#define SizeOfXLogShortPHD XLP_ALIGN(sizeof(XLogPageHeaderData)) typedef XLogPageHeaderData *XLogPageHeader; @@ -66,7 +74,7 @@ typedef struct XLogLongPageHeaderData uint32 xlp_xlog_blcksz; /* just as a cross-check */ } XLogLongPageHeaderData; -#define SizeOfXLogLongPHD MAXALIGN(sizeof(XLogLongPageHeaderData)) +#define SizeOfXLogLongPHD XLP_ALIGN(sizeof(XLogLongPageHeaderData)) typedef XLogLongPageHeaderData *XLogLongPageHeader; @@ -392,7 +400,7 @@ static inline int XLogWriteLength(uint32 length, XLogSizeClass sizeClass, case caseSC: \ if ((caseSC) <= maxSizeClass) \ { \ - field_type typedLength = length; \ + field_type typedLength = (field_type) length; \ memcpy(output, &typedLength, sizeof(field_type)); \ written = sizeof(field_type); \ } \ @@ -405,6 +413,9 @@ static inline int XLogWriteLength(uint32 length, XLogSizeClass sizeClass, WRITE_OP(XLS_UINT8, uint8); WRITE_OP(XLS_UINT16, uint16); WRITE_OP(XLS_UINT32, uint32); + default: + Assert(false); + pg_unreachable(); } #undef WRITE_OP @@ -451,12 +462,95 @@ static inline int XLogReadLength(uint32 *length, XLogSizeClass sizeClass, READ_OP(XLS_UINT8, uint8); READ_OP(XLS_UINT16, uint16); READ_OP(XLS_UINT32, uint32); + default: + Assert(false); + pg_unreachable(); } #undef READ_OP return readSize; } + +inline static uint8 XLRHdrGetRmgrInfo(XLogRecHdr record) +{ + XLogSizeClass recSizeClass; + int offset; + + if (!(record->xl_info & XLR_HAS_RMGRINFO)) + return 0; + + recSizeClass = XLR_SIZECLASS(record->xl_info); + /* xl_rmgrinfo is located immediately behind the xl_payload_len field */ + offset = XLogSizeClassToByteLength(recSizeClass); + + return (uint8) record->xl_hdrdata[offset]; +} + + +/* Works on any partial record */ +inline static Size XLogRecordTotalLength(XLogRecHdr record) +{ + uint8 xl_info = record->xl_info; + XLogSizeClass sizeClass; + uint32 length = 0; + sizeClass = XLR_SIZECLASS(xl_info); + + XLogReadLength(&length, sizeClass, XLS_UINT32, + &record->xl_hdrdata[0], 6); + + return (Size) length + XLogRecordHdrLen(xl_info); +} + +inline static void XLogReadRecHdrInto(XLogRecHdr recdata, Size length, + XLogRecord *record) +{ + Size offset = 0; + Size hdr_size PG_USED_FOR_ASSERTS_ONLY = 0; + XLogSizeClass sizeClass; + + Assert(length >= XLogRecordMinHdrSize); + + record->xl_info = recdata->xl_info; + + hdr_size = XLogRecordHdrLen(record->xl_info); + Assert(length >= hdr_size); + + record->xl_rmid = recdata->xl_rmid; + + sizeClass = XLR_SIZECLASS(record->xl_info); + offset += XLogReadLength(&record->xl_payload_len, sizeClass, + XLS_UINT32, &recdata->xl_hdrdata[offset], length - offset); + + if (record->xl_info & XLR_HAS_RMGRINFO) + { + record->xl_rmgrinfo = recdata->xl_hdrdata[offset]; + offset += sizeof(uint8); + } + else + { + record->xl_rmgrinfo = 0; + } + + if (record->xl_info & XLR_HAS_XID) + { + memcpy(&record->xl_xid, &recdata->xl_hdrdata[offset], sizeof(TransactionId)); + offset += sizeof(TransactionId); + } + else + { + record->xl_xid = InvalidTransactionId; + } + + memcpy(&record->xl_prev, &recdata->xl_hdrdata[offset], sizeof(XLogRecPtr)); + offset += sizeof(XLogRecPtr); + + memcpy(&record->xl_crc, &recdata->xl_hdrdata[offset], sizeof(pg_crc32c)); + offset += sizeof(pg_crc32c); + + Assert(hdr_size - 2 == offset); +} + /* * Method table for resource managers. * diff --git a/src/include/access/xlogreader.h b/src/include/access/xlogreader.h index a1d0216404..bb64f89dba 100644 --- a/src/include/access/xlogreader.h +++ b/src/include/access/xlogreader.h @@ -397,7 +397,7 @@ extern bool WALRead(XLogReaderState *state, extern size_t DecodeXLogRecordRequiredSpace(size_t xl_tot_len); extern bool DecodeXLogRecord(XLogReaderState *state, DecodedXLogRecord *decoded, - XLogRecord *record, + XLogRecHdr record, XLogRecPtr lsn, char **errormsg); @@ -405,7 +405,7 @@ extern bool DecodeXLogRecord(XLogReaderState *state, * Macros that provide access to parts of the record most recently returned by * XLogReadRecord() or XLogNextRecord(). */ -#define XLogRecGetTotalLen(decoder) ((decoder)->record->header.xl_tot_len) +#define XLogRecGetTotalLen(decoder) ((decoder)->record->header.xl_payload_len) #define XLogRecGetPrev(decoder) ((decoder)->record->header.xl_prev) #define XLogRecGetInfo(decoder) ((decoder)->record->header.xl_info) #define XLogRecGetRmgrInfo(decoder) ((decoder)->record->header.xl_rmgrinfo) diff --git a/src/include/access/xlogrecord.h b/src/include/access/xlogrecord.h index 0f32f20591..a0b054e1b5 100644 --- a/src/include/access/xlogrecord.h +++ b/src/include/access/xlogrecord.h @@ -39,9 +39,38 @@ typedef enum XLogSizeClass { XLS_UINT32 = 3 /* length <= UINT32_MAX; stored in uint32 (4B) */ } XLogSizeClass; +static inline int XLogSizeClassToByteLength(XLogSizeClass sz) +{ + switch (sz) { + case XLS_EMPTY: + return 0; + case XLS_UINT8: + return sizeof(uint8); + case XLS_UINT16: + return sizeof(uint16); + case XLS_UINT32: + return sizeof(uint32); + default: + pg_unreachable(); + } +} + +typedef struct XLogRecord +{ + uint8 xl_info; /* flag bits, see below */ + RmgrId xl_rmid; /* resource manager for this record */ + uint32 xl_payload_len; /* total len of entire record */ + uint8 xl_rmgrinfo; /* rmgr flag bits, see below */ + TransactionId xl_xid; /* xact id */ + XLogRecPtr xl_prev; /* ptr to previous record in log */ + pg_crc32c xl_crc; /* CRC for this record */ +} XLogRecord; + +#define SizeOfXLogRecord (offsetof(XLogRecord, xl_crc) + sizeof(pg_crc32c)) + /* * The overall layout of an XLOG record is: - * Fixed-size header (XLogRecord struct) + * Variable-size header (containing the data of the XLogRecord struct) * XLogRecordBlockHeader struct * XLogRecordBlockHeader struct * ... @@ -51,6 +80,15 @@ typedef enum XLogSizeClass { * ... * main data * + * Different xlog record headers need to store different header fields, so + * depending on flags in the xl_info field the layout may change. + * + * Note that for records with a payload larger than 0xFFFFFFF, the size field + * will be 4 bytes long, which can be larger than the minimum MAXALIGN quantum. + * To still be able to read the record correctly, we always align WAL to 8 + * bytes, so that the length bytes are always present in the first aligned + * quantum of data. + * * There can be zero or more XLogRecordBlockHeaders, and 0 or more bytes of * rmgr-specific data not associated with a block. XLogRecord structs * always start on MAXALIGN boundaries in the WAL files, but the rest of @@ -60,22 +98,25 @@ typedef enum XLogSizeClass { * XLogRecordDataHeaderLong structs all begin with a single 'id' byte. It's * used to distinguish between block references, and the main data structs. */ -typedef struct XLogRecord +typedef struct XLogRecHdrData { - uint32 xl_tot_len; /* total len of entire record */ - TransactionId xl_xid; /* xact id */ - XLogRecPtr xl_prev; /* ptr to previous record in log */ uint8 xl_info; /* flag bits, see below */ RmgrId xl_rmid; /* resource manager for this record */ - uint8 xl_rmgrinfo; /* rmgr flag bits, see below */ - /* 1 byte of padding here, initialize to zero */ - pg_crc32c xl_crc; /* CRC for this record */ - - /* XLogRecordBlockHeaders and XLogRecordDataHeader follow, no padding */ + char xl_hdrdata[FLEXIBLE_ARRAY_MEMBER]; /* variable length record data */ -} XLogRecord; + /* + * Payload is as follows, in order: + * XLogRecord data encoded + * xl_payload_len; 0, 1, 2 or 4 bytes + * xl_rmgrinfo; 0 or 1 byte + * xl_xid; 0 or 4 bytes + * xl_prev; 8 bytes + * xl_prev follows; 8 bytes + * record payload, if present. + */ +} XLogRecHdrData; -#define SizeOfXLogRecord (offsetof(XLogRecord, xl_crc) + sizeof(pg_crc32c)) +typedef XLogRecHdrData *XLogRecHdr; /* * XLogReader needs to allocate all the data of a WAL record in a single @@ -88,13 +129,28 @@ typedef struct XLogRecord */ #define XLogRecordMaxSize (1020 * 1024 * 1024) +#define XLR_SIZECLASS_MASK 0x03 +#define XLR_SIZECLASS(xl_info) ((XLogSizeClass) ((xl_info) & XLR_SIZECLASS_MASK)) + +/* + * There are several rmgrs which don't use info bits, so we omit that + * byte whenever possible. + */ +#define XLR_HAS_RMGRINFO 0x04 + +/* + * If a WAL record uses the current transaction ID, that will be included + * in the record header. In all other cases we omit the XID to save bytes. + */ +#define XLR_HAS_XID 0x08 + /* * If a WAL record modifies any relation files, in ways not covered by the * usual block references, this flag is set. This is not used for anything * by PostgreSQL itself, but it allows external tools that read WAL and keep * track of modified blocks to recognize such special record types. */ -#define XLR_SPECIAL_REL_UPDATE 0x01 +#define XLR_SPECIAL_REL_UPDATE 0x10 /* * Enforces consistency checks of replayed WAL at recovery. If enabled, @@ -103,7 +159,48 @@ typedef struct XLogRecord * of XLogInsert can use this value if necessary, but if * wal_consistency_checking is enabled for a rmgr this is set unconditionally. */ -#define XLR_CHECK_CONSISTENCY 0x02 +#define XLR_CHECK_CONSISTENCY (0x20) + +#define XLogRecordMaxHdrSize ( \ + sizeof(uint8) /* xl_info */ + \ + sizeof(uint8) /* xl_rmid */ + \ + sizeof(uint32) /* xl_payload_len */ + \ + sizeof(uint8) /* xl_rmgrinfo */ + \ + sizeof(TransactionId) /* xl_xid */ + \ + sizeof(XLogRecPtr) /* xl_prev */ + \ + sizeof(pg_crc32c) /* xl_crc */ \ +) + +#define XLogRecordMinHdrSize ( \ + sizeof(uint8) /* xl_info */ + \ + sizeof(uint8) /* xl_rmid */ + \ + 0 /* xl_payload_len */ + \ + 0 /* xl_rmgrinfo */ + \ + 0 /* xl_xid */ + \ + sizeof(XLogRecPtr) /* xl_prev */ + \ + sizeof(pg_crc32c) /* xl_crc */ \ +) + +static inline Size XLogRecordHdrLen(uint8 info) +{ + Size size = 0; + /* xl_info */ + size += sizeof(uint8); + /* xl_rmid */ + size += sizeof(uint8); + /* xl_payload_len */ + size += XLogSizeClassToByteLength(XLR_SIZECLASS(info)); + /* xl_rmgrinfo */ + size += ((info) & XLR_HAS_RMGRINFO) ? sizeof(uint8) : 0; + /* xl_xid */ + size += ((info) & XLR_HAS_XID) ? sizeof(TransactionId) : 0; + /* xl_prev */ + size += sizeof(XLogRecPtr); + /* xl_crc */ + size += sizeof(pg_crc32c); + + return size; +} /* * Header info for block data appended to an XLOG record. -- 2.40.1 ^ permalink raw reply [nested|flat] 35+ messages in thread
end of thread, other threads:[~2023-09-19 10:07 UTC | newest] Thread overview: 35+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2020-02-15 21:53 [PATCH v3 7/7] Update comment obsolete since 69c3936a Justin Pryzby <[email protected]> 2020-02-15 21:53 [PATCH 02/18] Update comment obsolete since 69c3936a Justin Pryzby <[email protected]> 2020-02-15 21:53 [PATCH v9 8/8] Update comment obsolete since 69c3936a Justin Pryzby <[email protected]> 2020-02-15 21:53 [PATCH 02/18] Update comment obsolete since 69c3936a Justin Pryzby <[email protected]> 2020-02-15 21:53 [PATCH 02/18] Update comment obsolete since 69c3936a Justin Pryzby <[email protected]> 2020-02-15 21:53 [PATCH 02/18] Update comment obsolete since 69c3936a Justin Pryzby <[email protected]> 2020-02-15 21:53 [PATCH 02/18] Update comment obsolete since 69c3936a Justin Pryzby <[email protected]> 2020-02-15 21:53 [PATCH 02/18] Update comment obsolete since 69c3936a Justin Pryzby <[email protected]> 2020-02-15 21:53 [PATCH v4 7/7] Update comment obsolete since 69c3936a Justin Pryzby <[email protected]> 2020-02-15 21:53 [PATCH 02/18] Update comment obsolete since 69c3936a Justin Pryzby <[email protected]> 2020-02-15 21:53 [PATCH 02/18] Update comment obsolete since 69c3936a Justin Pryzby <[email protected]> 2020-02-15 21:53 [PATCH 02/18] Update comment obsolete since 69c3936a Justin Pryzby <[email protected]> 2020-02-15 21:53 [PATCH 02/18] Update comment obsolete since 69c3936a Justin Pryzby <[email protected]> 2020-02-15 21:53 [PATCH 02/18] Update comment obsolete since 69c3936a Justin Pryzby <[email protected]> 2020-02-15 21:53 [PATCH v5 7/7] Update comment obsolete since 69c3936a Justin Pryzby <[email protected]> 2020-02-15 21:53 [PATCH 02/18] Update comment obsolete since 69c3936a Justin Pryzby <[email protected]> 2020-02-15 21:53 [PATCH 02/18] Update comment obsolete since 69c3936a Justin Pryzby <[email protected]> 2020-02-15 21:53 [PATCH 02/18] Update comment obsolete since 69c3936a Justin Pryzby <[email protected]> 2020-02-15 21:53 [PATCH 02/18] Update comment obsolete since 69c3936a Justin Pryzby <[email protected]> 2020-02-15 21:53 [PATCH v2 7/7] Update comment obsolete since 69c3936a Justin Pryzby <[email protected]> 2020-02-15 21:53 [PATCH 02/18] Update comment obsolete since 69c3936a Justin Pryzby <[email protected]> 2020-02-15 21:53 [PATCH 02/18] Update comment obsolete since 69c3936a Justin Pryzby <[email protected]> 2020-02-15 21:53 [PATCH 02/18] Update comment obsolete since 69c3936a Justin Pryzby <[email protected]> 2020-02-15 21:53 [PATCH 02/21] Update comment obsolete since 69c3936a Justin Pryzby <[email protected]> 2020-02-15 21:53 [PATCH 02/18] Update comment obsolete since 69c3936a Justin Pryzby <[email protected]> 2020-02-15 21:53 [PATCH 02/18] Update comment obsolete since 69c3936a Justin Pryzby <[email protected]> 2020-02-15 21:53 [PATCH v7 7/9] Update comment obsolete since 69c3936a Justin Pryzby <[email protected]> 2020-02-15 21:53 [PATCH v6 7/7] Update comment obsolete since 69c3936a Justin Pryzby <[email protected]> 2020-02-15 21:53 [PATCH 02/18] Update comment obsolete since 69c3936a Justin Pryzby <[email protected]> 2020-02-15 21:53 [PATCH 02/18] Update comment obsolete since 69c3936a Justin Pryzby <[email protected]> 2023-06-20 20:01 XLog size reductions: Reduced XLog record header size for PG17 Matthias van de Meent <[email protected]> 2023-06-30 15:36 ` Re: XLog size reductions: Reduced XLog record header size for PG17 Matthias van de Meent <[email protected]> 2023-07-03 11:08 ` Re: XLog size reductions: Reduced XLog record header size for PG17 Matthias van de Meent <[email protected]> 2023-07-12 12:50 ` Re: XLog size reductions: Reduced XLog record header size for PG17 Matthias van de Meent <[email protected]> 2023-09-19 10:07 ` Re: XLog size reductions: Reduced XLog record header size for PG17 Matthias van de Meent <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox