public inbox for [email protected]
help / color / mirror / Atom feed[PATCH 2/3] Allow composite types in bootstrap
2+ messages / 2 participants
[nested] [flat]
* [PATCH 2/3] Allow composite types in bootstrap
@ 2020-11-17 15:28 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 2+ messages in thread
From: Justin Pryzby @ 2020-11-17 15:28 UTC (permalink / raw)
---
src/backend/bootstrap/bootstrap.c | 28 ++++++++++++++++++++++++++++
1 file changed, 28 insertions(+)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 18eb62ca47..e4fc75ab84 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -916,6 +916,7 @@ gettype(char *type)
{
if (Typ != NIL)
{
+ static bool did_reread PG_USED_FOR_ASSERTS_ONLY = false; /* Already reread pg_types */
ListCell *lc;
foreach (lc, Typ)
@@ -927,6 +928,33 @@ gettype(char *type)
return app->am_oid;
}
}
+
+ /*
+ * The type wasn't known; check again to handle composite
+ * types, added since first populating the array.
+ */
+
+ /*
+ * Once all the types are populated and we handled composite
+ * types, shouldn't need to do that again.
+ */
+ Assert(!did_reread);
+ did_reread = true;
+
+ list_free_deep(Typ);
+ Typ = NULL;
+ populate_typ_array();
+
+ /* Need to avoid infinite recursion... */
+ foreach (lc, Typ)
+ {
+ struct typmap *app = lfirst(lc);
+ if (strncmp(NameStr(app->am_typ.typname), type, NAMEDATALEN) == 0)
+ {
+ Ap = app;
+ return app->am_oid;
+ }
+ }
}
else
{
--
2.26.2
--------------8FD28E9B65A94BB176003065
Content-Type: text/x-patch; charset=UTF-8;
name="0003-Extended-statistics-on-expressions-20210116.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="0003-Extended-statistics-on-expressions-20210116.patch"
^ permalink raw reply [nested|flat] 2+ messages in thread
* More tuple deformation speedups
@ 2024-10-30 21:27 David Rowley <[email protected]>
0 siblings, 0 replies; 2+ messages in thread
From: David Rowley @ 2024-10-30 21:27 UTC (permalink / raw)
To: PostgreSQL Developers <[email protected]>
While working on the other tuple deformation speedup patches in [1], I
noticed that the code in slot_deform_heap_tuple() is a bit "overly
branchy" and could be done more efficiently with the following
assumptions:
1. We only need to switch from !slow mode into slow mode, never the
other way around (for a given tuple)
2. We know upfront if a tuple contains any NULLs by checking the
infomask for HEAP_HASNULL.
Because of #1, we can lay the code out as two loops, the first of
which is coded to assume we're not in slow mode and anywhere we do "if
(!slow)" we can get rid of those checks as we already know we're not
in slow mode. Instead of having those checks, we can just break out
of the first loop if we hit a NULL or variable length attribute and
fall into a 2nd loop which has all of the "if (!slow)" branches
eliminated and does not have any code to cache the offset.
Because of #2, we can forego checking the "if (hasnulls &&
att_isnull(attnum, bp))" for every single attribute. If "hasnulls" is
false, there's no need to check that on every loop. We can simply have
a dedicated loop that handles !slow and !hasnulls. I think having no
nulls in a tuple is very common, so seems worthwhile having a version
without the NULL checks.
I think the best way to form this as C code is to have an always
inline function that we call with various combinations of "slow" and
"hasnulls" and allow the compiler to emit specialised code for the
various scenarios. I've done this in the attached and made it so there
are 3 specialisations 1) !slow && !hasnulls 2) !slow && hasnulls 3)
slow && (hasnulls || !hasnulls).
(I resisted having a dedicated version for slow && !hasnulls. It might
be worthwhile having that.)
I did some benchmarking of this with 3 different scenarios and with
each scenario, I did a count(col) on the 16th column in the given
table. The variation between each of the three test comes from the
first column in the table:
* t1: Zero NULLs and all columns fixed-width types
* t2: First column has a NULL value. Other columns on all rows are not
null. (forces slow mode early on)
* t3: First column is a varlena type all other columns fixed-width. No
nulls anywhere. (also forces slow mode early on)
I ran the benchmark using the attached script on 3 different machines
and graphed them. See attached deform_specialisations_bench.png.
There is a slowdown on test 2 with the Zen2 machine. The other patches
I have in [2] help reduce the amount of code required to figure out
the attribute alignment, so that helps to reduce the amount of
additional code that the compiler emits when I apply all patches
together. With that, the Zen2 regression goes away and overall results
on all 3 machines look nicer. See attached all_patches_bench.png
The attached v1-0001 does not apply cleanly on top of the patches in
[2], so I've also attached a version that does for anyone who wants to
try it out with the other patches.
I'd like to go ahead with the patches in [2] first then loop back
around to this one again. I'm posting here rather than overloading
that other thread with the new patch.
David
[1] https://postgr.es/m/CAApHDvrBztXP3yx%3DNKNmo3xwFAFhEdyPnvrDg3%3DM0RhDs%2B4vYw%40mail.gmail.com
[2] https://postgr.es/m/CAApHDvpwd76-goJ3J-g_VQEzhqqb7F-3Kd70LXNrS23UHYSLBg%40mail.gmail.com
#!/bin/bash
dbname=postgres
secs=15
rows=10000
extra_cols_start=16
extra_cols_end=16
psql -c "alter system set max_parallel_workers_per_gather = 0;" $dbname > /dev/null
psql -c "alter system set jit = 0;" $dbname > /dev/null
psql -c "select pg_reload_conf();" $dbname > /dev/null
psql -c "create extension if not exists pg_prewarm;" $dbname > /dev/null
psql -c "drop table if exists t1,t2,t3;" $dbname > /dev/null
psql -c "create table t1 (c1 bigint not null, c2 bigint not null, c3 bigint not null, c4 bigint not null, c5 bigint not null, c6 bigint not null, c7 bigint not null, c8 bigint not null, c9 bigint not null, c10 bigint not null, c11 bigint not null, c12 bigint not null, c13 bigint not null, c14 bigint not null, c15 bigint not null, c16 bigint not null);" $dbname > /dev/null
psql -c "create table t2 (c1 bigint, c2 bigint not null, c3 bigint not null, c4 bigint not null, c5 bigint not null, c6 bigint not null, c7 bigint not null, c8 bigint not null, c9 bigint not null, c10 bigint not null, c11 bigint not null, c12 bigint not null, c13 bigint not null, c14 bigint not null, c15 bigint not null, c16 bigint not null);" $dbname > /dev/null
psql -c "create table t3 (c1 text not null, c2 bigint not null, c3 bigint not null, c4 bigint not null, c5 bigint not null, c6 bigint not null, c7 bigint not null, c8 bigint not null, c9 bigint not null, c10 bigint not null, c11 bigint not null, c12 bigint not null, c13 bigint not null, c14 bigint not null, c15 bigint not null, c16 bigint not null);" $dbname > /dev/null
psql -c "insert into t1 select 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1 from generate_Series(1,$rows);" $dbname > /dev/null
psql -c "insert into t2 select null,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1 from generate_Series(1,$rows);" $dbname > /dev/null
psql -c "insert into t3 select '1',1,1,1,1,1,1,1,1,1,1,1,1,1,1,1 from generate_Series(1,$rows);" $dbname > /dev/null
for tbl in "t1" "t2" "t3"
do
echo "Table $tbl"
psql -c "vacuum freeze analyze $tbl;" $dbname > /dev/null
psql -c "select pg_prewarm('$tbl');" $dbname > /dev/null
echo "select count(c16) from $tbl;" > bench.sql
for i in {1..3}
do
pgbench -n -f bench.sql -M prepared -T $secs $dbname | grep tps
done
done
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 1f36d675a1..493e1a28b3 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -989,6 +989,118 @@ tts_buffer_heap_store_tuple(TupleTableSlot *slot, HeapTuple tuple,
}
}
+/*
+ * slot_deform_heap_tuple_internal
+ * An always inline helper function for use in slot_deform_heap_tuple to
+ * allow the compiler to emit specialized versions of this function for
+ * various combinations of "slow" and "hasnulls". For example, if a
+ * given tuple has no nulls, then we needn't check "hasnulls" for every
+ * attribute that we're deforming. The caller can just call this
+ * function with hasnulls set to constant-false and have the compiler
+ * remove the constant-false branches and emit more optimal code.
+ *
+ * Returns the next attnum to deform, which can be equal to natts when the
+ * function managed to deform all requested attributes. *offp is an input and
+ * output parameter which is the byte offset within the tuple to start deforming
+ * from which, on return, gets set to the offset where the next attribute
+ * should be deformed from. *slowp is set to true when subsequent deforming
+ * of this tuple must use a version of this function with "slow" passed as
+ * true.
+ *
+ * Callers cannot assume when we return "attnum" (i.e. all requested
+ * attributes have been deformed) that slow mode isn't required for any
+ * additional deforming as the final attribute may have caused a switch to
+ * slow mode.
+ */
+static pg_attribute_always_inline int
+slot_deform_heap_tuple_internal(TupleTableSlot *slot, HeapTuple tuple,
+ int attnum, int natts, bool slow,
+ bool hasnulls, uint32 *offp, bool *slowp)
+{
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+ Datum *values = slot->tts_values;
+ bool *isnull = slot->tts_isnull;
+ HeapTupleHeader tup = tuple->t_data;
+ char *tp; /* ptr to tuple data */
+ bits8 *bp = tup->t_bits; /* ptr to null bitmap in tuple */
+ bool slownext = false;
+
+ tp = (char *) tup + tup->t_hoff;
+
+ for (; attnum < natts; attnum++)
+ {
+ CompactAttribute *thisatt = TupleDescCompactAttr(tupleDesc, attnum);
+
+ if (hasnulls && att_isnull(attnum, bp))
+ {
+ values[attnum] = (Datum) 0;
+ isnull[attnum] = true;
+ if (!slow)
+ {
+ *slowp = true;
+ return attnum + 1;
+ }
+ else
+ continue;
+ }
+
+ isnull[attnum] = false;
+
+ /* calculate the offset of this attribute */
+ if (!slow && thisatt->attcacheoff >= 0)
+ *offp = thisatt->attcacheoff;
+ else if (thisatt->attlen == -1)
+ {
+ /*
+ * We can only cache the offset for a varlena attribute if the
+ * offset is already suitably aligned, so that there would be no
+ * pad bytes in any case: then the offset will be valid for either
+ * an aligned or unaligned value.
+ */
+ if (!slow && *offp == att_nominal_alignby(*offp, thisatt->attalignby))
+ thisatt->attcacheoff = *offp;
+ else
+ {
+ *offp = att_pointer_alignby(*offp,
+ thisatt->attalignby,
+ -1,
+ tp + *offp);
+
+ if (!slow)
+ slownext = true;
+ }
+ }
+ else
+ {
+ /* not varlena, so safe to use att_align_nominal */
+ *offp = att_nominal_alignby(*offp, thisatt->attalignby);
+
+ if (!slow)
+ thisatt->attcacheoff = *offp;
+ }
+
+ values[attnum] = fetchatt(thisatt, tp + *offp);
+
+ *offp = att_addlength_pointer(*offp, thisatt->attlen, tp + *offp);
+
+ /* check if we need to switch to slow mode */
+ if (!slow)
+ {
+ /*
+ * We're unable to deform any further if the above code set
+ * 'slownext', or if this isn't a fixed-width attribute.
+ */
+ if (slownext || thisatt->attlen <= 0)
+ {
+ *slowp = true;
+ return attnum + 1;
+ }
+ }
+ }
+
+ return natts;
+}
+
/*
* slot_deform_heap_tuple
* Given a TupleTableSlot, extract data from the slot's physical tuple
@@ -1007,16 +1119,10 @@ static pg_attribute_always_inline void
slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int natts)
{
- TupleDesc tupleDesc = slot->tts_tupleDescriptor;
- Datum *values = slot->tts_values;
- bool *isnull = slot->tts_isnull;
- HeapTupleHeader tup = tuple->t_data;
- bool hasnulls = HeapTupleHasNulls(tuple);
- int attnum;
- char *tp; /* ptr to tuple data */
- uint32 off; /* offset in tuple data */
- bits8 *bp = tup->t_bits; /* ptr to null bitmap in tuple */
- bool slow; /* can we use/set attcacheoff? */
+ bool hasnulls = HeapTupleHasNulls(tuple);
+ int attnum;
+ uint32 off; /* offset in tuple data */
+ bool slow; /* can we use/set attcacheoff? */
/* We can only fetch as many attributes as the tuple has. */
natts = Min(HeapTupleHeaderGetNatts(tuple->t_data), natts);
@@ -1039,57 +1145,52 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
slow = TTS_SLOW(slot);
}
- tp = (char *) tup + tup->t_hoff;
-
- for (; attnum < natts; attnum++)
+ /*
+ * If 'slow' isn't set, try deforming using deforming code that does not
+ * contain any of the extra checks required for non-fixed offset
+ * deforming. During deforming, if or when we find a NULL or a variable
+ * length attribute, we'll switch to a deforming method which includes the
+ * extra code required for non-fixed offset deforming, a.k.a slow mode.
+ * Because this is performance critical, we inline
+ * slot_deform_heap_tuple_internal passing the 'slow' and 'hasnull'
+ * parameters as constants to allow the compiler to emit specialized code
+ * with the known-const false comparisons and subsequent branches removed.
+ */
+ if (!slow)
{
- CompactAttribute *thisatt = TupleDescCompactAttr(tupleDesc, attnum);
-
- if (hasnulls && att_isnull(attnum, bp))
- {
- values[attnum] = (Datum) 0;
- isnull[attnum] = true;
- slow = true; /* can't use attcacheoff anymore */
- continue;
- }
-
- isnull[attnum] = false;
-
- if (!slow && thisatt->attcacheoff >= 0)
- off = thisatt->attcacheoff;
- else if (thisatt->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be no
- * pad bytes in any case: then the offset will be valid for either
- * an aligned or unaligned value.
- */
- if (!slow &&
- off == att_nominal_alignby(off, thisatt->attalignby))
- thisatt->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, thisatt->attalignby, -1,
- tp + off);
- slow = true;
- }
- }
+ /* Tuple without any NULLs? We can skip doing any NULL checking */
+ if (!hasnulls)
+ attnum = slot_deform_heap_tuple_internal(slot,
+ tuple,
+ attnum,
+ natts,
+ false, /* slow */
+ false, /* hasnulls */
+ &off,
+ &slow);
else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, thisatt->attalignby);
-
- if (!slow)
- thisatt->attcacheoff = off;
- }
-
- values[attnum] = fetchatt(thisatt, tp + off);
-
- off = att_addlength_pointer(off, thisatt->attlen, tp + off);
+ attnum = slot_deform_heap_tuple_internal(slot,
+ tuple,
+ attnum,
+ natts,
+ false, /* slow */
+ true, /* hasnulls */
+ &off,
+ &slow);
+ }
- if (thisatt->attlen <= 0)
- slow = true; /* can't use attcacheoff anymore */
+ /* If there's still work to do then we must be in slow mode */
+ if (attnum < natts)
+ {
+ /* XXX is it worth adding a separate call when hasnulls is false? */
+ attnum = slot_deform_heap_tuple_internal(slot,
+ tuple,
+ attnum,
+ natts,
+ true, /* slow */
+ hasnulls,
+ &off,
+ &slow);
}
/*
@@ -1103,7 +1204,6 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
slot->tts_flags &= ~TTS_FLAG_SLOW;
}
-
const TupleTableSlotOps TTSOpsVirtual = {
.base_slot_size = sizeof(VirtualTupleTableSlot),
.init = tts_virtual_init,
Attachments:
[text/plain] deform_test2.sh.txt (2.1K, ../../CAApHDvo9e0XG71WrefYaRv5n4xNPLK4k8LjD0mSR3c9KR2vi2Q@mail.gmail.com/2-deform_test2.sh.txt)
download | inline:
#!/bin/bash
dbname=postgres
secs=15
rows=10000
extra_cols_start=16
extra_cols_end=16
psql -c "alter system set max_parallel_workers_per_gather = 0;" $dbname > /dev/null
psql -c "alter system set jit = 0;" $dbname > /dev/null
psql -c "select pg_reload_conf();" $dbname > /dev/null
psql -c "create extension if not exists pg_prewarm;" $dbname > /dev/null
psql -c "drop table if exists t1,t2,t3;" $dbname > /dev/null
psql -c "create table t1 (c1 bigint not null, c2 bigint not null, c3 bigint not null, c4 bigint not null, c5 bigint not null, c6 bigint not null, c7 bigint not null, c8 bigint not null, c9 bigint not null, c10 bigint not null, c11 bigint not null, c12 bigint not null, c13 bigint not null, c14 bigint not null, c15 bigint not null, c16 bigint not null);" $dbname > /dev/null
psql -c "create table t2 (c1 bigint, c2 bigint not null, c3 bigint not null, c4 bigint not null, c5 bigint not null, c6 bigint not null, c7 bigint not null, c8 bigint not null, c9 bigint not null, c10 bigint not null, c11 bigint not null, c12 bigint not null, c13 bigint not null, c14 bigint not null, c15 bigint not null, c16 bigint not null);" $dbname > /dev/null
psql -c "create table t3 (c1 text not null, c2 bigint not null, c3 bigint not null, c4 bigint not null, c5 bigint not null, c6 bigint not null, c7 bigint not null, c8 bigint not null, c9 bigint not null, c10 bigint not null, c11 bigint not null, c12 bigint not null, c13 bigint not null, c14 bigint not null, c15 bigint not null, c16 bigint not null);" $dbname > /dev/null
psql -c "insert into t1 select 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1 from generate_Series(1,$rows);" $dbname > /dev/null
psql -c "insert into t2 select null,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1 from generate_Series(1,$rows);" $dbname > /dev/null
psql -c "insert into t3 select '1',1,1,1,1,1,1,1,1,1,1,1,1,1,1,1 from generate_Series(1,$rows);" $dbname > /dev/null
for tbl in "t1" "t2" "t3"
do
echo "Table $tbl"
psql -c "vacuum freeze analyze $tbl;" $dbname > /dev/null
psql -c "select pg_prewarm('$tbl');" $dbname > /dev/null
echo "select count(c16) from $tbl;" > bench.sql
for i in {1..3}
do
pgbench -n -f bench.sql -M prepared -T $secs $dbname | grep tps
done
done
[image/png] deform_specialisations_bench.png (67.4K, ../../CAApHDvo9e0XG71WrefYaRv5n4xNPLK4k8LjD0mSR3c9KR2vi2Q@mail.gmail.com/3-deform_specialisations_bench.png)
download | view image
[application/octet-stream] v1-0001-Speedup-tuple-deformation-with-additional-functio.patch (9.1K, ../../CAApHDvo9e0XG71WrefYaRv5n4xNPLK4k8LjD0mSR3c9KR2vi2Q@mail.gmail.com/4-v1-0001-Speedup-tuple-deformation-with-additional-functio.patch)
download | inline diff:
From 6c4c63a81b7cb7dfd25d84ecfc4fe65cd09390ec Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Wed, 30 Oct 2024 12:36:32 +1300
Subject: [PATCH v1] Speedup tuple deformation with additional function
inlining
---
src/backend/executor/execTuples.c | 207 ++++++++++++++++++++++--------
1 file changed, 153 insertions(+), 54 deletions(-)
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 00dc339615..afc43692be 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -991,54 +991,40 @@ tts_buffer_heap_store_tuple(TupleTableSlot *slot, HeapTuple tuple,
}
/*
- * slot_deform_heap_tuple
- * Given a TupleTableSlot, extract data from the slot's physical tuple
- * into its Datum/isnull arrays. Data is extracted up through the
- * natts'th column (caller must ensure this is a legal column number).
+ * slot_deform_heap_tuple_internal
+ * An always inline helper function for use in slot_deform_heap_tuple to
+ * allow the compiler to emit specialized versions of this function for
+ * various combinations of "slow" and "hasnulls". For example, if a
+ * given tuple has no nulls, then we needn't check "hasnulls" for every
+ * attribute that we're deforming. The caller can just call this
+ * function with hasnulls set to constant-false and have the compiler
+ * remove the constant-false branches and emit more optimal code.
*
- * This is essentially an incremental version of heap_deform_tuple:
- * on each call we extract attributes up to the one needed, without
- * re-computing information about previously extracted attributes.
- * slot->tts_nvalid is the number of attributes already extracted.
+ * Returns the next attnum to deform, which can be equal to natts when the
+ * function managed to deform all requested attributes. *offp is an input and
+ * output parameter which is the byte offset within the tuple to start deforming
+ * from which, on return, gets set to the offset where the next attribute
+ * should be deformed from. *slowp is set to true when subsequent deforming
+ * of this tuple must use a version of this function with "slow" passed as
+ * true.
*
- * This is marked as always inline, so the different offp for different types
- * of slots gets optimized away.
+ * Callers cannot assume when we return "attnum" (i.e. all requested
+ * attributes have been deformed) that slow mode isn't required for any
+ * additional deforming as the final attribute may have caused a switch to
+ * slow mode.
*/
-static pg_attribute_always_inline void
-slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
- int natts)
+static pg_attribute_always_inline int
+slot_deform_heap_tuple_internal(TupleTableSlot *slot, HeapTuple tuple,
+ int attnum, int natts, bool slow,
+ bool hasnulls, uint32 *offp, bool *slowp)
{
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
Datum *values = slot->tts_values;
bool *isnull = slot->tts_isnull;
HeapTupleHeader tup = tuple->t_data;
- bool hasnulls = HeapTupleHasNulls(tuple);
- int attnum;
char *tp; /* ptr to tuple data */
- uint32 off; /* offset in tuple data */
bits8 *bp = tup->t_bits; /* ptr to null bitmap in tuple */
- bool slow; /* can we use/set attcacheoff? */
-
- /* We can only fetch as many attributes as the tuple has. */
- natts = Min(HeapTupleHeaderGetNatts(tuple->t_data), natts);
-
- /*
- * Check whether the first call for this tuple, and initialize or restore
- * loop state.
- */
- attnum = slot->tts_nvalid;
- if (attnum == 0)
- {
- /* Start from the first attribute */
- off = 0;
- slow = false;
- }
- else
- {
- /* Restore state from previous execution */
- off = *offp;
- slow = TTS_SLOW(slot);
- }
+ bool slownext = false;
tp = (char *) tup + tup->t_hoff;
@@ -1050,14 +1036,20 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
{
values[attnum] = (Datum) 0;
isnull[attnum] = true;
- slow = true; /* can't use attcacheoff anymore */
- continue;
+ if (!slow)
+ {
+ *slowp = true;
+ return attnum + 1;
+ }
+ else
+ continue;
}
isnull[attnum] = false;
+ /* calculate the offset of this attribute */
if (!slow && thisatt->attcacheoff >= 0)
- off = thisatt->attcacheoff;
+ *offp = thisatt->attcacheoff;
else if (thisatt->attlen == -1)
{
/*
@@ -1066,31 +1058,139 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
* pad bytes in any case: then the offset will be valid for either
* an aligned or unaligned value.
*/
- if (!slow &&
- off == att_align_nominal(off, thisatt->attalign))
- thisatt->attcacheoff = off;
+ if (!slow && *offp == att_align_nominal(*offp, thisatt->attalign))
+ thisatt->attcacheoff = *offp;
else
{
- off = att_align_pointer(off, thisatt->attalign, -1,
- tp + off);
- slow = true;
+ *offp = att_align_pointer(*offp, thisatt->attalign, -1,
+ tp + *offp);
+
+ if (!slow)
+ slownext = true;
}
}
else
{
/* not varlena, so safe to use att_align_nominal */
- off = att_align_nominal(off, thisatt->attalign);
+ *offp = att_align_nominal(*offp, thisatt->attalign);
if (!slow)
- thisatt->attcacheoff = off;
+ thisatt->attcacheoff = *offp;
}
- values[attnum] = fetchatt(thisatt, tp + off);
- off = att_addlength_pointer(off, thisatt->attlen, tp + off);
+ values[attnum] = fetchatt(thisatt, tp + *offp);
+
+ *offp = att_addlength_pointer(*offp, thisatt->attlen, tp + *offp);
- if (thisatt->attlen <= 0)
- slow = true; /* can't use attcacheoff anymore */
+ /* check if we need to switch to slow mode */
+ if (!slow)
+ {
+ /*
+ * We're unable to deform any further if the above code set
+ * 'slownext', or if this isn't a fixed-width attribute.
+ */
+ if (slownext || thisatt->attlen <= 0)
+ {
+ *slowp = true;
+ return attnum + 1;
+ }
+ }
+ }
+
+ return natts;
+}
+
+/*
+ * slot_deform_heap_tuple
+ * Given a TupleTableSlot, extract data from the slot's physical tuple
+ * into its Datum/isnull arrays. Data is extracted up through the
+ * natts'th column (caller must ensure this is a legal column number).
+ *
+ * This is essentially an incremental version of heap_deform_tuple:
+ * on each call we extract attributes up to the one needed, without
+ * re-computing information about previously extracted attributes.
+ * slot->tts_nvalid is the number of attributes already extracted.
+ *
+ * This is marked as always inline, so the different offp for different types
+ * of slots gets optimized away.
+ */
+static pg_attribute_always_inline void
+slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
+ int natts)
+{
+ bool hasnulls = HeapTupleHasNulls(tuple);
+ int attnum;
+ uint32 off; /* offset in tuple data */
+ bool slow; /* can we use/set attcacheoff? */
+
+ /* We can only fetch as many attributes as the tuple has. */
+ natts = Min(HeapTupleHeaderGetNatts(tuple->t_data), natts);
+
+ /*
+ * Check whether the first call for this tuple, and initialize or restore
+ * loop state.
+ */
+ attnum = slot->tts_nvalid;
+ if (attnum == 0)
+ {
+ /* Start from the first attribute */
+ off = 0;
+ slow = false;
+ }
+ else
+ {
+ /* Restore state from previous execution */
+ off = *offp;
+ slow = TTS_SLOW(slot);
+ }
+
+ /*
+ * If 'slow' isn't set, try deforming using deforming code that does not
+ * contain any of the extra checks required for non-fixed offset
+ * deforming. During deforming, if or when we find a NULL or a variable
+ * length attribute, we'll switch to a deforming method which includes the
+ * extra code required for non-fixed offset deforming, a.k.a slow mode.
+ * Because this is performance critical, we inline
+ * slot_deform_heap_tuple_internal passing the 'slow' and 'hasnull'
+ * parameters as constants to allow the compiler to emit specialized code
+ * with the known-const false comparisons and subsequent branches removed.
+ */
+ if (!slow)
+ {
+ /* Tuple without any NULLs? We can skip doing any NULL checking */
+ if (!hasnulls)
+ attnum = slot_deform_heap_tuple_internal(slot,
+ tuple,
+ attnum,
+ natts,
+ false, /* slow */
+ false, /* hasnulls */
+ &off,
+ &slow);
+ else
+ attnum = slot_deform_heap_tuple_internal(slot,
+ tuple,
+ attnum,
+ natts,
+ false, /* slow */
+ true, /* hasnulls */
+ &off,
+ &slow);
+ }
+
+ /* If there's still work to do then we must be in slow mode */
+ if (attnum < natts)
+ {
+ /* XXX is it worth adding a separate call when hasnulls is false? */
+ attnum = slot_deform_heap_tuple_internal(slot,
+ tuple,
+ attnum,
+ natts,
+ true, /* slow */
+ hasnulls,
+ &off,
+ &slow);
}
/*
@@ -1104,7 +1204,6 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
slot->tts_flags &= ~TTS_FLAG_SLOW;
}
-
const TupleTableSlotOps TTSOpsVirtual = {
.base_slot_size = sizeof(VirtualTupleTableSlot),
.init = tts_virtual_init,
--
2.34.1
[image/png] all_patches_bench.png (70.1K, ../../CAApHDvo9e0XG71WrefYaRv5n4xNPLK4k8LjD0mSR3c9KR2vi2Q@mail.gmail.com/5-all_patches_bench.png)
download | view image
[text/plain] version_based_atop_of_other_patches.patch.txt (8.0K, ../../CAApHDvo9e0XG71WrefYaRv5n4xNPLK4k8LjD0mSR3c9KR2vi2Q@mail.gmail.com/6-version_based_atop_of_other_patches.patch.txt)
download | inline diff:
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 1f36d675a1..493e1a28b3 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -989,6 +989,118 @@ tts_buffer_heap_store_tuple(TupleTableSlot *slot, HeapTuple tuple,
}
}
+/*
+ * slot_deform_heap_tuple_internal
+ * An always inline helper function for use in slot_deform_heap_tuple to
+ * allow the compiler to emit specialized versions of this function for
+ * various combinations of "slow" and "hasnulls". For example, if a
+ * given tuple has no nulls, then we needn't check "hasnulls" for every
+ * attribute that we're deforming. The caller can just call this
+ * function with hasnulls set to constant-false and have the compiler
+ * remove the constant-false branches and emit more optimal code.
+ *
+ * Returns the next attnum to deform, which can be equal to natts when the
+ * function managed to deform all requested attributes. *offp is an input and
+ * output parameter which is the byte offset within the tuple to start deforming
+ * from which, on return, gets set to the offset where the next attribute
+ * should be deformed from. *slowp is set to true when subsequent deforming
+ * of this tuple must use a version of this function with "slow" passed as
+ * true.
+ *
+ * Callers cannot assume when we return "attnum" (i.e. all requested
+ * attributes have been deformed) that slow mode isn't required for any
+ * additional deforming as the final attribute may have caused a switch to
+ * slow mode.
+ */
+static pg_attribute_always_inline int
+slot_deform_heap_tuple_internal(TupleTableSlot *slot, HeapTuple tuple,
+ int attnum, int natts, bool slow,
+ bool hasnulls, uint32 *offp, bool *slowp)
+{
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+ Datum *values = slot->tts_values;
+ bool *isnull = slot->tts_isnull;
+ HeapTupleHeader tup = tuple->t_data;
+ char *tp; /* ptr to tuple data */
+ bits8 *bp = tup->t_bits; /* ptr to null bitmap in tuple */
+ bool slownext = false;
+
+ tp = (char *) tup + tup->t_hoff;
+
+ for (; attnum < natts; attnum++)
+ {
+ CompactAttribute *thisatt = TupleDescCompactAttr(tupleDesc, attnum);
+
+ if (hasnulls && att_isnull(attnum, bp))
+ {
+ values[attnum] = (Datum) 0;
+ isnull[attnum] = true;
+ if (!slow)
+ {
+ *slowp = true;
+ return attnum + 1;
+ }
+ else
+ continue;
+ }
+
+ isnull[attnum] = false;
+
+ /* calculate the offset of this attribute */
+ if (!slow && thisatt->attcacheoff >= 0)
+ *offp = thisatt->attcacheoff;
+ else if (thisatt->attlen == -1)
+ {
+ /*
+ * We can only cache the offset for a varlena attribute if the
+ * offset is already suitably aligned, so that there would be no
+ * pad bytes in any case: then the offset will be valid for either
+ * an aligned or unaligned value.
+ */
+ if (!slow && *offp == att_nominal_alignby(*offp, thisatt->attalignby))
+ thisatt->attcacheoff = *offp;
+ else
+ {
+ *offp = att_pointer_alignby(*offp,
+ thisatt->attalignby,
+ -1,
+ tp + *offp);
+
+ if (!slow)
+ slownext = true;
+ }
+ }
+ else
+ {
+ /* not varlena, so safe to use att_align_nominal */
+ *offp = att_nominal_alignby(*offp, thisatt->attalignby);
+
+ if (!slow)
+ thisatt->attcacheoff = *offp;
+ }
+
+ values[attnum] = fetchatt(thisatt, tp + *offp);
+
+ *offp = att_addlength_pointer(*offp, thisatt->attlen, tp + *offp);
+
+ /* check if we need to switch to slow mode */
+ if (!slow)
+ {
+ /*
+ * We're unable to deform any further if the above code set
+ * 'slownext', or if this isn't a fixed-width attribute.
+ */
+ if (slownext || thisatt->attlen <= 0)
+ {
+ *slowp = true;
+ return attnum + 1;
+ }
+ }
+ }
+
+ return natts;
+}
+
/*
* slot_deform_heap_tuple
* Given a TupleTableSlot, extract data from the slot's physical tuple
@@ -1007,16 +1119,10 @@ static pg_attribute_always_inline void
slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int natts)
{
- TupleDesc tupleDesc = slot->tts_tupleDescriptor;
- Datum *values = slot->tts_values;
- bool *isnull = slot->tts_isnull;
- HeapTupleHeader tup = tuple->t_data;
- bool hasnulls = HeapTupleHasNulls(tuple);
- int attnum;
- char *tp; /* ptr to tuple data */
- uint32 off; /* offset in tuple data */
- bits8 *bp = tup->t_bits; /* ptr to null bitmap in tuple */
- bool slow; /* can we use/set attcacheoff? */
+ bool hasnulls = HeapTupleHasNulls(tuple);
+ int attnum;
+ uint32 off; /* offset in tuple data */
+ bool slow; /* can we use/set attcacheoff? */
/* We can only fetch as many attributes as the tuple has. */
natts = Min(HeapTupleHeaderGetNatts(tuple->t_data), natts);
@@ -1039,57 +1145,52 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
slow = TTS_SLOW(slot);
}
- tp = (char *) tup + tup->t_hoff;
-
- for (; attnum < natts; attnum++)
+ /*
+ * If 'slow' isn't set, try deforming using deforming code that does not
+ * contain any of the extra checks required for non-fixed offset
+ * deforming. During deforming, if or when we find a NULL or a variable
+ * length attribute, we'll switch to a deforming method which includes the
+ * extra code required for non-fixed offset deforming, a.k.a slow mode.
+ * Because this is performance critical, we inline
+ * slot_deform_heap_tuple_internal passing the 'slow' and 'hasnull'
+ * parameters as constants to allow the compiler to emit specialized code
+ * with the known-const false comparisons and subsequent branches removed.
+ */
+ if (!slow)
{
- CompactAttribute *thisatt = TupleDescCompactAttr(tupleDesc, attnum);
-
- if (hasnulls && att_isnull(attnum, bp))
- {
- values[attnum] = (Datum) 0;
- isnull[attnum] = true;
- slow = true; /* can't use attcacheoff anymore */
- continue;
- }
-
- isnull[attnum] = false;
-
- if (!slow && thisatt->attcacheoff >= 0)
- off = thisatt->attcacheoff;
- else if (thisatt->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be no
- * pad bytes in any case: then the offset will be valid for either
- * an aligned or unaligned value.
- */
- if (!slow &&
- off == att_nominal_alignby(off, thisatt->attalignby))
- thisatt->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, thisatt->attalignby, -1,
- tp + off);
- slow = true;
- }
- }
+ /* Tuple without any NULLs? We can skip doing any NULL checking */
+ if (!hasnulls)
+ attnum = slot_deform_heap_tuple_internal(slot,
+ tuple,
+ attnum,
+ natts,
+ false, /* slow */
+ false, /* hasnulls */
+ &off,
+ &slow);
else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, thisatt->attalignby);
-
- if (!slow)
- thisatt->attcacheoff = off;
- }
-
- values[attnum] = fetchatt(thisatt, tp + off);
-
- off = att_addlength_pointer(off, thisatt->attlen, tp + off);
+ attnum = slot_deform_heap_tuple_internal(slot,
+ tuple,
+ attnum,
+ natts,
+ false, /* slow */
+ true, /* hasnulls */
+ &off,
+ &slow);
+ }
- if (thisatt->attlen <= 0)
- slow = true; /* can't use attcacheoff anymore */
+ /* If there's still work to do then we must be in slow mode */
+ if (attnum < natts)
+ {
+ /* XXX is it worth adding a separate call when hasnulls is false? */
+ attnum = slot_deform_heap_tuple_internal(slot,
+ tuple,
+ attnum,
+ natts,
+ true, /* slow */
+ hasnulls,
+ &off,
+ &slow);
}
/*
@@ -1103,7 +1204,6 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
slot->tts_flags &= ~TTS_FLAG_SLOW;
}
-
const TupleTableSlotOps TTSOpsVirtual = {
.base_slot_size = sizeof(VirtualTupleTableSlot),
.init = tts_virtual_init,
^ permalink raw reply [nested|flat] 2+ messages in thread
end of thread, other threads:[~2024-10-30 21:27 UTC | newest]
Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-11-17 15:28 [PATCH 2/3] Allow composite types in bootstrap Justin Pryzby <[email protected]>
2024-10-30 21:27 More tuple deformation speedups David Rowley <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox