public inbox for [email protected]
help / color / mirror / Atom feedRe: More speedups for tuple deformation
30+ messages / 9 participants
[nested] [flat]
* Re: More speedups for tuple deformation
@ 2026-02-03 00:33 Andres Freund <[email protected]>
0 siblings, 1 reply; 30+ messages in thread
From: Andres Freund @ 2026-02-03 00:33 UTC (permalink / raw)
To: David Rowley <[email protected]>; +Cc: Chao Li <[email protected]>; PostgreSQL Developers <[email protected]>
Hi,
On 2026-02-01 00:27:02 +1300, David Rowley wrote:
> On Sat, 31 Jan 2026 at 06:11, Andres Freund <[email protected]> wrote:
> > This is why I like the idea of keeping track of whether we can rely on NOT
> > NULL columns to be present (I think that means we're evaluating expressions
> > other than constraint checks for new rows). It allows the leading NOT NULL
> > fixed-width columns to be decoded without having to wait for a good chunk of
> > the computations above. That's a performance boon even if we later have
> > nullable or varlength columns.
>
> I can look into this. As we both know, we can't apply this
> optimisation in every case as there are places in the code which form
> then deform tuples before NOT NULL constraints are checked.
Right.
> Perhaps the slot can store a flag to mention if the optimisation is valid to
> apply or not. It doesn't look like the flag can be part of the TupleDesc
> since we cache those in relcache.
I wonder if we should do it the other way round - use a special flag (and
perhaps tuple descriptor) iff we are evaluating "unsanitizes" tuples,
i.e. ones where the NOT NULLness might not yet be correct.
> I'm imagining that TupleDescFinalize() calculates another field which could
> be the max cached offset that's got a NOT NULL constraint and isn't
> attmissing. I think this will need another dedicated loop in
> slot_deform_heap_tuple() to loop up to that attribute before doing the
> firstNonCacheOffsetAttr loop.
I was imagining that we'd use the new value to enter the
firstNonCacheOffsetAttr loop without having to depend on
HeapTupleHeaderGetNatts() & HeapTupleHasNulls(). I.e. just use it to avoid the
dependency on having to have completed the memory fetch for the header.
> > > > Have you experimented setting isnull[] in a dedicated loop if there are nulls
> > > > and then in this loop just checking isnull[attnum]? Seems like that could
> > > > perhaps be combined with the work in first_null_attr() and be more efficient
> > > > than doing an att_isnull() separately for each column.
> > >
> > > Yes. I experiment with that quite a bit. I wasn't able to make it any
> > > faster than setting the isnull element in the same loop as the
> > > tts_values element. What I did try was having a dedicated tight loop
> > > like; for (int i = attnum; i < firstNullAttr; i++) isnull[i] = false;,
> > > but the compiler would always try to optimise that into an inlined
> > > memset which would result in poorly performing code in cases with a
> > > small number of columns due to the size and alignment prechecks.
> >
> > Yea, that kind of transformation is pretty annoying and makes little sense
> > here :(.
> >
> > I was thinking of actually computing the value of isnull[] based on the null
> > bitmap (as you also try below).
>
> I've taken the code you posted in [1] to do this. Thanks for that. It
> works very well.
Nice!
> I made it so the tts_isnull array size is rounded up to the next multiple of
> 8.
Right, that's what I assumed we'd need.
> I've attached 3 graphs, which are now looking a bit better. The gcc
> results are not quite as good. There's still a small regression with 0
> extra column test, and overall, the results are not as impressive as
> clang's. I've not yet studied why.
I suspect it's due to gcc thinking it'd be a good idea to vectorize the
loop. I saw that happening on godbolt.
Are your results better if you use
#if defined(__clang__)
#define pg_nounroll _Pragma("clang loop unroll(disable)")
#define pg_novector _Pragma("clang loop vectorize(disable)")
#elif defined(__GNUC__)
#define pg_nounroll _Pragma("GCC unroll 0")
#define pg_novector _Pragma("GCC novector")
#else
#define pg_nounroll
#define pg_novector _Pragma("loop( no_vector )")
#endif
and put "pg_nounroll pg_novector" before the loop in populate_isnull_array()?
That improves both gcc and clang code generation substantially for me, but
with a considerably bigger improvement for gcc.
Compiler Opt Isns
gcc -O2 165
clang -O2 135
gcc -O3 532
clang -O3 135
Preventing vectorization & unrolling:
gcc -O2 26
clang -O2 25
gcc -O3 26
clang -O3 25
It's somewhat scary to see this level of code size increase in a case where
the compiler really has no information to think vectorizing really is
beneficial...
I'd expect it makes sense to combine the loop for first_null_attr() with the
one for populate_isnull_array(). It might also prevent gcc from trying to
vectorize the loop...
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: More speedups for tuple deformation
@ 2026-02-03 05:24 John Naylor <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 1 reply; 30+ messages in thread
From: John Naylor @ 2026-02-03 05:24 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: David Rowley <[email protected]>; Chao Li <[email protected]>; PostgreSQL Developers <[email protected]>
On Tue, Feb 3, 2026 at 7:33 AM Andres Freund <[email protected]> wrote:
> It's somewhat scary to see this level of code size increase in a case where
> the compiler really has no information to think vectorizing really is
> beneficial...
I tried building on gcc 15.2 with -fno-tree-loop-vectorize, and the
server .text segment was only 5kB smaller, but that may understate the
impact on the decisions that get made. Maybe it's better to opt in for
unrolling and vectorization? I think this option also controls whether
to turn loops into libc memset/memmove etc calls.
--
John Naylor
Amazon Web Services
^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: More speedups for tuple deformation
@ 2026-02-03 14:41 Andres Freund <[email protected]>
parent: John Naylor <[email protected]>
0 siblings, 1 reply; 30+ messages in thread
From: Andres Freund @ 2026-02-03 14:41 UTC (permalink / raw)
To: John Naylor <[email protected]>; +Cc: David Rowley <[email protected]>; Chao Li <[email protected]>; PostgreSQL Developers <[email protected]>
Hi,
On 2026-02-03 12:24:39 +0700, John Naylor wrote:
> On Tue, Feb 3, 2026 at 7:33 AM Andres Freund <[email protected]> wrote:
> > It's somewhat scary to see this level of code size increase in a case where
> > the compiler really has no information to think vectorizing really is
> > beneficial...
>
> I tried building on gcc 15.2 with -fno-tree-loop-vectorize, and the
> server .text segment was only 5kB smaller, but that may understate the
> impact on the decisions that get made.
5kB of instructions isn't nothing. But I guess the fact that most of our loops
are too complicated might be protecting us :)
> Maybe it's better to opt in for unrolling and vectorization?
I think I'd go for opting out. There's too many cases where it's going to be
hard to see the optimization potential, e.g. due to inlining etc. I do think
we probably should add an evolved version of those pragma macros I showed
(including perhaps also ones hinting the other way), and use them in the
places we manually analyzed...
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: More speedups for tuple deformation
@ 2026-02-24 02:23 David Rowley <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 3 replies; 30+ messages in thread
From: David Rowley @ 2026-02-24 02:23 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: John Naylor <[email protected]>; Chao Li <[email protected]>; PostgreSQL Developers <[email protected]>
I've attached an updated version of the patch (v9). This includes some
changes to the main patch so that we no longer access the tuple's
natts when we're only fetching <= maximum guaranteed column. I've also
included what John mentioned about using pg_rightmost_one_pos32()
directly rather than trying to reinvent it in a slightly more optimal
way.
The changes in 0004 and 0005 are new. 0004 makes calling
slot_getmissingattrs() the responsibility of the
TupleTableSlotOps.getsomeattrs() function. Doing this allows
getsomeattrs() to be called with the sibling call optimisation in
slot_getsomeattrs_int() and since slot_getsomeattrs_int() is such a
trivial function now, I ended up just modifying slot_getsomeattrs() to
call getsomeattrs() in a way that allows the compiler to apply the
sibling call optimisation. This seems to help reduce some overheads
and makes the 0 extra column tests look better.
I've also modified slot_getmissingattrs() replacing the memsets with a
for loop to zero the tts_values and set the nulls in the tts_isnull
array. Other experimentations showed that doing this in a loop is
faster than memset, so I applied those learnings there too. I've moved
the elog(ERROR) that checks for invalid attnums into there too. That
does mean we'll deform a tuple before raising that error, but I don't
see the issue with that given that it's a "can't happen" error anyway.
Moving it there saves a compare and jump.
0005 reduces the size of CompactAttribute. It shrinks the struct down
to 8 bytes from 16 by using some bitflags for some lesser-used
booleans and by shrinking attcacheoff down to int16. The idea is that
we just don't cache any offsets larger than 2^15. It's likely if we
get a tuple that big that there's a variable-length attribute anyway,
which caching the offset of isn't possible.
I'm not getting great results from benchmarking the 0005 patch. I
verified that gcc does access the array without calculating the
element address from scratch each time and calculates it once, then
increments the pointer by sizeof(CompactAttribute). See the attached
.csv for the results on the 3 machines I tested on.
I've also resequenced the patches to make the deform_bench test module
part of the 0001 patch. This makes it easier to test the performance
of master.
I've not yet made it so the TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS
tts_flag gets set in all places it currently could be set. There are a
few more scan types where it could be set. I understand you mentioned
that you thought the flag should disable the optimisation rather than
enable it, but I've not yet looked to check all the places that it
needs to be disabled.
David
From 54bd00f3c1b577db15fc6c3a94cdce5f61f586d1 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Tue, 27 Jan 2026 15:08:09 +1300
Subject: [PATCH v9 1/5] Introduce deform_bench test module
For benchmaring tuple deformation.
---
src/test/modules/deform_bench/.gitignore | 4 +
src/test/modules/deform_bench/Makefile | 21 ++++
.../deform_bench/deform_bench--1.0.sql | 8 ++
src/test/modules/deform_bench/deform_bench.c | 107 ++++++++++++++++++
.../modules/deform_bench/deform_bench.control | 4 +
src/test/modules/deform_bench/meson.build | 22 ++++
src/test/modules/meson.build | 1 +
7 files changed, 167 insertions(+)
create mode 100644 src/test/modules/deform_bench/.gitignore
create mode 100644 src/test/modules/deform_bench/Makefile
create mode 100644 src/test/modules/deform_bench/deform_bench--1.0.sql
create mode 100644 src/test/modules/deform_bench/deform_bench.c
create mode 100644 src/test/modules/deform_bench/deform_bench.control
create mode 100644 src/test/modules/deform_bench/meson.build
diff --git a/src/test/modules/deform_bench/.gitignore b/src/test/modules/deform_bench/.gitignore
new file mode 100644
index 00000000000..5dcb3ff9723
--- /dev/null
+++ b/src/test/modules/deform_bench/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/src/test/modules/deform_bench/Makefile b/src/test/modules/deform_bench/Makefile
new file mode 100644
index 00000000000..b5fc0f7a583
--- /dev/null
+++ b/src/test/modules/deform_bench/Makefile
@@ -0,0 +1,21 @@
+# src/test/modules/deform_bench/Makefile
+
+MODULE_big = deform_bench
+OBJS = deform_bench.o
+
+EXTENSION = deform_bench
+DATA = deform_bench--1.0.sql
+PGFILEDESC = "deform_bench - tuple deform benchmarking"
+
+REGRESS = deform_bench
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = src/test/modules/deform_bench
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/src/test/modules/deform_bench/deform_bench--1.0.sql b/src/test/modules/deform_bench/deform_bench--1.0.sql
new file mode 100644
index 00000000000..492b71dba3b
--- /dev/null
+++ b/src/test/modules/deform_bench/deform_bench--1.0.sql
@@ -0,0 +1,8 @@
+/* deform_bench--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION deform_bench" to load this file. \quit
+
+CREATE FUNCTION deform_bench(tableoid Oid, attnum int[]) RETURNS FLOAT
+AS 'MODULE_PATHNAME', 'deform_bench'
+LANGUAGE C VOLATILE STRICT;
diff --git a/src/test/modules/deform_bench/deform_bench.c b/src/test/modules/deform_bench/deform_bench.c
new file mode 100644
index 00000000000..7838f639bef
--- /dev/null
+++ b/src/test/modules/deform_bench/deform_bench.c
@@ -0,0 +1,107 @@
+/*-------------------------------------------------------------------------
+ *
+ * deform_bench.c
+ *
+ * for benchmarking tuple deformation routines
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include <time.h>
+#include <sys/time.h>
+
+#include "access/heapam.h"
+#include "access/relscan.h"
+#include "catalog/pg_am_d.h"
+#include "catalog/pg_type_d.h"
+#include "funcapi.h"
+#include "miscadmin.h"
+#include "utils/array.h"
+#include "utils/arrayaccess.h"
+#include "utils/builtins.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(deform_bench);
+
+Datum
+deform_bench(PG_FUNCTION_ARGS)
+{
+ Oid tableoid = PG_GETARG_OID(0);
+ ArrayType *array = PG_GETARG_ARRAYTYPE_P(1);
+ TableScanDesc scan;
+ Relation rel;
+ TupleDesc tupdesc;
+ TupleTableSlot *slot;
+ Datum *elem_datums = NULL;
+ bool *elem_nulls = NULL;
+ int elem_count;
+ int *attnums;
+ clock_t start,
+ end;
+
+ rel = relation_open(tableoid, AccessShareLock);
+
+ if (rel->rd_rel->relam != HEAP_TABLE_AM_OID)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("only heap AM is supported")));
+
+ tupdesc = RelationGetDescr(rel);
+ slot = MakeTupleTableSlot(tupdesc, &TTSOpsBufferHeapTuple);
+ scan = table_beginscan_strat(rel, GetActiveSnapshot(), 0, NULL, true, false);
+
+ /*
+ * The array is used to allow callers to define how many atts to deform.
+ * e.g: '{1,10}'::int[] would deform attnum=1, then in a 2nd pass deform
+ * the remainder up to attnum=10. Passing an element as NULL means all
+ * attnums. This allows simulation of incremental deformation. Generally
+ * if you're passing an array with more than 1 element, then the array
+ * should be in ascending order. Doing something like '{10,1}' would mean
+ * we've already deformed 10 attributes and on the 2nd pass there's
+ * nothing to do since attnum=1 was already deformed in the first pass.
+ *
+ * You'll get an ERROR if you pass a number higher than the number of
+ * attributes in the table.
+ */
+ deconstruct_array(array,
+ INT4OID,
+ sizeof(int32),
+ true,
+ 'i',
+ &elem_datums,
+ &elem_nulls,
+ &elem_count);
+
+ attnums = palloc_array(int, elem_count);
+
+ for (int i = 0; i < elem_count; i++)
+ {
+ /* Make a NULL element mean all attributes */
+ if (elem_nulls[i])
+ attnums[i] = tupdesc->natts;
+ else
+ attnums[i] = DatumGetInt32(elem_datums[i]);
+ }
+
+ start = clock();
+
+ while (heap_getnextslot(scan, ForwardScanDirection, slot))
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ /* Deform in stages according to the attnums array */
+ for (int i = 0; i < elem_count; i++)
+ slot_getsomeattrs(slot, attnums[i]);
+ }
+
+ end = clock();
+
+ ExecDropSingleTupleTableSlot(slot);
+ table_endscan(scan);
+ relation_close(rel, AccessShareLock);
+
+
+ /* Returns the number of milliseconds to run the test */
+ PG_RETURN_FLOAT8((double) (end - start) / (CLOCKS_PER_SEC / 1000));
+}
diff --git a/src/test/modules/deform_bench/deform_bench.control b/src/test/modules/deform_bench/deform_bench.control
new file mode 100644
index 00000000000..a2023f9d738
--- /dev/null
+++ b/src/test/modules/deform_bench/deform_bench.control
@@ -0,0 +1,4 @@
+# deform_bench extension
+comment = 'functions for benchmarking tuple deformation'
+default_version = '1.0'
+module_pathname = '$libdir/deform_bench'
diff --git a/src/test/modules/deform_bench/meson.build b/src/test/modules/deform_bench/meson.build
new file mode 100644
index 00000000000..82049585244
--- /dev/null
+++ b/src/test/modules/deform_bench/meson.build
@@ -0,0 +1,22 @@
+# Copyright (c) 2026, PostgreSQL Global Development Group
+
+deform_bench_sources = files(
+ 'deform_bench.c',
+)
+
+if host_system == 'windows'
+ deform_bench_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+ '--NAME', 'deform_bench',
+ '--FILEDESC', 'deform_bench - benchmarking tuple deformation',])
+endif
+
+deform_bench = shared_module('deform_bench',
+ deform_bench_sources,
+ kwargs: pg_test_mod_args,
+)
+test_install_libs += deform_bench
+
+test_install_data += files(
+ 'deform_bench--1.0.sql',
+ 'deform_bench.control',
+)
diff --git a/src/test/modules/meson.build b/src/test/modules/meson.build
index 2634a519935..ef2b0af4581 100644
--- a/src/test/modules/meson.build
+++ b/src/test/modules/meson.build
@@ -2,6 +2,7 @@
subdir('brin')
subdir('commit_ts')
+subdir('deform_bench')
subdir('delay_execution')
subdir('dummy_index_am')
subdir('dummy_seclabel')
--
2.51.0
From 75a47b93e6cdba24109eb5b6a1eda57bf2e98002 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Wed, 21 Jan 2026 15:41:37 +1300
Subject: [PATCH v9 2/5] Add empty TupleDescFinalize() function
Currently does nothing, but will in a future commit.
---
contrib/dblink/dblink.c | 4 ++++
contrib/pg_buffercache/pg_buffercache_pages.c | 2 ++
contrib/pg_visibility/pg_visibility.c | 2 ++
src/backend/access/brin/brin_tuple.c | 1 +
src/backend/access/common/tupdesc.c | 13 +++++++++++++
src/backend/access/gin/ginutil.c | 1 +
src/backend/access/gist/gistscan.c | 1 +
src/backend/access/spgist/spgutils.c | 1 +
src/backend/access/transam/twophase.c | 1 +
src/backend/access/transam/xlogfuncs.c | 1 +
src/backend/backup/basebackup_copy.c | 3 +++
src/backend/catalog/index.c | 2 ++
src/backend/catalog/pg_publication.c | 1 +
src/backend/catalog/toasting.c | 6 ++++++
src/backend/commands/explain.c | 1 +
src/backend/commands/functioncmds.c | 1 +
src/backend/commands/sequence.c | 1 +
src/backend/commands/tablecmds.c | 4 ++++
src/backend/commands/wait.c | 1 +
src/backend/executor/execSRF.c | 2 ++
src/backend/executor/execTuples.c | 4 ++++
src/backend/executor/nodeFunctionscan.c | 2 ++
src/backend/parser/parse_relation.c | 4 +++-
src/backend/parser/parse_target.c | 2 ++
.../replication/libpqwalreceiver/libpqwalreceiver.c | 1 +
src/backend/replication/walsender.c | 5 +++++
src/backend/utils/adt/acl.c | 1 +
src/backend/utils/adt/genfile.c | 1 +
src/backend/utils/adt/lockfuncs.c | 1 +
src/backend/utils/adt/orderedsetaggs.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 5 +++++
src/backend/utils/adt/tsvector_op.c | 1 +
src/backend/utils/cache/relcache.c | 8 ++++++++
src/backend/utils/fmgr/funcapi.c | 6 ++++++
src/backend/utils/misc/guc_funcs.c | 5 +++++
src/include/access/tupdesc.h | 1 +
src/pl/plpgsql/src/pl_comp.c | 2 ++
.../test_custom_stats/test_custom_fixed_stats.c | 1 +
src/test/modules/test_predtest/test_predtest.c | 1 +
39 files changed, 100 insertions(+), 1 deletion(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 2498d80c8e7..4038950a6ef 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -881,6 +881,7 @@ materializeResult(FunctionCallInfo fcinfo, PGconn *conn, PGresult *res)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "status",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
ntuples = 1;
nfields = 1;
}
@@ -1044,6 +1045,7 @@ materializeQueryResult(FunctionCallInfo fcinfo,
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "status",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
attinmeta = TupleDescGetAttInMetadata(tupdesc);
oldcontext = MemoryContextSwitchTo(rsinfo->econtext->ecxt_per_query_memory);
@@ -1529,6 +1531,8 @@ dblink_get_pkey(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 2, "colname",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+
/*
* Generate attribute metadata needed later to produce tuples from raw
* C strings
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 89b86855243..a6b4fb5252b 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -174,6 +174,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
INT4OID, -1, 0);
+ TupleDescFinalize(tupledesc);
fctx->tupdesc = BlessTupleDesc(tupledesc);
/* Allocate NBuffers worth of BufferCachePagesRec records. */
@@ -442,6 +443,7 @@ pg_buffercache_os_pages_internal(FunctionCallInfo fcinfo, bool include_numa)
TupleDescInitEntry(tupledesc, (AttrNumber) 3, "numa_node",
INT4OID, -1, 0);
+ TupleDescFinalize(tupledesc);
fctx->tupdesc = BlessTupleDesc(tupledesc);
fctx->include_numa = include_numa;
diff --git a/contrib/pg_visibility/pg_visibility.c b/contrib/pg_visibility/pg_visibility.c
index 9bc3a784bf7..dfab0b64cf5 100644
--- a/contrib/pg_visibility/pg_visibility.c
+++ b/contrib/pg_visibility/pg_visibility.c
@@ -469,6 +469,8 @@ pg_visibility_tupdesc(bool include_blkno, bool include_pd)
TupleDescInitEntry(tupdesc, ++a, "pd_all_visible", BOOLOID, -1, 0);
Assert(a == maxattr);
+ TupleDescFinalize(tupdesc);
+
return BlessTupleDesc(tupdesc);
}
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 69c233c62eb..742ac089a28 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -84,6 +84,7 @@ brtuple_disk_tupdesc(BrinDesc *brdesc)
MemoryContextSwitchTo(oldcxt);
+ TupleDescFinalize(tupdesc);
brdesc->bd_disktdesc = tupdesc;
}
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index b69d10f0a45..2137385a833 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -221,6 +221,9 @@ CreateTupleDesc(int natts, Form_pg_attribute *attrs)
memcpy(TupleDescAttr(desc, i), attrs[i], ATTRIBUTE_FIXED_PART_SIZE);
populate_compact_attribute(desc, i);
}
+
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -265,6 +268,8 @@ CreateTupleDescCopy(TupleDesc tupdesc)
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -311,6 +316,8 @@ CreateTupleDescTruncatedCopy(TupleDesc tupdesc, int natts)
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -396,6 +403,8 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -438,6 +447,8 @@ TupleDescCopy(TupleDesc dst, TupleDesc src)
* source's refcount would be wrong in any case.)
*/
dst->tdrefcount = -1;
+
+ TupleDescFinalize(dst);
}
/*
@@ -1065,6 +1076,8 @@ BuildDescFromLists(const List *names, const List *types, const List *typmods, co
TupleDescInitEntryCollation(desc, attnum, attcollation);
}
+ TupleDescFinalize(desc);
+
return desc;
}
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index ff927279cc3..fe7b984ff32 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -129,6 +129,7 @@ initGinState(GinState *state, Relation index)
attr->attndims);
TupleDescInitEntryCollation(state->tupdesc[i], (AttrNumber) 2,
attr->attcollation);
+ TupleDescFinalize(state->tupdesc[i]);
}
/*
diff --git a/src/backend/access/gist/gistscan.c b/src/backend/access/gist/gistscan.c
index f23bc4a6757..c65f93abdae 100644
--- a/src/backend/access/gist/gistscan.c
+++ b/src/backend/access/gist/gistscan.c
@@ -201,6 +201,7 @@ gistrescan(IndexScanDesc scan, ScanKey key, int nkeys,
attno - 1)->atttypid,
-1, 0);
}
+ TupleDescFinalize(so->giststate->fetchTupdesc);
scan->xs_hitupdesc = so->giststate->fetchTupdesc;
/* Also create a memory context that will hold the returned tuples */
diff --git a/src/backend/access/spgist/spgutils.c b/src/backend/access/spgist/spgutils.c
index 9f5379b87ac..b246e8127db 100644
--- a/src/backend/access/spgist/spgutils.c
+++ b/src/backend/access/spgist/spgutils.c
@@ -340,6 +340,7 @@ getSpGistTupleDesc(Relation index, SpGistTypeDesc *keyType)
TupleDescCompactAttr(outTupDesc, i)->attcacheoff = -1;
populate_compact_attribute(outTupDesc, spgKeyColumn);
+ TupleDescFinalize(outTupDesc);
}
return outTupDesc;
}
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index e4340b59640..7f4ed02a6b9 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -744,6 +744,7 @@ pg_prepared_xact(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 5, "dbid",
OIDOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
/*
diff --git a/src/backend/access/transam/xlogfuncs.c b/src/backend/access/transam/xlogfuncs.c
index 2efe4105efb..b6bc616c74c 100644
--- a/src/backend/access/transam/xlogfuncs.c
+++ b/src/backend/access/transam/xlogfuncs.c
@@ -400,6 +400,7 @@ pg_walfile_name_offset(PG_FUNCTION_ARGS)
TupleDescInitEntry(resultTupleDesc, (AttrNumber) 2, "file_offset",
INT4OID, -1, 0);
+ TupleDescFinalize(resultTupleDesc);
resultTupleDesc = BlessTupleDesc(resultTupleDesc);
/*
diff --git a/src/backend/backup/basebackup_copy.c b/src/backend/backup/basebackup_copy.c
index fecfad9ab7b..29dbd0cb32f 100644
--- a/src/backend/backup/basebackup_copy.c
+++ b/src/backend/backup/basebackup_copy.c
@@ -357,6 +357,8 @@ SendXlogRecPtrResult(XLogRecPtr ptr, TimeLineID tli)
*/
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "tli", INT8OID, -1, 0);
+ TupleDescFinalize(tupdesc);
+
/* send RowDescription */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -388,6 +390,7 @@ SendTablespaceList(List *tablespaces)
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 1, "spcoid", OIDOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "spclocation", TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 3, "size", INT8OID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* send RowDescription */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 43de42ce39e..75e97fb394a 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -481,6 +481,8 @@ ConstructTupleDescriptor(Relation heapRelation,
populate_compact_attribute(indexTupDesc, i);
}
+ TupleDescFinalize(indexTupDesc);
+
return indexTupDesc;
}
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index 9a4791c573e..fa353a0dd37 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -1230,6 +1230,7 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 4, "qual",
PG_NODE_TREEOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
funcctx->user_fctx = table_infos;
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index c78dcea98c1..078a1cf5127 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -229,6 +229,12 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attcompression = InvalidCompressionMethod;
TupleDescAttr(tupdesc, 2)->attcompression = InvalidCompressionMethod;
+ populate_compact_attribute(tupdesc, 0);
+ populate_compact_attribute(tupdesc, 1);
+ populate_compact_attribute(tupdesc, 2);
+
+ TupleDescFinalize(tupdesc);
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 93918a223b8..5f922c3f5c2 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -281,6 +281,7 @@ ExplainResultDesc(ExplainStmt *stmt)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "QUERY PLAN",
result_type, -1, 0);
+ TupleDescFinalize(tupdesc);
return tupdesc;
}
diff --git a/src/backend/commands/functioncmds.c b/src/backend/commands/functioncmds.c
index 242372b1e68..3afd762e9dc 100644
--- a/src/backend/commands/functioncmds.c
+++ b/src/backend/commands/functioncmds.c
@@ -2424,6 +2424,7 @@ CallStmtResultDesc(CallStmt *stmt)
-1,
0);
}
+ TupleDescFinalize(tupdesc);
}
return tupdesc;
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index e1b808bbb60..551667650ba 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -1808,6 +1808,7 @@ pg_get_sequence_data(PG_FUNCTION_ARGS)
BOOLOID, -1, 0);
TupleDescInitEntry(resultTupleDesc, (AttrNumber) 3, "page_lsn",
LSNOID, -1, 0);
+ TupleDescFinalize(resultTupleDesc);
resultTupleDesc = BlessTupleDesc(resultTupleDesc);
seqrel = try_relation_open(relid, AccessShareLock);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index df1ba112b35..5794d421cae 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -1030,6 +1030,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
}
}
+ TupleDescFinalize(descriptor);
+
/*
* For relations with table AM and partitioned tables, select access
* method to use: an explicitly indicated one, or (in the case of a
@@ -1458,6 +1460,8 @@ BuildDescForRelation(const List *columns)
populate_compact_attribute(desc, attnum - 1);
}
+ TupleDescFinalize(desc);
+
return desc;
}
diff --git a/src/backend/commands/wait.c b/src/backend/commands/wait.c
index 1290df10c6f..8e920a72372 100644
--- a/src/backend/commands/wait.c
+++ b/src/backend/commands/wait.c
@@ -338,5 +338,6 @@ WaitStmtResultDesc(WaitStmt *stmt)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "status",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
return tupdesc;
}
diff --git a/src/backend/executor/execSRF.c b/src/backend/executor/execSRF.c
index a0b111dc0e4..b481e50acfb 100644
--- a/src/backend/executor/execSRF.c
+++ b/src/backend/executor/execSRF.c
@@ -272,6 +272,7 @@ ExecMakeTableFunctionResult(SetExprState *setexpr,
funcrettype,
-1,
0);
+ TupleDescFinalize(tupdesc);
rsinfo.setDesc = tupdesc;
}
MemoryContextSwitchTo(oldcontext);
@@ -776,6 +777,7 @@ init_sexpr(Oid foid, Oid input_collation, Expr *node,
funcrettype,
-1,
0);
+ TupleDescFinalize(tupdesc);
sexpr->funcResultDesc = tupdesc;
sexpr->funcReturnsTuple = false;
}
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index b768eae9e53..e6ab51e6404 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -2173,6 +2173,8 @@ ExecTypeFromTLInternal(List *targetList, bool skipjunk)
cur_resno++;
}
+ TupleDescFinalize(typeInfo);
+
return typeInfo;
}
@@ -2207,6 +2209,8 @@ ExecTypeFromExprList(List *exprList)
cur_resno++;
}
+ TupleDescFinalize(typeInfo);
+
return typeInfo;
}
diff --git a/src/backend/executor/nodeFunctionscan.c b/src/backend/executor/nodeFunctionscan.c
index 63e605e1f81..feb82d64967 100644
--- a/src/backend/executor/nodeFunctionscan.c
+++ b/src/backend/executor/nodeFunctionscan.c
@@ -414,6 +414,7 @@ ExecInitFunctionScan(FunctionScan *node, EState *estate, int eflags)
TupleDescInitEntryCollation(tupdesc,
(AttrNumber) 1,
exprCollation(funcexpr));
+ TupleDescFinalize(tupdesc);
}
else
{
@@ -485,6 +486,7 @@ ExecInitFunctionScan(FunctionScan *node, EState *estate, int eflags)
0);
}
+ TupleDescFinalize(scan_tupdesc);
Assert(attno == natts);
}
diff --git a/src/backend/parser/parse_relation.c b/src/backend/parser/parse_relation.c
index e003db520de..9c415e166ee 100644
--- a/src/backend/parser/parse_relation.c
+++ b/src/backend/parser/parse_relation.c
@@ -1883,6 +1883,7 @@ addRangeTableEntryForFunction(ParseState *pstate,
TupleDescInitEntryCollation(tupdesc,
(AttrNumber) 1,
exprCollation(funcexpr));
+ TupleDescFinalize(tupdesc);
}
else if (functypclass == TYPEFUNC_RECORD)
{
@@ -1940,6 +1941,7 @@ addRangeTableEntryForFunction(ParseState *pstate,
i++;
}
+ TupleDescFinalize(tupdesc);
/*
* Ensure that the coldeflist defines a legal set of names (no
@@ -2008,7 +2010,7 @@ addRangeTableEntryForFunction(ParseState *pstate,
0);
/* no need to set collation */
}
-
+ TupleDescFinalize(tupdesc);
Assert(natts == totalatts);
}
else
diff --git a/src/backend/parser/parse_target.c b/src/backend/parser/parse_target.c
index dbf5b2b5c01..a03d82c0540 100644
--- a/src/backend/parser/parse_target.c
+++ b/src/backend/parser/parse_target.c
@@ -1572,6 +1572,8 @@ expandRecordVariable(ParseState *pstate, Var *var, int levelsup)
}
Assert(lname == NULL && lvar == NULL); /* lists same length? */
+ TupleDescFinalize(tupleDesc);
+
return tupleDesc;
}
diff --git a/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c b/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
index 7c8639b32e9..9f04c9ed25d 100644
--- a/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
+++ b/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
@@ -1073,6 +1073,7 @@ libpqrcv_processTuples(PGresult *pgres, WalRcvExecResult *walres,
for (coln = 0; coln < nRetTypes; coln++)
TupleDescInitEntry(walres->tupledesc, (AttrNumber) coln + 1,
PQfname(pgres, coln), retTypes[coln], -1, 0);
+ TupleDescFinalize(walres->tupledesc);
attinmeta = TupleDescGetAttInMetadata(walres->tupledesc);
/* No point in doing more here if there were no tuples returned. */
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c
index 2cde8ebc729..33a9e8d7f21 100644
--- a/src/backend/replication/walsender.c
+++ b/src/backend/replication/walsender.c
@@ -451,6 +451,7 @@ IdentifySystem(void)
TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 4, "dbname",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -496,6 +497,7 @@ ReadReplicationSlot(ReadReplicationSlotCmd *cmd)
/* TimeLineID is unsigned, so int4 is not wide enough. */
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 3, "restart_tli",
INT8OID, -1, 0);
+ TupleDescFinalize(tupdesc);
memset(nulls, true, READ_REPLICATION_SLOT_COLS * sizeof(bool));
@@ -598,6 +600,7 @@ SendTimeLineHistory(TimeLineHistoryCmd *cmd)
tupdesc = CreateTemplateTupleDesc(2);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 1, "filename", TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "content", TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
TLHistoryFileName(histfname, cmd->timeline);
TLHistoryFilePath(path, cmd->timeline);
@@ -1015,6 +1018,7 @@ StartReplication(StartReplicationCmd *cmd)
INT8OID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "next_tli_startpos",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuple */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -1369,6 +1373,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd)
TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 4, "output_plugin",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
diff --git a/src/backend/utils/adt/acl.c b/src/backend/utils/adt/acl.c
index 641673f0b0e..ce07f2bc046 100644
--- a/src/backend/utils/adt/acl.c
+++ b/src/backend/utils/adt/acl.c
@@ -1819,6 +1819,7 @@ aclexplode(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 4, "is_grantable",
BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
/* allocate memory for user context */
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index c083608b1d5..bfb949401d0 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -454,6 +454,7 @@ pg_stat_file(PG_FUNCTION_ARGS)
"creation", TIMESTAMPTZOID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 6,
"isdir", BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
memset(isnull, false, sizeof(isnull));
diff --git a/src/backend/utils/adt/lockfuncs.c b/src/backend/utils/adt/lockfuncs.c
index 9dadd6da672..4481c354fd6 100644
--- a/src/backend/utils/adt/lockfuncs.c
+++ b/src/backend/utils/adt/lockfuncs.c
@@ -146,6 +146,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 16, "waitstart",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
/*
diff --git a/src/backend/utils/adt/orderedsetaggs.c b/src/backend/utils/adt/orderedsetaggs.c
index 3b6da8e36ac..fd8b8676470 100644
--- a/src/backend/utils/adt/orderedsetaggs.c
+++ b/src/backend/utils/adt/orderedsetaggs.c
@@ -233,6 +233,7 @@ ordered_set_startup(FunctionCallInfo fcinfo, bool use_tuples)
-1,
0);
+ TupleDescFinalize(newdesc);
FreeTupleDesc(qstate->tupdesc);
qstate->tupdesc = newdesc;
}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index b1df96e7b0b..0b10da3b180 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -769,6 +769,7 @@ pg_stat_get_backend_subxact(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 2, "subxact_overflow",
BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
if ((local_beentry = pgstat_get_local_beentry_by_proc_number(procNumber)) != NULL)
@@ -1670,6 +1671,7 @@ pg_stat_wal_build_tuple(PgStat_WalCounters wal_counters,
TupleDescInitEntry(tupdesc, (AttrNumber) 6, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
/* Fill values and NULLs */
@@ -2097,6 +2099,7 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 7, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
/* Get statistics about the archiver process */
@@ -2178,6 +2181,7 @@ pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
TIMESTAMPTZOID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 13, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
namestrcpy(&slotname, text_to_cstring(slotname_text));
@@ -2265,6 +2269,7 @@ pg_stat_get_subscription_stats(PG_FUNCTION_ARGS)
INT8OID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 13, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
if (!subentry)
diff --git a/src/backend/utils/adt/tsvector_op.c b/src/backend/utils/adt/tsvector_op.c
index 71c7c7d3b3c..d8dece42b9b 100644
--- a/src/backend/utils/adt/tsvector_op.c
+++ b/src/backend/utils/adt/tsvector_op.c
@@ -651,6 +651,7 @@ tsvector_unnest(PG_FUNCTION_ARGS)
TEXTARRAYOID, -1, 0);
if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
elog(ERROR, "return type must be a row type");
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = tupdesc;
funcctx->user_fctx = PG_GETARG_TSVECTOR_COPY(0);
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 6b634c9fff1..770edb34e08 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -729,6 +729,8 @@ RelationBuildTupleDesc(Relation relation)
pfree(constr);
relation->rd_att->constr = NULL;
}
+
+ TupleDescFinalize(relation->rd_att);
}
/*
@@ -1985,6 +1987,7 @@ formrdesc(const char *relationName, Oid relationReltype,
/* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
TupleDescCompactAttr(relation->rd_att, 0)->attcacheoff = 0;
+ TupleDescFinalize(relation->rd_att);
/* mark not-null status */
if (has_not_null)
@@ -3688,6 +3691,8 @@ RelationBuildLocalRelation(const char *relname,
for (i = 0; i < natts; i++)
TupleDescAttr(rel->rd_att, i)->attrelid = relid;
+ TupleDescFinalize(rel->rd_att);
+
rel->rd_rel->reltablespace = reltablespace;
if (mapped_relation)
@@ -4443,6 +4448,7 @@ BuildHardcodedDescriptor(int natts, const FormData_pg_attribute *attrs)
/* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
TupleDescCompactAttr(result, 0)->attcacheoff = 0;
+ TupleDescFinalize(result);
/* Note: we don't bother to set up a TupleConstr entry */
@@ -6268,6 +6274,8 @@ load_relcache_init_file(bool shared)
populate_compact_attribute(rel->rd_att, i);
}
+ TupleDescFinalize(rel->rd_att);
+
/* next read the access method specific field */
if (fread(&len, 1, sizeof(len), fp) != sizeof(len))
goto read_failed;
diff --git a/src/backend/utils/fmgr/funcapi.c b/src/backend/utils/fmgr/funcapi.c
index 8a934ea8dca..516d02cfb82 100644
--- a/src/backend/utils/fmgr/funcapi.c
+++ b/src/backend/utils/fmgr/funcapi.c
@@ -340,6 +340,8 @@ get_expr_result_type(Node *expr,
exprCollation(col));
i++;
}
+ TupleDescFinalize(tupdesc);
+
if (resultTypeId)
*resultTypeId = rexpr->row_typeid;
if (resultTupleDesc)
@@ -1044,6 +1046,7 @@ resolve_polymorphic_tupdesc(TupleDesc tupdesc, oidvector *declared_args,
}
}
+ TupleDescFinalize(tupdesc);
return true;
}
@@ -1853,6 +1856,8 @@ build_function_result_tupdesc_d(char prokind,
0);
}
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -1970,6 +1975,7 @@ TypeGetTupleDesc(Oid typeoid, List *colaliases)
typeoid,
-1,
0);
+ TupleDescFinalize(tupdesc);
}
else if (functypclass == TYPEFUNC_RECORD)
{
diff --git a/src/backend/utils/misc/guc_funcs.c b/src/backend/utils/misc/guc_funcs.c
index 8524dd3a981..472cb5393ce 100644
--- a/src/backend/utils/misc/guc_funcs.c
+++ b/src/backend/utils/misc/guc_funcs.c
@@ -444,6 +444,7 @@ GetPGVariableResultDesc(const char *name)
TupleDescInitEntry(tupdesc, (AttrNumber) 1, varname,
TEXTOID, -1, 0);
}
+ TupleDescFinalize(tupdesc);
return tupdesc;
}
@@ -465,6 +466,7 @@ ShowGUCConfigOption(const char *name, DestReceiver *dest)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 1, varname,
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -499,6 +501,7 @@ ShowAllGUCConfig(DestReceiver *dest)
TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 3, "description",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -934,6 +937,8 @@ show_all_settings(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 17, "pending_restart",
BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+
/*
* Generate attribute metadata needed later to produce tuples from raw
* C strings
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index d46cdbf7a3c..595413dbbc5 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -195,6 +195,7 @@ extern TupleDesc CreateTupleDescTruncatedCopy(TupleDesc tupdesc, int natts);
extern TupleDesc CreateTupleDescCopyConstr(TupleDesc tupdesc);
+#define TupleDescFinalize(d) ((void) 0)
#define TupleDescSize(src) \
(offsetof(struct TupleDescData, compact_attrs) + \
(src)->natts * sizeof(CompactAttribute) + \
diff --git a/src/pl/plpgsql/src/pl_comp.c b/src/pl/plpgsql/src/pl_comp.c
index 5ecc7766757..b72c963b3be 100644
--- a/src/pl/plpgsql/src/pl_comp.c
+++ b/src/pl/plpgsql/src/pl_comp.c
@@ -1912,6 +1912,8 @@ build_row_from_vars(PLpgSQL_variable **vars, int numvars)
TupleDescInitEntryCollation(row->rowtupdesc, i + 1, typcoll);
}
+ TupleDescFinalize(row->rowtupdesc);
+
return row;
}
diff --git a/src/test/modules/test_custom_stats/test_custom_fixed_stats.c b/src/test/modules/test_custom_stats/test_custom_fixed_stats.c
index 485e08e5c19..f9e7c717280 100644
--- a/src/test/modules/test_custom_stats/test_custom_fixed_stats.c
+++ b/src/test/modules/test_custom_stats/test_custom_fixed_stats.c
@@ -206,6 +206,7 @@ test_custom_stats_fixed_report(PG_FUNCTION_ARGS)
INT8OID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 2, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
values[0] = Int64GetDatum(stats->numcalls);
diff --git a/src/test/modules/test_predtest/test_predtest.c b/src/test/modules/test_predtest/test_predtest.c
index 679a5de456d..48ca2a4ea70 100644
--- a/src/test/modules/test_predtest/test_predtest.c
+++ b/src/test/modules/test_predtest/test_predtest.c
@@ -230,6 +230,7 @@ test_predtest(PG_FUNCTION_ARGS)
"s_r_holds", BOOLOID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 8,
"w_r_holds", BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
tupdesc = BlessTupleDesc(tupdesc);
values[0] = BoolGetDatum(strong_implied_by);
--
2.51.0
From 9ada415253bf5c36c48145cdda2548f25588d59b Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Tue, 31 Dec 2024 09:19:24 +1300
Subject: [PATCH v9 3/5] Optimize tuple deformation
This commit includes various optimizations to improve the performance of
tuple deformation.
We now precalculate CompactAttribute's attcacheoff, which allows us to
remove the code from the deform routines which was setting the
attcacheoff. Setting the attcacheoff is handled by TupleDescFinalize(),
which must be called before the TupleDesc is used for anything. Having
this TupleDescFinalize() function means we can store the first
attribute in the TupleDesc which does not have an offset cached. That
allows us to add a dedicated deforming loop to deform all attributes up
to the final one with a attcacheoff set, or up to the first NULL
attribute, whichever comes first.
We also record the maximum attribute number which is guaranteed to exist
in the tuple, that is, has a NOT NULL constraint and isn't an
atthasmissing attribute. When deforming only attributes prior to the
guaranteed attnum, we've no need to access the tuple's natt count. As an
additional optimization, we only count fixed-width columns when
calculating the maximum guaranteed column as this eliminates the need to
emit code to fetch byref types in the deformation loop for guaranteed
attributes.
Some locations in the code deform tuples that have yet to go through NOT
NULL constraint validation. We're unable to perform the guaranteed
attribute optimization when that's the case. The optimization is opt-in
via the TupleTableSlot using the TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS
flag.
This commit also adds a more efficient way of populating the isnull
array by using a bit-wise trick which performs multiplication on the
inverse of the tuple's bitmap byte and masking out all but the lower bit
of each of the boolean's byte. This results in much more optimal code
when compared to determining the NULLness via att_isnull(). 8 isnull
elements are processed at once using this method, which means we need to
round the tts_isnull array size up to the next 8 bytes. The palloc code
does this anyway, but the round-up needed to be formalized so as not to
overwrite the sentinel byte in debug builds.
---
src/backend/access/common/heaptuple.c | 362 ++++++++----------
src/backend/access/common/indextuple.c | 371 ++++++++-----------
src/backend/access/common/tupdesc.c | 53 +++
src/backend/access/spgist/spgutils.c | 3 -
src/backend/executor/execTuples.c | 360 +++++++++---------
src/backend/executor/nodeSeqscan.c | 2 +
src/backend/jit/llvm/llvmjit_deform.c | 6 -
src/backend/utils/cache/relcache.c | 12 -
src/include/access/tupdesc.h | 19 +-
src/include/access/tupmacs.h | 191 +++++++++-
src/include/executor/tuptable.h | 16 +-
src/test/modules/deform_bench/deform_bench.c | 1 +
12 files changed, 773 insertions(+), 623 deletions(-)
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 11bec20e82e..94d11bdc4fd 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -498,19 +498,7 @@ heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc)
* nocachegetattr
*
* This only gets called from fastgetattr(), in cases where we
- * can't use a cacheoffset and the value is not null.
- *
- * This caches attribute offsets in the attribute descriptor.
- *
- * An alternative way to speed things up would be to cache offsets
- * with the tuple, but that seems more difficult unless you take
- * the storage hit of actually putting those offsets into the
- * tuple you send to disk. Yuck.
- *
- * This scheme will be slightly slower than that, but should
- * perform well for queries which hit large #'s of tuples. After
- * you cache the offsets once, examining all the other tuples using
- * the same attribute descriptor will go much quicker. -cim 5/4/91
+ * can't use the attcacheoff and the value is not null.
*
* NOTE: if you need to change this code, see also heap_deform_tuple.
* Also see nocache_index_getattr, which is the same code for index
@@ -522,194 +510,123 @@ nocachegetattr(HeapTuple tup,
int attnum,
TupleDesc tupleDesc)
{
+ CompactAttribute *cattr;
HeapTupleHeader td = tup->t_data;
char *tp; /* ptr to data part of tuple */
bits8 *bp = td->t_bits; /* ptr to null bitmap in tuple */
- bool slow = false; /* do we have to walk attrs? */
int off; /* current offset within data */
+ int startAttr;
+ int firstNullAttr;
+ int i;
+ bool hasnulls = HeapTupleHasNulls(tup);
- /* ----------------
- * Three cases:
- *
- * 1: No nulls and no variable-width attributes.
- * 2: Has a null or a var-width AFTER att.
- * 3: Has nulls or var-widths BEFORE att.
- * ----------------
- */
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
attnum--;
- if (!HeapTupleNoNulls(tup))
- {
- /*
- * there's a null somewhere in the tuple
- *
- * check to see if any preceding bits are null...
- */
- int byte = attnum >> 3;
- int finalbit = attnum & 0x07;
-
- /* check for nulls "before" final bit of last byte */
- if ((~bp[byte]) & ((1 << finalbit) - 1))
- slow = true;
- else
- {
- /* check for nulls in any "earlier" bytes */
- int i;
+ /*
+ * To reduce the number of attributes we need to look at, we start at the
+ * highest attribute that we can which has a cached offset. Since the
+ * attcacheoff for an attribute is only valid if there are no NULLs in
+ * prior attribute, we must look for NULLs to determine the start attr.
+ */
+ if (hasnulls)
+ firstNullAttr = first_null_attr(bp, attnum);
+ else
+ firstNullAttr = attnum;
- for (i = 0; i < byte; i++)
- {
- if (bp[i] != 0xFF)
- {
- slow = true;
- break;
- }
- }
- }
+ if (tupleDesc->firstNonCachedOffsetAttr > 0)
+ {
+ startAttr = Min(tupleDesc->firstNonCachedOffsetAttr - 1, firstNullAttr);
+ off = TupleDescCompactAttr(tupleDesc, startAttr)->attcacheoff;
+ }
+ else
+ {
+ startAttr = 0;
+ off = 0;
}
tp = (char *) td + td->t_hoff;
- if (!slow)
+ /*
+ * Calculate 'off' up to the first NULL attr. We use two cheaper loops
+ * when the tuple has no variable-width columns. When variable-width
+ * columns exist we use att_addlength_pointer() to move the offset beyond
+ * the current attribute.
+ */
+ if (!HeapTupleHasVarWidth(tup))
{
- CompactAttribute *att;
-
- /*
- * If we get here, there are no nulls up to and including the target
- * attribute. If we have a cached offset, we can use it.
- */
- att = TupleDescCompactAttr(tupleDesc, attnum);
- if (att->attcacheoff >= 0)
- return fetchatt(att, tp + att->attcacheoff);
-
- /*
- * Otherwise, check for non-fixed-length attrs up to and including
- * target. If there aren't any, it's safe to cheaply initialize the
- * cached offsets for these attrs.
- */
- if (HeapTupleHasVarWidth(tup))
+ for (i = startAttr; i < firstNullAttr; i++)
{
- int j;
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- for (j = 0; j <= attnum; j++)
- {
- if (TupleDescCompactAttr(tupleDesc, j)->attlen <= 0)
- {
- slow = true;
- break;
- }
- }
+ off = att_nominal_alignby(off, cattr->attalignby);
+ off += cattr->attlen;
}
- }
-
- if (!slow)
- {
- int natts = tupleDesc->natts;
- int j = 1;
-
- /*
- * If we get here, we have a tuple with no nulls or var-widths up to
- * and including the target attribute, so we can use the cached offset
- * ... only we don't have it yet, or we'd not have got here. Since
- * it's cheap to compute offsets for fixed-width columns, we take the
- * opportunity to initialize the cached offsets for *all* the leading
- * fixed-width columns, in hope of avoiding future visits to this
- * routine.
- */
- TupleDescCompactAttr(tupleDesc, 0)->attcacheoff = 0;
-
- /* we might have set some offsets in the slow path previously */
- while (j < natts && TupleDescCompactAttr(tupleDesc, j)->attcacheoff > 0)
- j++;
- off = TupleDescCompactAttr(tupleDesc, j - 1)->attcacheoff +
- TupleDescCompactAttr(tupleDesc, j - 1)->attlen;
-
- for (; j < natts; j++)
+ for (; i < attnum; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, j);
-
- if (att->attlen <= 0)
- break;
-
- off = att_nominal_alignby(off, att->attalignby);
+ if (att_isnull(i, bp))
+ continue;
- att->attcacheoff = off;
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- off += att->attlen;
+ off = att_pointer_alignby(off, cattr->attalignby, cattr->attlen,
+ tp + off);
+ off += cattr->attlen;
}
-
- Assert(j > attnum);
-
- off = TupleDescCompactAttr(tupleDesc, attnum)->attcacheoff;
}
else
{
- bool usecache = true;
- int i;
-
- /*
- * Now we know that we have to walk the tuple CAREFULLY. But we still
- * might be able to cache some offsets for next time.
- *
- * Note - This loop is a little tricky. For each non-null attribute,
- * we have to first account for alignment padding before the attr,
- * then advance over the attr based on its length. Nulls have no
- * storage and no alignment padding either. We can use/set
- * attcacheoff until we reach either a null or a var-width attribute.
- */
- off = 0;
- for (i = 0;; i++) /* loop exit is at "break" */
+ for (i = startAttr; i < firstNullAttr; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, i);
+ int attlen;
- if (HeapTupleHasNulls(tup) && att_isnull(i, bp))
- {
- usecache = false;
- continue; /* this cannot be the target att */
- }
+ cattr = TupleDescCompactAttr(tupleDesc, i);
+ attlen = cattr->attlen;
- /* If we know the next offset, we can skip the rest */
- if (usecache && att->attcacheoff >= 0)
- off = att->attcacheoff;
- else if (att->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be
- * no pad bytes in any case: then the offset will be valid for
- * either an aligned or unaligned value.
- */
- if (usecache &&
- off == att_nominal_alignby(off, att->attalignby))
- att->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, att->attalignby, -1,
- tp + off);
- usecache = false;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, att->attalignby);
+ /*
+ * cstrings don't exist in heap tuples. Use pg_assume to instruct
+ * the compiler not to emit the cstring related code in
+ * att_addlength_pointer().
+ */
+ pg_assume(attlen > 0 || attlen == -1);
- if (usecache)
- att->attcacheoff = off;
- }
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ attlen,
+ tp + off);
+ off = att_addlength_pointer(off, attlen, tp + off);
+ }
- if (i == attnum)
- break;
+ for (; i < attnum; i++)
+ {
+ int attlen;
- off = att_addlength_pointer(off, att->attlen, tp + off);
+ if (att_isnull(i, bp))
+ continue;
- if (usecache && att->attlen <= 0)
- usecache = false;
+ cattr = TupleDescCompactAttr(tupleDesc, i);
+ attlen = cattr->attlen;
+
+ /* As above, heaptuples have no cstrings */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ off = att_pointer_alignby(off, cattr->attalignby, attlen,
+ tp + off);
+ off = att_addlength_pointer(off, attlen, tp + off);
}
+
}
- return fetchatt(TupleDescCompactAttr(tupleDesc, attnum), tp + off);
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+
+ return fetchatt(cattr, tp + off);
}
/* ----------------
@@ -1347,6 +1264,7 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,
Datum *values, bool *isnull)
{
HeapTupleHeader tup = tuple->t_data;
+ CompactAttribute *cattr;
bool hasnulls = HeapTupleHasNulls(tuple);
int tdesc_natts = tupleDesc->natts;
int natts; /* number of atts to extract */
@@ -1354,70 +1272,98 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,
char *tp; /* ptr to tuple data */
uint32 off; /* offset in tuple data */
bits8 *bp = tup->t_bits; /* ptr to null bitmap in tuple */
- bool slow = false; /* can we use/set attcacheoff? */
+ int firstNonCacheOffsetAttr;
+ int firstNullAttr;
natts = HeapTupleHeaderGetNatts(tup);
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
+
/*
* In inheritance situations, it is possible that the given tuple actually
* has more fields than the caller is expecting. Don't run off the end of
* the caller's arrays.
*/
natts = Min(natts, tdesc_natts);
+ firstNonCacheOffsetAttr = Min(tupleDesc->firstNonCachedOffsetAttr, natts);
+
+ if (hasnulls)
+ {
+ firstNullAttr = first_null_attr(bp, natts);
+
+ /*
+ * XXX: it'd be nice to use populate_isnull_array() here, but that
+ * requires that the isnull array's size is rounded up to the next 8
+ * elements. Doing that would require adjusting many location that
+ * allocate the array.
+ */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, firstNullAttr);
+ }
+ else
+ firstNullAttr = natts;
tp = (char *) tup + tup->t_hoff;
+ attnum = 0;
- off = 0;
+ if (firstNonCacheOffsetAttr > 0)
+ {
+#ifdef USE_ASSERT_CHECKING
+ /* In Assert enabled builds, verify attcacheoff is correct */
+ int offcheck = 0;
+#endif
+ do
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ off = cattr->attcacheoff;
- for (attnum = 0; attnum < natts; attnum++)
+#ifdef USE_ASSERT_CHECKING
+ offcheck = att_nominal_alignby(offcheck, cattr->attalignby);
+ Assert(offcheck == cattr->attcacheoff);
+ offcheck += cattr->attlen;
+#endif
+
+ values[attnum] = fetch_att_noerr(tp + off,
+ cattr->attbyval,
+ cattr->attlen);
+ } while (++attnum < firstNonCacheOffsetAttr);
+ off += cattr->attlen;
+ }
+ else
+ off = 0;
+
+ for (; attnum < firstNullAttr; attnum++)
{
- CompactAttribute *thisatt = TupleDescCompactAttr(tupleDesc, attnum);
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
+ }
+
+ for (; attnum < natts; attnum++)
+ {
+ Assert(hasnulls);
- if (hasnulls && att_isnull(attnum, bp))
+ if (att_isnull(attnum, bp))
{
values[attnum] = (Datum) 0;
isnull[attnum] = true;
- slow = true; /* can't use attcacheoff anymore */
continue;
}
isnull[attnum] = false;
-
- if (!slow && thisatt->attcacheoff >= 0)
- off = thisatt->attcacheoff;
- else if (thisatt->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be no
- * pad bytes in any case: then the offset will be valid for either
- * an aligned or unaligned value.
- */
- if (!slow &&
- off == att_nominal_alignby(off, thisatt->attalignby))
- thisatt->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, thisatt->attalignby, -1,
- tp + off);
- slow = true;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, thisatt->attalignby);
-
- if (!slow)
- thisatt->attcacheoff = off;
- }
-
- values[attnum] = fetchatt(thisatt, tp + off);
-
- off = att_addlength_pointer(off, thisatt->attlen, tp + off);
-
- if (thisatt->attlen <= 0)
- slow = true; /* can't use attcacheoff anymore */
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
}
/*
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index d6350201e01..92282039671 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -223,18 +223,6 @@ index_form_tuple_context(TupleDesc tupleDescriptor,
*
* This gets called from index_getattr() macro, and only in cases
* where we can't use cacheoffset and the value is not null.
- *
- * This caches attribute offsets in the attribute descriptor.
- *
- * An alternative way to speed things up would be to cache offsets
- * with the tuple, but that seems more difficult unless you take
- * the storage hit of actually putting those offsets into the
- * tuple you send to disk. Yuck.
- *
- * This scheme will be slightly slower than that, but should
- * perform well for queries which hit large #'s of tuples. After
- * you cache the offsets once, examining all the other tuples using
- * the same attribute descriptor will go much quicker. -cim 5/4/91
* ----------------
*/
Datum
@@ -242,205 +230,126 @@ nocache_index_getattr(IndexTuple tup,
int attnum,
TupleDesc tupleDesc)
{
+ CompactAttribute *cattr;
char *tp; /* ptr to data part of tuple */
bits8 *bp = NULL; /* ptr to null bitmap in tuple */
- bool slow = false; /* do we have to walk attrs? */
int data_off; /* tuple data offset */
int off; /* current offset within data */
+ int startAttr;
+ int firstNullAttr;
+ bool hasnulls = IndexTupleHasNulls(tup);
+ int i;
- /* ----------------
- * Three cases:
- *
- * 1: No nulls and no variable-width attributes.
- * 2: Has a null or a var-width AFTER att.
- * 3: Has nulls or var-widths BEFORE att.
- * ----------------
- */
-
- data_off = IndexInfoFindDataOffset(tup->t_info);
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
attnum--;
- if (IndexTupleHasNulls(tup))
- {
- /*
- * there's a null somewhere in the tuple
- *
- * check to see if desired att is null
- */
+ data_off = IndexInfoFindDataOffset(tup->t_info);
+ tp = (char *) tup + data_off;
- /* XXX "knows" t_bits are just after fixed tuple header! */
+ /*
+ * To reduce the number of attributes we need to look at, we start at the
+ * highest attribute that we can which has a cached offset. Since the
+ * attcacheoff for an attribute is only valid if there are no NULLs in
+ * prior attribute, we must look for NULLs to determine the start attr.
+ */
+ if (hasnulls)
+ {
bp = (bits8 *) ((char *) tup + sizeof(IndexTupleData));
-
- /*
- * Now check to see if any preceding bits are null...
- */
- {
- int byte = attnum >> 3;
- int finalbit = attnum & 0x07;
-
- /* check for nulls "before" final bit of last byte */
- if ((~bp[byte]) & ((1 << finalbit) - 1))
- slow = true;
- else
- {
- /* check for nulls in any "earlier" bytes */
- int i;
-
- for (i = 0; i < byte; i++)
- {
- if (bp[i] != 0xFF)
- {
- slow = true;
- break;
- }
- }
- }
- }
+ firstNullAttr = first_null_attr(bp, attnum);
}
+ else
+ firstNullAttr = attnum;
- tp = (char *) tup + data_off;
-
- if (!slow)
+ if (tupleDesc->firstNonCachedOffsetAttr > 0)
{
- CompactAttribute *att;
-
- /*
- * If we get here, there are no nulls up to and including the target
- * attribute. If we have a cached offset, we can use it.
- */
- att = TupleDescCompactAttr(tupleDesc, attnum);
- if (att->attcacheoff >= 0)
- return fetchatt(att, tp + att->attcacheoff);
-
- /*
- * Otherwise, check for non-fixed-length attrs up to and including
- * target. If there aren't any, it's safe to cheaply initialize the
- * cached offsets for these attrs.
- */
- if (IndexTupleHasVarwidths(tup))
- {
- int j;
-
- for (j = 0; j <= attnum; j++)
- {
- if (TupleDescCompactAttr(tupleDesc, j)->attlen <= 0)
- {
- slow = true;
- break;
- }
- }
- }
+ startAttr = Min(tupleDesc->firstNonCachedOffsetAttr - 1, firstNullAttr);
+ off = TupleDescCompactAttr(tupleDesc, startAttr)->attcacheoff;
}
-
- if (!slow)
+ else
{
- int natts = tupleDesc->natts;
- int j = 1;
-
- /*
- * If we get here, we have a tuple with no nulls or var-widths up to
- * and including the target attribute, so we can use the cached offset
- * ... only we don't have it yet, or we'd not have got here. Since
- * it's cheap to compute offsets for fixed-width columns, we take the
- * opportunity to initialize the cached offsets for *all* the leading
- * fixed-width columns, in hope of avoiding future visits to this
- * routine.
- */
- TupleDescCompactAttr(tupleDesc, 0)->attcacheoff = 0;
+ startAttr = 0;
+ off = 0;
+ }
- /* we might have set some offsets in the slow path previously */
- while (j < natts && TupleDescCompactAttr(tupleDesc, j)->attcacheoff > 0)
- j++;
+ /*
+ * Calculate 'off' up to the first NULL attr. We use two cheaper loops
+ * when the tuple has no variable-width columns. When variable-width
+ * columns exist we use att_addlength_pointer() to move the offset beyond
+ * the current attribute.
+ */
+ if (IndexTupleHasVarwidths(tup))
+ {
+ /* Calculate the offset up until the first NULL */
+ for (i = startAttr; i < firstNullAttr; i++)
+ {
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- off = TupleDescCompactAttr(tupleDesc, j - 1)->attcacheoff +
- TupleDescCompactAttr(tupleDesc, j - 1)->attlen;
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+ off = att_addlength_pointer(off, cattr->attlen, tp + off);
+ }
- for (; j < natts; j++)
+ /* Calculate the offset for any remaining columns. */
+ for (; i < attnum; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, j);
+ Assert(hasnulls);
- if (att->attlen <= 0)
- break;
+ if (att_isnull(i, bp))
+ continue;
- off = att_nominal_alignby(off, att->attalignby);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- att->attcacheoff = off;
-
- off += att->attlen;
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+ off = att_addlength_pointer(off, cattr->attlen, tp + off);
}
-
- Assert(j > attnum);
-
- off = TupleDescCompactAttr(tupleDesc, attnum)->attcacheoff;
}
else
{
- bool usecache = true;
- int i;
+ /* Handle tuples with only fixed-width attributes */
- /*
- * Now we know that we have to walk the tuple CAREFULLY. But we still
- * might be able to cache some offsets for next time.
- *
- * Note - This loop is a little tricky. For each non-null attribute,
- * we have to first account for alignment padding before the attr,
- * then advance over the attr based on its length. Nulls have no
- * storage and no alignment padding either. We can use/set
- * attcacheoff until we reach either a null or a var-width attribute.
- */
- off = 0;
- for (i = 0;; i++) /* loop exit is at "break" */
+ /* Calculate the offset up until the first NULL */
+ for (i = startAttr; i < firstNullAttr; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, i);
-
- if (IndexTupleHasNulls(tup) && att_isnull(i, bp))
- {
- usecache = false;
- continue; /* this cannot be the target att */
- }
-
- /* If we know the next offset, we can skip the rest */
- if (usecache && att->attcacheoff >= 0)
- off = att->attcacheoff;
- else if (att->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be
- * no pad bytes in any case: then the offset will be valid for
- * either an aligned or unaligned value.
- */
- if (usecache &&
- off == att_nominal_alignby(off, att->attalignby))
- att->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, att->attalignby, -1,
- tp + off);
- usecache = false;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, att->attalignby);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
+
+ Assert(cattr->attlen > 0);
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+ off += cattr->attlen;
+ }
- if (usecache)
- att->attcacheoff = off;
- }
+ /* Calculate the offset for any remaining columns. */
+ for (; i < attnum; i++)
+ {
+ Assert(hasnulls);
- if (i == attnum)
- break;
+ if (att_isnull(i, bp))
+ continue;
- off = att_addlength_pointer(off, att->attlen, tp + off);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- if (usecache && att->attlen <= 0)
- usecache = false;
+ Assert(cattr->attlen > 0);
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+ off += cattr->attlen;
}
}
- return fetchatt(TupleDescCompactAttr(tupleDesc, attnum), tp + off);
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ off = att_pointer_alignby(off, cattr->attalignby,
+ cattr->attlen, tp + off);
+ return fetchatt(cattr, tp + off);
}
/*
@@ -480,63 +389,87 @@ index_deform_tuple_internal(TupleDesc tupleDescriptor,
Datum *values, bool *isnull,
char *tp, bits8 *bp, int hasnulls)
{
+ CompactAttribute *cattr;
int natts = tupleDescriptor->natts; /* number of atts to extract */
- int attnum;
- int off = 0; /* offset in tuple data */
- bool slow = false; /* can we use/set attcacheoff? */
+ int attnum = 0;
+ uint32 off = 0; /* offset in tuple data */
+ int firstNonCacheOffsetAttr;
+ int firstNullAttr;
/* Assert to protect callers who allocate fixed-size arrays */
Assert(natts <= INDEX_MAX_KEYS);
- for (attnum = 0; attnum < natts; attnum++)
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDescriptor->firstNonCachedOffsetAttr >= 0);
+
+ firstNonCacheOffsetAttr = Min(tupleDescriptor->firstNonCachedOffsetAttr, natts);
+
+ if (hasnulls)
+ {
+ firstNullAttr = first_null_attr(bp, natts);
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, firstNullAttr);
+ }
+ else
+ firstNullAttr = natts;
+
+ if (firstNonCacheOffsetAttr > 0)
+ {
+#ifdef USE_ASSERT_CHECKING
+ /* In Assert enabled builds, verify attcacheoff is correct */
+ off = 0;
+#endif
+
+ do
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDescriptor, attnum);
+
+#ifdef USE_ASSERT_CHECKING
+ off = att_nominal_alignby(off, cattr->attalignby);
+ Assert(off == cattr->attcacheoff);
+ off += cattr->attlen;
+#endif
+
+ values[attnum] = fetch_att_noerr(tp + cattr->attcacheoff, cattr->attbyval,
+ cattr->attlen);
+ } while (++attnum < firstNonCacheOffsetAttr);
+
+ off = cattr->attcacheoff + cattr->attlen;
+ }
+
+ for (; attnum < firstNullAttr; attnum++)
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDescriptor, attnum);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
+ }
+
+ for (; attnum < natts; attnum++)
{
- CompactAttribute *thisatt = TupleDescCompactAttr(tupleDescriptor, attnum);
+ Assert(hasnulls);
- if (hasnulls && att_isnull(attnum, bp))
+ if (att_isnull(attnum, bp))
{
values[attnum] = (Datum) 0;
isnull[attnum] = true;
- slow = true; /* can't use attcacheoff anymore */
continue;
}
isnull[attnum] = false;
-
- if (!slow && thisatt->attcacheoff >= 0)
- off = thisatt->attcacheoff;
- else if (thisatt->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be no
- * pad bytes in any case: then the offset will be valid for either
- * an aligned or unaligned value.
- */
- if (!slow &&
- off == att_nominal_alignby(off, thisatt->attalignby))
- thisatt->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, thisatt->attalignby, -1,
- tp + off);
- slow = true;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, thisatt->attalignby);
-
- if (!slow)
- thisatt->attcacheoff = off;
- }
-
- values[attnum] = fetchatt(thisatt, tp + off);
-
- off = att_addlength_pointer(off, thisatt->attlen, tp + off);
-
- if (thisatt->attlen <= 0)
- slow = true; /* can't use attcacheoff anymore */
+ cattr = TupleDescCompactAttr(tupleDescriptor, attnum);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
}
}
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 2137385a833..028fcd53734 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -197,6 +197,10 @@ CreateTemplateTupleDesc(int natts)
desc->tdtypmod = -1;
desc->tdrefcount = -1; /* assume not reference-counted */
+ /* This will be set to the correct value by TupleDescFinalize() */
+ desc->firstNonCachedOffsetAttr = -1;
+ desc->firstNonGuaranteedAttr = -1;
+
return desc;
}
@@ -457,6 +461,9 @@ TupleDescCopy(TupleDesc dst, TupleDesc src)
* descriptor to another.
*
* !!! Constraints and defaults are not copied !!!
+ *
+ * The caller must take care of calling TupleDescFinalize() on once all
+ * TupleDesc changes have been made.
*/
void
TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
@@ -489,6 +496,52 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
populate_compact_attribute(dst, dstAttno - 1);
}
+/*
+ * TupleDescFinalize
+ * Finalize the given TupleDesc. This must be called after the
+ * attributes arrays have been populated or adjusted by any code.
+ *
+ * Must be called after populate_compact_attribute() and before
+ * BlessTupleDesc().
+ */
+void
+TupleDescFinalize(TupleDesc tupdesc)
+{
+ int firstNonCachedOffsetAttr = 0;
+ int firstNonGuaranteedAttr = tupdesc->natts;
+ int off = 0;
+
+ for (int i = 0; i < tupdesc->natts; i++)
+ {
+ CompactAttribute *cattr = TupleDescCompactAttr(tupdesc, i);
+
+ if (firstNonGuaranteedAttr == tupdesc->natts &&
+ (cattr->attnullability != ATTNULLABLE_VALID || !cattr->attbyval ||
+ cattr->atthasmissing || cattr->attisdropped || cattr->attlen <= 0))
+ firstNonGuaranteedAttr = i;
+
+ if (cattr->attlen <= 0)
+ break;
+
+ off = att_nominal_alignby(off, cattr->attalignby);
+
+ /*
+ * attcacheoff is an int16, so don't try and cache any offsets larger
+ * than will fit in that type.
+ */
+ if (off > PG_INT16_MAX)
+ break;
+
+ cattr->attcacheoff = off;
+
+ off += cattr->attlen;
+ firstNonCachedOffsetAttr = i + 1;
+ }
+
+ tupdesc->firstNonCachedOffsetAttr = firstNonCachedOffsetAttr;
+ tupdesc->firstNonGuaranteedAttr = firstNonGuaranteedAttr;
+}
+
/*
* Free a TupleDesc including all substructure
*/
diff --git a/src/backend/access/spgist/spgutils.c b/src/backend/access/spgist/spgutils.c
index b246e8127db..a4694bd8065 100644
--- a/src/backend/access/spgist/spgutils.c
+++ b/src/backend/access/spgist/spgutils.c
@@ -335,9 +335,6 @@ getSpGistTupleDesc(Relation index, SpGistTypeDesc *keyType)
/* We shouldn't need to bother with making these valid: */
att->attcompression = InvalidCompressionMethod;
att->attcollation = InvalidOid;
- /* In case we changed typlen, we'd better reset following offsets */
- for (int i = spgFirstIncludeColumn; i < outTupDesc->natts; i++)
- TupleDescCompactAttr(outTupDesc, i)->attcacheoff = -1;
populate_compact_attribute(outTupDesc, spgKeyColumn);
TupleDescFinalize(outTupDesc);
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index e6ab51e6404..80faf29b797 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -993,218 +993,242 @@ tts_buffer_heap_store_tuple(TupleTableSlot *slot, HeapTuple tuple,
}
/*
- * slot_deform_heap_tuple_internal
- * An always inline helper function for use in slot_deform_heap_tuple to
- * allow the compiler to emit specialized versions of this function for
- * various combinations of "slow" and "hasnulls". For example, if a
- * given tuple has no nulls, then we needn't check "hasnulls" for every
- * attribute that we're deforming. The caller can just call this
- * function with hasnulls set to constant-false and have the compiler
- * remove the constant-false branches and emit more optimal code.
- *
- * Returns the next attnum to deform, which can be equal to natts when the
- * function manages to deform all requested attributes. *offp is an input and
- * output parameter which is the byte offset within the tuple to start deforming
- * from which, on return, gets set to the offset where the next attribute
- * should be deformed from. *slowp is set to true when subsequent deforming
- * of this tuple must use a version of this function with "slow" passed as
- * true.
- *
- * Callers cannot assume when we return "attnum" (i.e. all requested
- * attributes have been deformed) that slow mode isn't required for any
- * additional deforming as the final attribute may have caused a switch to
- * slow mode.
+ * slot_deform_heap_tuple
+ * Given a TupleTableSlot, extract data from the slot's physical tuple
+ * into its Datum/isnull arrays. Data is extracted up through the
+ * natts'th column (caller must ensure this is a legal column number).
+ *
+ * This is essentially an incremental version of heap_deform_tuple:
+ * on each call we extract attributes up to the one needed, without
+ * re-computing information about previously extracted attributes.
+ * slot->tts_nvalid is the number of attributes already extracted.
+ *
+ * This is marked as always inline, so the different offp for different types
+ * of slots gets optimized away.
*/
-static pg_attribute_always_inline int
-slot_deform_heap_tuple_internal(TupleTableSlot *slot, HeapTuple tuple,
- int attnum, int natts, bool slow,
- bool hasnulls, uint32 *offp, bool *slowp)
+static pg_attribute_always_inline void
+slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
+ int natts)
{
+ CompactAttribute *cattr;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
- Datum *values = slot->tts_values;
- bool *isnull = slot->tts_isnull;
HeapTupleHeader tup = tuple->t_data;
+ int attnum;
+ int firstNonCacheOffsetAttr;
+ int firstNonGuaranteedAttr;
+ int firstNullAttr;
+ Datum *values;
+ bool *isnull;
char *tp; /* ptr to tuple data */
- bits8 *bp = tup->t_bits; /* ptr to null bitmap in tuple */
- bool slownext = false;
+ uint32 off; /* offset in tuple data */
- tp = (char *) tup + tup->t_hoff;
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
- for (; attnum < natts; attnum++)
+ isnull = slot->tts_isnull;
+
+ /*
+ * Some callers may form and deform tuples prior to NOT NULL constraints
+ * being checked. Here we'd like to optimize the case where we only need
+ * to fetch attributes before or up to the point where the attribute is
+ * guaranteed to exist in the tuple. We rely on the slot flag being set
+ * correctly to only enable this optimization when it's valid to do so.
+ * This optimization allows us to save fetching the number of attributes
+ * from the tuple and saves the additional cost of handling non-byval
+ * attrs.
+ */
+ if (TTS_OBEYS_NOT_NULL_CONSTRAINTS(slot))
+ firstNonGuaranteedAttr = Min(natts, tupleDesc->firstNonGuaranteedAttr);
+ else
+ firstNonGuaranteedAttr = 0;
+
+ firstNonCacheOffsetAttr = tupleDesc->firstNonCachedOffsetAttr;
+
+ if (HeapTupleHasNulls(tuple))
{
- CompactAttribute *thisatt = TupleDescCompactAttr(tupleDesc, attnum);
+ int tupnatts = HeapTupleHeaderGetNatts(tup);
- if (hasnulls && att_isnull(attnum, bp))
+ tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits) +
+ BITMAPLEN(tupnatts));
+
+ natts = Min(tupnatts, natts);
+ if (natts > firstNonGuaranteedAttr)
{
- values[attnum] = (Datum) 0;
- isnull[attnum] = true;
- if (!slow)
- {
- *slowp = true;
- return attnum + 1;
- }
- else
- continue;
- }
+ bits8 *bp = tup->t_bits;
- isnull[attnum] = false;
+ /* Find the first NULL attr */
+ firstNullAttr = first_null_attr(bp, natts);
- /* calculate the offset of this attribute */
- if (!slow && thisatt->attcacheoff >= 0)
- *offp = thisatt->attcacheoff;
- else if (thisatt->attlen == -1)
- {
/*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be no
- * pad bytes in any case: then the offset will be valid for either
- * an aligned or unaligned value.
+ * And populate the isnull array for all attributes being fetched
+ * from the tuple.
*/
- if (!slow && *offp == att_nominal_alignby(*offp, thisatt->attalignby))
- thisatt->attcacheoff = *offp;
- else
- {
- *offp = att_pointer_alignby(*offp,
- thisatt->attalignby,
- -1,
- tp + *offp);
+ populate_isnull_array(bp, natts, isnull);
- if (!slow)
- slownext = true;
- }
+ /* We can only use any cached offsets until the first NULL attr */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, firstNullAttr);
}
else
{
- /* not varlena, so safe to use att_nominal_alignby */
- *offp = att_nominal_alignby(*offp, thisatt->attalignby);
-
- if (!slow)
- thisatt->attcacheoff = *offp;
+ /* Otherwise all required columns are guaranteed to exist */
+ firstNullAttr = natts;
}
+ }
+ else
+ {
+ tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits));
+
+ /*
+ * We only need to look at the tuple's natts if we need more than the
+ * guaranteed number of columns
+ */
+ if (natts > firstNonGuaranteedAttr)
+ natts = Min(HeapTupleHeaderGetNatts(tup), natts);
+
+ /* All attrs can be fetched without checking for NULLs */
+ firstNullAttr = natts;
+ }
- values[attnum] = fetchatt(thisatt, tp + *offp);
+ attnum = slot->tts_nvalid;
+ values = slot->tts_values;
+ slot->tts_nvalid = natts;
- *offp = att_addlength_pointer(*offp, thisatt->attlen, tp + *offp);
+ /* Ensure we calculated tp correctly */
+ Assert(tp == (char *) tup + tup->t_hoff);
- /* check if we need to switch to slow mode */
- if (!slow)
+ if (attnum < firstNonGuaranteedAttr)
+ {
+ do
{
+ int attlen;
+
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ attlen = cattr->attlen;
+
+ /* We don't expect any non-byval types */
+ pg_assume(attlen > 0);
+
/*
- * We're unable to deform any further if the above code set
- * 'slownext', or if this isn't a fixed-width attribute.
+ * Technically we could support non-byval fixed-width types, but
+ * not doing so allows us to pass true to fetch_att_noerr() which
+ * eliminates the !attbyval branch.
*/
- if (slownext || thisatt->attlen <= 0)
- {
- *slowp = true;
- return attnum + 1;
- }
- }
- }
+ Assert(cattr->attbyval == true);
- return natts;
-}
-
-/*
- * slot_deform_heap_tuple
- * Given a TupleTableSlot, extract data from the slot's physical tuple
- * into its Datum/isnull arrays. Data is extracted up through the
- * natts'th column (caller must ensure this is a legal column number).
- *
- * This is essentially an incremental version of heap_deform_tuple:
- * on each call we extract attributes up to the one needed, without
- * re-computing information about previously extracted attributes.
- * slot->tts_nvalid is the number of attributes already extracted.
- *
- * This is marked as always inline, so the different offp for different types
- * of slots gets optimized away.
- */
-static pg_attribute_always_inline void
-slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
- int natts)
-{
- bool hasnulls = HeapTupleHasNulls(tuple);
- int attnum;
- uint32 off; /* offset in tuple data */
- bool slow; /* can we use/set attcacheoff? */
+ off = cattr->attcacheoff;
+ values[attnum] = fetch_att_noerr(tp + off, true, attlen);
+ attnum++;
+ } while (attnum < firstNonGuaranteedAttr);
- /* We can only fetch as many attributes as the tuple has. */
- natts = Min(HeapTupleHeaderGetNatts(tuple->t_data), natts);
+ off += cattr->attlen;
- /*
- * Check whether the first call for this tuple, and initialize or restore
- * loop state.
- */
- attnum = slot->tts_nvalid;
- if (attnum == 0)
- {
- /* Start from the first attribute */
- off = 0;
- slow = false;
+ if (attnum == natts)
+ goto done;
}
else
{
/* Restore state from previous execution */
off = *offp;
- slow = TTS_SLOW(slot);
+
+ /* We expect *offp to be set to 0 when attnum == 0 */
+ Assert(off == 0 || attnum > 0);
}
+ /* We can only fetch as many attributes as the tuple has. */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, natts);
+
/*
- * If 'slow' isn't set, try deforming using deforming code that does not
- * contain any of the extra checks required for non-fixed offset
- * deforming. During deforming, if or when we find a NULL or a variable
- * length attribute, we'll switch to a deforming method which includes the
- * extra code required for non-fixed offset deforming, a.k.a slow mode.
- * Because this is performance critical, we inline
- * slot_deform_heap_tuple_internal passing the 'slow' and 'hasnull'
- * parameters as constants to allow the compiler to emit specialized code
- * with the known-const false comparisons and subsequent branches removed.
+ * Handle the portion of the tuple that we have cached the offset for up
+ * to the first NULL attribute. The offset is effectively fixed for these
+ * so we can use the CompactAttribute's attcacheoff.
*/
- if (!slow)
+ if (attnum < firstNonCacheOffsetAttr)
{
- /* Tuple without any NULLs? We can skip doing any NULL checking */
- if (!hasnulls)
- attnum = slot_deform_heap_tuple_internal(slot,
- tuple,
- attnum,
- natts,
- false, /* slow */
- false, /* hasnulls */
- &off,
- &slow);
- else
- attnum = slot_deform_heap_tuple_internal(slot,
- tuple,
- attnum,
- natts,
- false, /* slow */
- true, /* hasnulls */
- &off,
- &slow);
+ do
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+
+ off = cattr->attcacheoff;
+ values[attnum] = fetch_att_noerr(tp + off,
+ cattr->attbyval,
+ cattr->attlen);
+ } while (++attnum < firstNonCacheOffsetAttr);
+
+ /*
+ * Point the offset after the end of the last attribute with a cached
+ * offset. We expect the final cached offset attribute to have a
+ * fixed width, so just add the attlen to the attcacheoff
+ */
+ Assert(cattr->attlen > 0);
+ off += cattr->attlen;
+ }
+
+ /*
+ * Handle any portion of the tuple that doesn't have a fixed offset up
+ * until the first NULL attribute. This loops only differs from the one
+ * after it by the NULL checks.
+ */
+ for (; attnum < firstNullAttr; attnum++)
+ {
+ int attlen;
+
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ attlen = cattr->attlen;
+
+ /*
+ * cstrings don't exist in heap tuples. Use pg_assume to instruct the
+ * compiler not to emit the cstring related code in
+ * align_fetch_then_add().
+ */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ attlen,
+ cattr->attalignby);
}
- /* If there's still work to do then we must be in slow mode */
- if (attnum < natts)
+ /*
+ * Now handle any remaining attributes in the tuple up to the requested
+ * attnum. This time, include NULL checks as we're now at the first NULL
+ * attribute.
+ */
+ for (; attnum < natts; attnum++)
{
- /* XXX is it worth adding a separate call when hasnulls is false? */
- attnum = slot_deform_heap_tuple_internal(slot,
- tuple,
- attnum,
- natts,
- true, /* slow */
- hasnulls,
- &off,
- &slow);
+ int attlen;
+
+ if (isnull[attnum])
+ {
+ values[attnum] = (Datum) 0;
+ continue;
+ }
+
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ attlen = cattr->attlen;
+
+ /* As above, we don't expect cstrings */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ attlen,
+ cattr->attalignby);
}
+done:
+
/*
* Save state for next execution
*/
slot->tts_nvalid = attnum;
*offp = off;
- if (slow)
- slot->tts_flags |= TTS_FLAG_SLOW;
- else
- slot->tts_flags &= ~TTS_FLAG_SLOW;
}
const TupleTableSlotOps TTSOpsVirtual = {
@@ -1508,7 +1532,7 @@ ExecSetSlotDescriptor(TupleTableSlot *slot, /* slot to change */
slot->tts_values = (Datum *)
MemoryContextAlloc(slot->tts_mcxt, tupdesc->natts * sizeof(Datum));
slot->tts_isnull = (bool *)
- MemoryContextAlloc(slot->tts_mcxt, tupdesc->natts * sizeof(bool));
+ MemoryContextAlloc(slot->tts_mcxt, MAXALIGN(tupdesc->natts * sizeof(bool)));
}
/* --------------------------------
@@ -2259,10 +2283,16 @@ ExecTypeSetColNames(TupleDesc typeInfo, List *namesList)
* This happens "for free" if the tupdesc came from a relcache entry, but
* not if we have manufactured a tupdesc for a transient RECORD datatype.
* In that case we have to notify typcache.c of the existence of the type.
+ *
+ * TupleDescFinalize() must be called on the TupleDesc before calling this
+ * function.
*/
TupleDesc
BlessTupleDesc(TupleDesc tupdesc)
{
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupdesc->firstNonCachedOffsetAttr >= 0);
+
if (tupdesc->tdtypeid == RECORDOID &&
tupdesc->tdtypmod < 0)
assign_record_type_typmod(tupdesc);
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index af3c788ce8b..7f74a8ddcb2 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -246,6 +246,8 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags)
RelationGetDescr(scanstate->ss.ss_currentRelation),
table_slot_callbacks(scanstate->ss.ss_currentRelation));
+ scanstate->ss.ss_ScanTupleSlot->tts_flags |= TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
+
/*
* Initialize result type and projection.
*/
diff --git a/src/backend/jit/llvm/llvmjit_deform.c b/src/backend/jit/llvm/llvmjit_deform.c
index 3eb087eb56b..12521e3e46a 100644
--- a/src/backend/jit/llvm/llvmjit_deform.c
+++ b/src/backend/jit/llvm/llvmjit_deform.c
@@ -62,7 +62,6 @@ slot_compile_deform(LLVMJitContext *context, TupleDesc desc,
LLVMValueRef v_tts_values;
LLVMValueRef v_tts_nulls;
LLVMValueRef v_slotoffp;
- LLVMValueRef v_flagsp;
LLVMValueRef v_nvalidp;
LLVMValueRef v_nvalid;
LLVMValueRef v_maxatt;
@@ -178,7 +177,6 @@ slot_compile_deform(LLVMJitContext *context, TupleDesc desc,
v_tts_nulls =
l_load_struct_gep(b, StructTupleTableSlot, v_slot, FIELDNO_TUPLETABLESLOT_ISNULL,
"tts_ISNULL");
- v_flagsp = l_struct_gep(b, StructTupleTableSlot, v_slot, FIELDNO_TUPLETABLESLOT_FLAGS, "");
v_nvalidp = l_struct_gep(b, StructTupleTableSlot, v_slot, FIELDNO_TUPLETABLESLOT_NVALID, "");
if (ops == &TTSOpsHeapTuple || ops == &TTSOpsBufferHeapTuple)
@@ -747,14 +745,10 @@ slot_compile_deform(LLVMJitContext *context, TupleDesc desc,
{
LLVMValueRef v_off = l_load(b, TypeSizeT, v_offp, "");
- LLVMValueRef v_flags;
LLVMBuildStore(b, l_int16_const(lc, natts), v_nvalidp);
v_off = LLVMBuildTrunc(b, v_off, LLVMInt32TypeInContext(lc), "");
LLVMBuildStore(b, v_off, v_slotoffp);
- v_flags = l_load(b, LLVMInt16TypeInContext(lc), v_flagsp, "tts_flags");
- v_flags = LLVMBuildOr(b, v_flags, l_int16_const(lc, TTS_FLAG_SLOW), "");
- LLVMBuildStore(b, v_flags, v_flagsp);
LLVMBuildRetVoid(b);
}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 770edb34e08..998be24ac41 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -666,14 +666,6 @@ RelationBuildTupleDesc(Relation relation)
elog(ERROR, "pg_attribute catalog is missing %d attribute(s) for relation OID %u",
need, RelationGetRelid(relation));
- /*
- * We can easily set the attcacheoff value for the first attribute: it
- * must be zero. This eliminates the need for special cases for attnum=1
- * that used to exist in fastgetattr() and index_getattr().
- */
- if (RelationGetNumberOfAttributes(relation) > 0)
- TupleDescCompactAttr(relation->rd_att, 0)->attcacheoff = 0;
-
/*
* Set up constraint/default info
*/
@@ -1985,8 +1977,6 @@ formrdesc(const char *relationName, Oid relationReltype,
populate_compact_attribute(relation->rd_att, i);
}
- /* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
- TupleDescCompactAttr(relation->rd_att, 0)->attcacheoff = 0;
TupleDescFinalize(relation->rd_att);
/* mark not-null status */
@@ -4446,8 +4436,6 @@ BuildHardcodedDescriptor(int natts, const FormData_pg_attribute *attrs)
populate_compact_attribute(result, i);
}
- /* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
- TupleDescCompactAttr(result, 0)->attcacheoff = 0;
TupleDescFinalize(result);
/* Note: we don't bother to set up a TupleConstr entry */
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 595413dbbc5..d4c3a749558 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -131,6 +131,19 @@ typedef struct CompactAttribute
* Any code making changes manually to and fields in the FormData_pg_attribute
* array must subsequently call populate_compact_attribute() to flush the
* changes out to the corresponding 'compact_attrs' element.
+ *
+ * firstNonCachedOffsetAttr stores the index into the compact_attrs array for
+ * the first attribute that we don't have a known attcacheoff for.
+ *
+ * firstNonGuaranteedAttr stores the index to info the compact_attrs array for
+ * the first attribute that is either NULLable, missing, or !attbyval. This
+ * can be used in locations as a guarantee that attributes before this will
+ * always exist in tuples. The !attbyval part isn't required for this, but
+ * including this allows various tuple deforming routines to forego any checks
+ * for !attbyval.
+ *
+ * Once a TupleDesc has been populated, before it is used for any purpose
+ * TupleDescFinalize() must be called on it.
*/
typedef struct TupleDescData
{
@@ -138,6 +151,10 @@ typedef struct TupleDescData
Oid tdtypeid; /* composite type ID for tuple type */
int32 tdtypmod; /* typmod for tuple type */
int tdrefcount; /* reference count, or -1 if not counting */
+ int firstNonCachedOffsetAttr; /* index of the first att without
+ * an attcacheoff */
+ int firstNonGuaranteedAttr; /* index of the first nullable,
+ * missing or !attbyval attribute. */
TupleConstr *constr; /* constraints, or NULL if none */
/* compact_attrs[N] is the compact metadata of Attribute Number N+1 */
CompactAttribute compact_attrs[FLEXIBLE_ARRAY_MEMBER];
@@ -195,7 +212,6 @@ extern TupleDesc CreateTupleDescTruncatedCopy(TupleDesc tupdesc, int natts);
extern TupleDesc CreateTupleDescCopyConstr(TupleDesc tupdesc);
-#define TupleDescFinalize(d) ((void) 0)
#define TupleDescSize(src) \
(offsetof(struct TupleDescData, compact_attrs) + \
(src)->natts * sizeof(CompactAttribute) + \
@@ -206,6 +222,7 @@ extern void TupleDescCopy(TupleDesc dst, TupleDesc src);
extern void TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
TupleDesc src, AttrNumber srcAttno);
+extern void TupleDescFinalize(TupleDesc tupdesc);
extern void FreeTupleDesc(TupleDesc tupdesc);
extern void IncrTupleDescRefCount(TupleDesc tupdesc);
diff --git a/src/include/access/tupmacs.h b/src/include/access/tupmacs.h
index d64c18b950b..c9587a1adc6 100644
--- a/src/include/access/tupmacs.h
+++ b/src/include/access/tupmacs.h
@@ -15,7 +15,8 @@
#define TUPMACS_H
#include "catalog/pg_type_d.h" /* for TYPALIGN macros */
-
+#include "port/pg_bitutils.h"
+#include "varatt.h"
/*
* Check a tuple's null bitmap to determine whether the attribute is null.
@@ -28,6 +29,49 @@ att_isnull(int ATT, const bits8 *BITS)
return !(BITS[ATT >> 3] & (1 << (ATT & 0x07)));
}
+/*
+ * populate_isnull_array
+ * Transform a tuple's null bitmap into a boolean array.
+ *
+ * Caller must ensure that the isnull array is sized so it contains
+ * at least as many elements as there are bits in the 'bits' array.
+ * This is required because we always round 'natts' up to the next multiple
+ * of 8.
+ */
+static inline void
+populate_isnull_array(const bits8 *bits, int natts, bool *isnull)
+{
+ int nbytes = (natts + 7) >> 3;
+
+ /*
+ * Multiplying an inverted NULL bitmap byte by this value results in the
+ * lowest bit in each byte being set the same as each bit of the inverted
+ * byte. We perform this as 2 32-bit operations rather than a single
+ * 64-bit operation as multiplying by the required value to do this in
+ * 64-bits would result in overflowing a uint64 in some cases.
+ */
+#define SPREAD_BITS_MULTIPLIER_32 0x204081U
+
+ for (int i = 0; i < nbytes; i++, isnull += 8)
+ {
+ uint64 isnull_8;
+ bits8 nullbyte = ~bits[i];
+
+ /* convert the lower 4 bits of null bitmap word into 32 bit int */
+ isnull_8 = (nullbyte & 0xf) * SPREAD_BITS_MULTIPLIER_32;
+
+ /*
+ * convert the upper 4 bits of null bitmap word into 32 bit int, shift
+ * into the upper 32 bit
+ */
+ isnull_8 |= ((uint64) ((nullbyte >> 4) * SPREAD_BITS_MULTIPLIER_32)) << 32;
+
+ /* mask out all other bits apart from the lowest bit of each byte */
+ isnull_8 &= UINT64CONST(0x0101010101010101);
+ memcpy(isnull, &isnull_8, sizeof(uint64));
+ }
+}
+
#ifndef FRONTEND
/*
* Given an attbyval and an attlen from either a Form_pg_attribute or
@@ -69,6 +113,151 @@ fetch_att(const void *T, bool attbyval, int attlen)
else
return PointerGetDatum(T);
}
+
+/*
+ * Same, but no error checking for invalid attlens for byval types. This
+ * is safe to use when attlen comes from CompactAttribute as we validate the
+ * length when populating that struct.
+ */
+static inline Datum
+fetch_att_noerr(const void *T, bool attbyval, int attlen)
+{
+ if (attbyval)
+ {
+ switch (attlen)
+ {
+ case sizeof(int32):
+ return Int32GetDatum(*((const int32 *) T));
+ case sizeof(int16):
+ return Int16GetDatum(*((const int16 *) T));
+ case sizeof(char):
+ return CharGetDatum(*((const char *) T));
+ default:
+ Assert(attlen == sizeof(int64));
+ return Int64GetDatum(*((const int64 *) T));
+ }
+ }
+ else
+ return PointerGetDatum(T);
+}
+
+
+/*
+ * align_fetch_then_add
+ * Applies all the functionality of att_pointer_alignby(),
+ * fetch_att_noerr() and att_addlength_pointer() resulting in *off
+ * pointer to the perhaps unaligned number of bytes into 'tupptr', ready
+ * to deform the next attribute.
+ *
+ * tupptr: pointer to the beginning of the tuple, after the header and any
+ * NULL bitmask.
+ * off: offset in bytes for reading tuple data, possibly unaligned.
+ * attbyval, attlen, attalignby are values from CompactAttribute.
+ */
+static inline Datum
+align_fetch_then_add(const char *tupptr, uint32 *off, bool attbyval, int attlen,
+ uint8 attalignby)
+{
+ Datum res;
+
+ if (attlen > 0)
+ {
+ const char *offset_ptr;
+
+ *off = TYPEALIGN(attalignby, *off);
+ offset_ptr = tupptr + *off;
+ *off += attlen;
+ if (attbyval)
+ {
+ switch (attlen)
+ {
+ case sizeof(char):
+ return CharGetDatum(*((const char *) offset_ptr));
+ case sizeof(int16):
+ return Int16GetDatum(*((const int16 *) offset_ptr));
+ case sizeof(int32):
+ return Int32GetDatum(*((const int32 *) offset_ptr));
+ default:
+
+ /*
+ * populate_compact_attribute_internal() should have
+ * checked
+ */
+ Assert(attlen == sizeof(int64));
+ return Int64GetDatum(*((const int64 *) offset_ptr));
+ }
+ }
+ return PointerGetDatum(offset_ptr);
+ }
+ else if (attlen == -1)
+ {
+ if (!VARATT_IS_SHORT(tupptr + *off))
+ *off = TYPEALIGN(attalignby, *off);
+
+ res = PointerGetDatum(tupptr + *off);
+ *off += VARSIZE_ANY(DatumGetPointer(res));
+ return res;
+ }
+ else
+ {
+ Assert(attlen == -2);
+ *off = TYPEALIGN(attalignby, *off);
+ res = PointerGetDatum(tupptr + *off);
+ *off += strlen(tupptr + *off) + 1;
+ return res;
+ }
+}
+
+/*
+ * first_null_attr
+ * Inspect a NULL bitmask from a tuple and return the 0-based attnum of the
+ * first NULL attribute. Returns natts if no NULLs were found.
+ *
+ * We expect that 'bits' contains at least one 0 bit somewhere in the mask,
+ * not necessarily < natts.
+ */
+static inline int
+first_null_attr(const bits8 *bits, int natts)
+{
+ int lastByte = natts >> 3;
+ int bytenum;
+ int res;
+
+#ifdef USE_ASSERT_CHECKING
+ int firstnull_check = natts;
+
+ /* Do it the slow way and check we get the same answer. */
+ for (int i = 0; i < natts; i++)
+ {
+ if (att_isnull(i, bits))
+ {
+ firstnull_check = i;
+ break;
+ }
+ }
+#endif
+
+ /* Process all bytes up to just before the byte for the natts index */
+ for (bytenum = 0; bytenum < lastByte; bytenum++)
+ {
+ /* break if there's any NULL attrs (a 0 bit) */
+ if (bits[bytenum] != 0xFF)
+ break;
+ }
+
+ res = bytenum << 3;
+ res += pg_rightmost_one_pos32(~bits[bytenum]);
+
+ /*
+ * Since we did no masking to mask out bits beyond natts, we may have
+ * found a bit higher than natts, so we must cap to natts
+ */
+ res = Min(res, natts);
+
+ Assert(res == firstnull_check);
+
+ return res;
+}
#endif /* FRONTEND */
/*
diff --git a/src/include/executor/tuptable.h b/src/include/executor/tuptable.h
index a2dfd707e78..8346be77302 100644
--- a/src/include/executor/tuptable.h
+++ b/src/include/executor/tuptable.h
@@ -84,9 +84,6 @@
* tts_values/tts_isnull are allocated either when the slot is created (when
* the descriptor is provided), or when a descriptor is assigned to the slot;
* they are of length equal to the descriptor's natts.
- *
- * The TTS_FLAG_SLOW flag is saved state for
- * slot_deform_heap_tuple, and should not be touched by any other code.
*----------
*/
@@ -98,12 +95,13 @@
#define TTS_FLAG_SHOULDFREE (1 << 2)
#define TTS_SHOULDFREE(slot) (((slot)->tts_flags & TTS_FLAG_SHOULDFREE) != 0)
-/* saved state for slot_deform_heap_tuple */
-#define TTS_FLAG_SLOW (1 << 3)
-#define TTS_SLOW(slot) (((slot)->tts_flags & TTS_FLAG_SLOW) != 0)
+/* true = formed tuple guaranteed to not have NULLs in NOT NULLable columns */
+#define TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS (1 << 3)
+#define TTS_OBEYS_NOT_NULL_CONSTRAINTS(slot) \
+ (((slot)->tts_flags & TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS) != 0)
/* fixed tuple descriptor */
-#define TTS_FLAG_FIXED (1 << 4)
+#define TTS_FLAG_FIXED (1 << 4) /* XXX change to #3? */
#define TTS_FIXED(slot) (((slot)->tts_flags & TTS_FLAG_FIXED) != 0)
struct TupleTableSlotOps;
@@ -123,7 +121,9 @@ typedef struct TupleTableSlot
#define FIELDNO_TUPLETABLESLOT_VALUES 5
Datum *tts_values; /* current per-attribute values */
#define FIELDNO_TUPLETABLESLOT_ISNULL 6
- bool *tts_isnull; /* current per-attribute isnull flags */
+ bool *tts_isnull; /* current per-attribute isnull flags. Array
+ * size must always be rounded up to the next
+ * multiple of 8 elements. */
MemoryContext tts_mcxt; /* slot itself is in this context */
ItemPointerData tts_tid; /* stored tuple's tid */
Oid tts_tableOid; /* table oid of tuple */
diff --git a/src/test/modules/deform_bench/deform_bench.c b/src/test/modules/deform_bench/deform_bench.c
index 7838f639bef..de39fecf8fd 100644
--- a/src/test/modules/deform_bench/deform_bench.c
+++ b/src/test/modules/deform_bench/deform_bench.c
@@ -49,6 +49,7 @@ deform_bench(PG_FUNCTION_ARGS)
tupdesc = RelationGetDescr(rel);
slot = MakeTupleTableSlot(tupdesc, &TTSOpsBufferHeapTuple);
+ slot->tts_flags |= TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
scan = table_beginscan_strat(rel, GetActiveSnapshot(), 0, NULL, true, false);
/*
--
2.51.0
From 29ddc46737a5180786b36daba3c723d910a44bc1 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Mon, 16 Feb 2026 14:20:19 +1300
Subject: [PATCH v9 4/5] Allow sibling call optimization in
slot_getsomeattrs_int()
This changes the TupleTableSlotOps contract to make it so the
getsomeattrs() function is in charge of calling
slot_getmissingattrs().
Since this removes all code from slot_getsomeattrs_int() aside from the
getsomeattrs() call itself, we may as well adjust slot_getsomeattrs() so
that it calls getsomeattrs() directly. We leave slot_getsomeattrs_int()
intact as this is still called from the JIT code.
---
src/backend/executor/execTuples.c | 79 ++++++++++++++++---------------
src/include/executor/tuptable.h | 7 +--
2 files changed, 44 insertions(+), 42 deletions(-)
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 80faf29b797..2070c665d2f 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -73,7 +73,7 @@
static TupleDesc ExecTypeFromTLInternal(List *targetList,
bool skipjunk);
static pg_attribute_always_inline void slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
- int natts);
+ int reqnatts);
static inline void tts_buffer_heap_store_tuple(TupleTableSlot *slot,
HeapTuple tuple,
Buffer buffer,
@@ -996,7 +996,10 @@ tts_buffer_heap_store_tuple(TupleTableSlot *slot, HeapTuple tuple,
* slot_deform_heap_tuple
* Given a TupleTableSlot, extract data from the slot's physical tuple
* into its Datum/isnull arrays. Data is extracted up through the
- * natts'th column (caller must ensure this is a legal column number).
+ * reqnatts'th column. If there are insufficient attributes in the given
+ * tuple, then slot_getmissingattrs() is called to populate the
+ * remainder. If reqnatts is above the number of attributes in the
+ * slot's TupleDesc, an error is raised.
*
* This is essentially an incremental version of heap_deform_tuple:
* on each call we extract attributes up to the one needed, without
@@ -1008,7 +1011,7 @@ tts_buffer_heap_store_tuple(TupleTableSlot *slot, HeapTuple tuple,
*/
static pg_attribute_always_inline void
slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
- int natts)
+ int reqnatts)
{
CompactAttribute *cattr;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
@@ -1017,6 +1020,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int firstNonCacheOffsetAttr;
int firstNonGuaranteedAttr;
int firstNullAttr;
+ int natts;
Datum *values;
bool *isnull;
char *tp; /* ptr to tuple data */
@@ -1038,7 +1042,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
* attrs.
*/
if (TTS_OBEYS_NOT_NULL_CONSTRAINTS(slot))
- firstNonGuaranteedAttr = Min(natts, tupleDesc->firstNonGuaranteedAttr);
+ firstNonGuaranteedAttr = Min(reqnatts, tupleDesc->firstNonGuaranteedAttr);
else
firstNonGuaranteedAttr = 0;
@@ -1046,12 +1050,11 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
if (HeapTupleHasNulls(tuple))
{
- int tupnatts = HeapTupleHeaderGetNatts(tup);
-
+ natts = HeapTupleHeaderGetNatts(tup);
tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits) +
- BITMAPLEN(tupnatts));
+ BITMAPLEN(natts));
- natts = Min(tupnatts, natts);
+ natts = Min(natts, reqnatts);
if (natts > firstNonGuaranteedAttr)
{
bits8 *bp = tup->t_bits;
@@ -1082,8 +1085,13 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
* We only need to look at the tuple's natts if we need more than the
* guaranteed number of columns
*/
- if (natts > firstNonGuaranteedAttr)
- natts = Min(HeapTupleHeaderGetNatts(tup), natts);
+ if (reqnatts > firstNonGuaranteedAttr)
+ natts = Min(HeapTupleHeaderGetNatts(tup), reqnatts);
+ else
+ {
+ /* No need to access the number of attributes in the tuple */
+ natts = reqnatts;
+ }
/* All attrs can be fetched without checking for NULLs */
firstNullAttr = natts;
@@ -1091,7 +1099,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
attnum = slot->tts_nvalid;
values = slot->tts_values;
- slot->tts_nvalid = natts;
+ slot->tts_nvalid = reqnatts;
/* Ensure we calculated tp correctly */
Assert(tp == (char *) tup + tup->t_hoff);
@@ -1123,7 +1131,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
off += cattr->attlen;
- if (attnum == natts)
+ if (attnum == reqnatts)
goto done;
}
else
@@ -1222,12 +1230,12 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
cattr->attalignby);
}
+ /* Fetch any missing attrs and raise an error if reqnatts is invalid */
+ if (unlikely(attnum < reqnatts))
+ slot_getmissingattrs(slot, attnum, reqnatts);
done:
- /*
- * Save state for next execution
- */
- slot->tts_nvalid = attnum;
+ /* Save current offset for next execution */
*offp = off;
}
@@ -2088,28 +2096,29 @@ slot_getmissingattrs(TupleTableSlot *slot, int startAttNum, int lastAttNum)
if (!attrmiss)
{
/* no missing values array at all, so just fill everything in as NULL */
- memset(slot->tts_values + startAttNum, 0,
- (lastAttNum - startAttNum) * sizeof(Datum));
- memset(slot->tts_isnull + startAttNum, 1,
- (lastAttNum - startAttNum) * sizeof(bool));
+ for (int attnum = startAttNum; attnum < lastAttNum; attnum++)
+ {
+ slot->tts_values[attnum] = (Datum) 0;
+ slot->tts_isnull[attnum] = true;
+ }
}
else
{
- int missattnum;
-
- /* if there is a missing values array we must process them one by one */
- for (missattnum = startAttNum;
- missattnum < lastAttNum;
- missattnum++)
+ /* use attrmiss to set the missing values */
+ for (int attnum = startAttNum; attnum < lastAttNum; attnum++)
{
- slot->tts_values[missattnum] = attrmiss[missattnum].am_value;
- slot->tts_isnull[missattnum] = !attrmiss[missattnum].am_present;
+ slot->tts_values[attnum] = attrmiss[attnum].am_value;
+ slot->tts_isnull[attnum] = !attrmiss[attnum].am_present;
}
}
+
+ if (unlikely(lastAttNum > slot->tts_tupleDescriptor->natts))
+ elog(ERROR, "invalid attribute number %d", lastAttNum);
}
/*
- * slot_getsomeattrs_int - workhorse for slot_getsomeattrs()
+ * slot_getsomeattrs_int
+ * external function to call getsomeattrs() for use in JIT
*/
void
slot_getsomeattrs_int(TupleTableSlot *slot, int attnum)
@@ -2118,21 +2127,13 @@ slot_getsomeattrs_int(TupleTableSlot *slot, int attnum)
Assert(slot->tts_nvalid < attnum); /* checked in slot_getsomeattrs */
Assert(attnum > 0);
- if (unlikely(attnum > slot->tts_tupleDescriptor->natts))
- elog(ERROR, "invalid attribute number %d", attnum);
-
/* Fetch as many attributes as possible from the underlying tuple. */
slot->tts_ops->getsomeattrs(slot, attnum);
/*
- * If the underlying tuple doesn't have enough attributes, tuple
- * descriptor must have the missing attributes.
+ * Avoid putting new code here as that would prevent the compiler from
+ * using the sibling call optimization for the above function.
*/
- if (unlikely(slot->tts_nvalid < attnum))
- {
- slot_getmissingattrs(slot, slot->tts_nvalid, attnum);
- slot->tts_nvalid = attnum;
- }
}
/* ----------------------------------------------------------------
diff --git a/src/include/executor/tuptable.h b/src/include/executor/tuptable.h
index 8346be77302..1922c912089 100644
--- a/src/include/executor/tuptable.h
+++ b/src/include/executor/tuptable.h
@@ -153,8 +153,8 @@ struct TupleTableSlotOps
* Fill up first natts entries of tts_values and tts_isnull arrays with
* values from the tuple contained in the slot. The function may be called
* with natts more than the number of attributes available in the tuple,
- * in which case it should set tts_nvalid to the number of returned
- * columns.
+ * in which case the function must call slot_getmissingattrs() to populate
+ * the remaining attributes.
*/
void (*getsomeattrs) (TupleTableSlot *slot, int natts);
@@ -357,8 +357,9 @@ extern void slot_getsomeattrs_int(TupleTableSlot *slot, int attnum);
static inline void
slot_getsomeattrs(TupleTableSlot *slot, int attnum)
{
+ /* Populate slot with attributes up to 'attnum', if it's not already */
if (slot->tts_nvalid < attnum)
- slot_getsomeattrs_int(slot, attnum);
+ slot->tts_ops->getsomeattrs(slot, attnum);
}
/*
--
2.51.0
From db8daebf6a46aef82a0f8de70344165d3c171fb4 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Mon, 23 Feb 2026 09:39:37 +1300
Subject: [PATCH v9 5/5] Reduce size of CompactAttribute struct to 8 bytes
Previously, this was 16 bytes. With the use of some bitflags and by
reducing the attcacheoff field size to a 16-bit type, we can halve the
size of the struct.
It's unlikely that caching the offsets for offsets larger than what will
fit in a 16-bit int will help much as the tuple is very likely to have
some non-fixed-width types anyway, the offsets of which we cannot cache.
---
src/backend/access/common/tupdesc.c | 9 +++++++++
src/backend/executor/execTuples.c | 16 ++++++++++++----
src/include/access/tupdesc.h | 16 ++++++++--------
3 files changed, 29 insertions(+), 12 deletions(-)
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 028fcd53734..590dd8ec331 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -534,6 +534,15 @@ TupleDescFinalize(TupleDesc tupdesc)
cattr->attcacheoff = off;
+ /*
+ * attcacheoff is an int16, so don't try and cache any offsets larger
+ * than will fit in that type.
+ */
+ if (off > PG_INT16_MAX)
+ break;
+
+ cattr->attcacheoff = off;
+
off += cattr->attlen;
firstNonCachedOffsetAttr = i + 1;
}
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 2070c665d2f..b3699b77649 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -1013,6 +1013,7 @@ static pg_attribute_always_inline void
slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int reqnatts)
{
+ CompactAttribute *cattrs;
CompactAttribute *cattr;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
HeapTupleHeader tup = tuple->t_data;
@@ -1101,6 +1102,13 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
values = slot->tts_values;
slot->tts_nvalid = reqnatts;
+ /*
+ * We store the tupleDesc's CompactAttribute array in 'cattrs' as gcc
+ * seems to be unwilling to optimize accessing the CompactAttribute
+ * element efficiently when accessing it via TupleDescCompactAttr().
+ */
+ cattrs = tupleDesc->compact_attrs;
+
/* Ensure we calculated tp correctly */
Assert(tp == (char *) tup + tup->t_hoff);
@@ -1111,7 +1119,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int attlen;
isnull[attnum] = false;
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
attlen = cattr->attlen;
/* We don't expect any non-byval types */
@@ -1156,7 +1164,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
do
{
isnull[attnum] = false;
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
off = cattr->attcacheoff;
values[attnum] = fetch_att_noerr(tp + off,
@@ -1183,7 +1191,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int attlen;
isnull[attnum] = false;
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
attlen = cattr->attlen;
/*
@@ -1216,7 +1224,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
continue;
}
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
attlen = cattr->attlen;
/* As above, we don't expect cstrings */
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index d4c3a749558..5e54b82c296 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -55,7 +55,7 @@ typedef struct TupleConstr
* directly after the FormData_pg_attribute struct is populated or
* altered in any way.
*
- * Currently, this struct is 16 bytes. Any code changes which enlarge this
+ * Currently, this struct is 8 bytes. Any code changes which enlarge this
* struct should be considered very carefully.
*
* Code which must access a TupleDesc's attribute data should always make use
@@ -67,17 +67,17 @@ typedef struct TupleConstr
*/
typedef struct CompactAttribute
{
- int32 attcacheoff; /* fixed offset into tuple, if known, or -1 */
+ int16 attcacheoff; /* fixed offset into tuple, if known, or -1 */
int16 attlen; /* attr len in bytes or -1 = varlen, -2 =
* cstring */
bool attbyval; /* as FormData_pg_attribute.attbyval */
- bool attispackable; /* FormData_pg_attribute.attstorage !=
- * TYPSTORAGE_PLAIN */
- bool atthasmissing; /* as FormData_pg_attribute.atthasmissing */
- bool attisdropped; /* as FormData_pg_attribute.attisdropped */
- bool attgenerated; /* FormData_pg_attribute.attgenerated != '\0' */
- char attnullability; /* status of not-null constraint, see below */
uint8 attalignby; /* alignment requirement in bytes */
+ bool attispackable:1; /* FormData_pg_attribute.attstorage !=
+ * TYPSTORAGE_PLAIN */
+ bool atthasmissing:1; /* as FormData_pg_attribute.atthasmissing */
+ bool attisdropped:1; /* as FormData_pg_attribute.attisdropped */
+ bool attgenerated:1; /* FormData_pg_attribute.attgenerated != '\0' */
+ char attnullability; /* status of not-null constraint, see below */
} CompactAttribute;
/* Valid values for CompactAttribute->attnullability */
--
2.51.0
Attachments:
[text/plain] v9-0001-Introduce-deform_bench-test-module.patch (7.3K, 2-v9-0001-Introduce-deform_bench-test-module.patch)
download | inline diff:
From 54bd00f3c1b577db15fc6c3a94cdce5f61f586d1 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Tue, 27 Jan 2026 15:08:09 +1300
Subject: [PATCH v9 1/5] Introduce deform_bench test module
For benchmaring tuple deformation.
---
src/test/modules/deform_bench/.gitignore | 4 +
src/test/modules/deform_bench/Makefile | 21 ++++
.../deform_bench/deform_bench--1.0.sql | 8 ++
src/test/modules/deform_bench/deform_bench.c | 107 ++++++++++++++++++
.../modules/deform_bench/deform_bench.control | 4 +
src/test/modules/deform_bench/meson.build | 22 ++++
src/test/modules/meson.build | 1 +
7 files changed, 167 insertions(+)
create mode 100644 src/test/modules/deform_bench/.gitignore
create mode 100644 src/test/modules/deform_bench/Makefile
create mode 100644 src/test/modules/deform_bench/deform_bench--1.0.sql
create mode 100644 src/test/modules/deform_bench/deform_bench.c
create mode 100644 src/test/modules/deform_bench/deform_bench.control
create mode 100644 src/test/modules/deform_bench/meson.build
diff --git a/src/test/modules/deform_bench/.gitignore b/src/test/modules/deform_bench/.gitignore
new file mode 100644
index 00000000000..5dcb3ff9723
--- /dev/null
+++ b/src/test/modules/deform_bench/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/src/test/modules/deform_bench/Makefile b/src/test/modules/deform_bench/Makefile
new file mode 100644
index 00000000000..b5fc0f7a583
--- /dev/null
+++ b/src/test/modules/deform_bench/Makefile
@@ -0,0 +1,21 @@
+# src/test/modules/deform_bench/Makefile
+
+MODULE_big = deform_bench
+OBJS = deform_bench.o
+
+EXTENSION = deform_bench
+DATA = deform_bench--1.0.sql
+PGFILEDESC = "deform_bench - tuple deform benchmarking"
+
+REGRESS = deform_bench
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = src/test/modules/deform_bench
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/src/test/modules/deform_bench/deform_bench--1.0.sql b/src/test/modules/deform_bench/deform_bench--1.0.sql
new file mode 100644
index 00000000000..492b71dba3b
--- /dev/null
+++ b/src/test/modules/deform_bench/deform_bench--1.0.sql
@@ -0,0 +1,8 @@
+/* deform_bench--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION deform_bench" to load this file. \quit
+
+CREATE FUNCTION deform_bench(tableoid Oid, attnum int[]) RETURNS FLOAT
+AS 'MODULE_PATHNAME', 'deform_bench'
+LANGUAGE C VOLATILE STRICT;
diff --git a/src/test/modules/deform_bench/deform_bench.c b/src/test/modules/deform_bench/deform_bench.c
new file mode 100644
index 00000000000..7838f639bef
--- /dev/null
+++ b/src/test/modules/deform_bench/deform_bench.c
@@ -0,0 +1,107 @@
+/*-------------------------------------------------------------------------
+ *
+ * deform_bench.c
+ *
+ * for benchmarking tuple deformation routines
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include <time.h>
+#include <sys/time.h>
+
+#include "access/heapam.h"
+#include "access/relscan.h"
+#include "catalog/pg_am_d.h"
+#include "catalog/pg_type_d.h"
+#include "funcapi.h"
+#include "miscadmin.h"
+#include "utils/array.h"
+#include "utils/arrayaccess.h"
+#include "utils/builtins.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(deform_bench);
+
+Datum
+deform_bench(PG_FUNCTION_ARGS)
+{
+ Oid tableoid = PG_GETARG_OID(0);
+ ArrayType *array = PG_GETARG_ARRAYTYPE_P(1);
+ TableScanDesc scan;
+ Relation rel;
+ TupleDesc tupdesc;
+ TupleTableSlot *slot;
+ Datum *elem_datums = NULL;
+ bool *elem_nulls = NULL;
+ int elem_count;
+ int *attnums;
+ clock_t start,
+ end;
+
+ rel = relation_open(tableoid, AccessShareLock);
+
+ if (rel->rd_rel->relam != HEAP_TABLE_AM_OID)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("only heap AM is supported")));
+
+ tupdesc = RelationGetDescr(rel);
+ slot = MakeTupleTableSlot(tupdesc, &TTSOpsBufferHeapTuple);
+ scan = table_beginscan_strat(rel, GetActiveSnapshot(), 0, NULL, true, false);
+
+ /*
+ * The array is used to allow callers to define how many atts to deform.
+ * e.g: '{1,10}'::int[] would deform attnum=1, then in a 2nd pass deform
+ * the remainder up to attnum=10. Passing an element as NULL means all
+ * attnums. This allows simulation of incremental deformation. Generally
+ * if you're passing an array with more than 1 element, then the array
+ * should be in ascending order. Doing something like '{10,1}' would mean
+ * we've already deformed 10 attributes and on the 2nd pass there's
+ * nothing to do since attnum=1 was already deformed in the first pass.
+ *
+ * You'll get an ERROR if you pass a number higher than the number of
+ * attributes in the table.
+ */
+ deconstruct_array(array,
+ INT4OID,
+ sizeof(int32),
+ true,
+ 'i',
+ &elem_datums,
+ &elem_nulls,
+ &elem_count);
+
+ attnums = palloc_array(int, elem_count);
+
+ for (int i = 0; i < elem_count; i++)
+ {
+ /* Make a NULL element mean all attributes */
+ if (elem_nulls[i])
+ attnums[i] = tupdesc->natts;
+ else
+ attnums[i] = DatumGetInt32(elem_datums[i]);
+ }
+
+ start = clock();
+
+ while (heap_getnextslot(scan, ForwardScanDirection, slot))
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ /* Deform in stages according to the attnums array */
+ for (int i = 0; i < elem_count; i++)
+ slot_getsomeattrs(slot, attnums[i]);
+ }
+
+ end = clock();
+
+ ExecDropSingleTupleTableSlot(slot);
+ table_endscan(scan);
+ relation_close(rel, AccessShareLock);
+
+
+ /* Returns the number of milliseconds to run the test */
+ PG_RETURN_FLOAT8((double) (end - start) / (CLOCKS_PER_SEC / 1000));
+}
diff --git a/src/test/modules/deform_bench/deform_bench.control b/src/test/modules/deform_bench/deform_bench.control
new file mode 100644
index 00000000000..a2023f9d738
--- /dev/null
+++ b/src/test/modules/deform_bench/deform_bench.control
@@ -0,0 +1,4 @@
+# deform_bench extension
+comment = 'functions for benchmarking tuple deformation'
+default_version = '1.0'
+module_pathname = '$libdir/deform_bench'
diff --git a/src/test/modules/deform_bench/meson.build b/src/test/modules/deform_bench/meson.build
new file mode 100644
index 00000000000..82049585244
--- /dev/null
+++ b/src/test/modules/deform_bench/meson.build
@@ -0,0 +1,22 @@
+# Copyright (c) 2026, PostgreSQL Global Development Group
+
+deform_bench_sources = files(
+ 'deform_bench.c',
+)
+
+if host_system == 'windows'
+ deform_bench_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+ '--NAME', 'deform_bench',
+ '--FILEDESC', 'deform_bench - benchmarking tuple deformation',])
+endif
+
+deform_bench = shared_module('deform_bench',
+ deform_bench_sources,
+ kwargs: pg_test_mod_args,
+)
+test_install_libs += deform_bench
+
+test_install_data += files(
+ 'deform_bench--1.0.sql',
+ 'deform_bench.control',
+)
diff --git a/src/test/modules/meson.build b/src/test/modules/meson.build
index 2634a519935..ef2b0af4581 100644
--- a/src/test/modules/meson.build
+++ b/src/test/modules/meson.build
@@ -2,6 +2,7 @@
subdir('brin')
subdir('commit_ts')
+subdir('deform_bench')
subdir('delay_execution')
subdir('dummy_index_am')
subdir('dummy_seclabel')
--
2.51.0
[text/plain] v9-0002-Add-empty-TupleDescFinalize-function.patch (29.0K, 3-v9-0002-Add-empty-TupleDescFinalize-function.patch)
download | inline diff:
From 75a47b93e6cdba24109eb5b6a1eda57bf2e98002 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Wed, 21 Jan 2026 15:41:37 +1300
Subject: [PATCH v9 2/5] Add empty TupleDescFinalize() function
Currently does nothing, but will in a future commit.
---
contrib/dblink/dblink.c | 4 ++++
contrib/pg_buffercache/pg_buffercache_pages.c | 2 ++
contrib/pg_visibility/pg_visibility.c | 2 ++
src/backend/access/brin/brin_tuple.c | 1 +
src/backend/access/common/tupdesc.c | 13 +++++++++++++
src/backend/access/gin/ginutil.c | 1 +
src/backend/access/gist/gistscan.c | 1 +
src/backend/access/spgist/spgutils.c | 1 +
src/backend/access/transam/twophase.c | 1 +
src/backend/access/transam/xlogfuncs.c | 1 +
src/backend/backup/basebackup_copy.c | 3 +++
src/backend/catalog/index.c | 2 ++
src/backend/catalog/pg_publication.c | 1 +
src/backend/catalog/toasting.c | 6 ++++++
src/backend/commands/explain.c | 1 +
src/backend/commands/functioncmds.c | 1 +
src/backend/commands/sequence.c | 1 +
src/backend/commands/tablecmds.c | 4 ++++
src/backend/commands/wait.c | 1 +
src/backend/executor/execSRF.c | 2 ++
src/backend/executor/execTuples.c | 4 ++++
src/backend/executor/nodeFunctionscan.c | 2 ++
src/backend/parser/parse_relation.c | 4 +++-
src/backend/parser/parse_target.c | 2 ++
.../replication/libpqwalreceiver/libpqwalreceiver.c | 1 +
src/backend/replication/walsender.c | 5 +++++
src/backend/utils/adt/acl.c | 1 +
src/backend/utils/adt/genfile.c | 1 +
src/backend/utils/adt/lockfuncs.c | 1 +
src/backend/utils/adt/orderedsetaggs.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 5 +++++
src/backend/utils/adt/tsvector_op.c | 1 +
src/backend/utils/cache/relcache.c | 8 ++++++++
src/backend/utils/fmgr/funcapi.c | 6 ++++++
src/backend/utils/misc/guc_funcs.c | 5 +++++
src/include/access/tupdesc.h | 1 +
src/pl/plpgsql/src/pl_comp.c | 2 ++
.../test_custom_stats/test_custom_fixed_stats.c | 1 +
src/test/modules/test_predtest/test_predtest.c | 1 +
39 files changed, 100 insertions(+), 1 deletion(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 2498d80c8e7..4038950a6ef 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -881,6 +881,7 @@ materializeResult(FunctionCallInfo fcinfo, PGconn *conn, PGresult *res)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "status",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
ntuples = 1;
nfields = 1;
}
@@ -1044,6 +1045,7 @@ materializeQueryResult(FunctionCallInfo fcinfo,
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "status",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
attinmeta = TupleDescGetAttInMetadata(tupdesc);
oldcontext = MemoryContextSwitchTo(rsinfo->econtext->ecxt_per_query_memory);
@@ -1529,6 +1531,8 @@ dblink_get_pkey(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 2, "colname",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+
/*
* Generate attribute metadata needed later to produce tuples from raw
* C strings
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 89b86855243..a6b4fb5252b 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -174,6 +174,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
INT4OID, -1, 0);
+ TupleDescFinalize(tupledesc);
fctx->tupdesc = BlessTupleDesc(tupledesc);
/* Allocate NBuffers worth of BufferCachePagesRec records. */
@@ -442,6 +443,7 @@ pg_buffercache_os_pages_internal(FunctionCallInfo fcinfo, bool include_numa)
TupleDescInitEntry(tupledesc, (AttrNumber) 3, "numa_node",
INT4OID, -1, 0);
+ TupleDescFinalize(tupledesc);
fctx->tupdesc = BlessTupleDesc(tupledesc);
fctx->include_numa = include_numa;
diff --git a/contrib/pg_visibility/pg_visibility.c b/contrib/pg_visibility/pg_visibility.c
index 9bc3a784bf7..dfab0b64cf5 100644
--- a/contrib/pg_visibility/pg_visibility.c
+++ b/contrib/pg_visibility/pg_visibility.c
@@ -469,6 +469,8 @@ pg_visibility_tupdesc(bool include_blkno, bool include_pd)
TupleDescInitEntry(tupdesc, ++a, "pd_all_visible", BOOLOID, -1, 0);
Assert(a == maxattr);
+ TupleDescFinalize(tupdesc);
+
return BlessTupleDesc(tupdesc);
}
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 69c233c62eb..742ac089a28 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -84,6 +84,7 @@ brtuple_disk_tupdesc(BrinDesc *brdesc)
MemoryContextSwitchTo(oldcxt);
+ TupleDescFinalize(tupdesc);
brdesc->bd_disktdesc = tupdesc;
}
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index b69d10f0a45..2137385a833 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -221,6 +221,9 @@ CreateTupleDesc(int natts, Form_pg_attribute *attrs)
memcpy(TupleDescAttr(desc, i), attrs[i], ATTRIBUTE_FIXED_PART_SIZE);
populate_compact_attribute(desc, i);
}
+
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -265,6 +268,8 @@ CreateTupleDescCopy(TupleDesc tupdesc)
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -311,6 +316,8 @@ CreateTupleDescTruncatedCopy(TupleDesc tupdesc, int natts)
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -396,6 +403,8 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -438,6 +447,8 @@ TupleDescCopy(TupleDesc dst, TupleDesc src)
* source's refcount would be wrong in any case.)
*/
dst->tdrefcount = -1;
+
+ TupleDescFinalize(dst);
}
/*
@@ -1065,6 +1076,8 @@ BuildDescFromLists(const List *names, const List *types, const List *typmods, co
TupleDescInitEntryCollation(desc, attnum, attcollation);
}
+ TupleDescFinalize(desc);
+
return desc;
}
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index ff927279cc3..fe7b984ff32 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -129,6 +129,7 @@ initGinState(GinState *state, Relation index)
attr->attndims);
TupleDescInitEntryCollation(state->tupdesc[i], (AttrNumber) 2,
attr->attcollation);
+ TupleDescFinalize(state->tupdesc[i]);
}
/*
diff --git a/src/backend/access/gist/gistscan.c b/src/backend/access/gist/gistscan.c
index f23bc4a6757..c65f93abdae 100644
--- a/src/backend/access/gist/gistscan.c
+++ b/src/backend/access/gist/gistscan.c
@@ -201,6 +201,7 @@ gistrescan(IndexScanDesc scan, ScanKey key, int nkeys,
attno - 1)->atttypid,
-1, 0);
}
+ TupleDescFinalize(so->giststate->fetchTupdesc);
scan->xs_hitupdesc = so->giststate->fetchTupdesc;
/* Also create a memory context that will hold the returned tuples */
diff --git a/src/backend/access/spgist/spgutils.c b/src/backend/access/spgist/spgutils.c
index 9f5379b87ac..b246e8127db 100644
--- a/src/backend/access/spgist/spgutils.c
+++ b/src/backend/access/spgist/spgutils.c
@@ -340,6 +340,7 @@ getSpGistTupleDesc(Relation index, SpGistTypeDesc *keyType)
TupleDescCompactAttr(outTupDesc, i)->attcacheoff = -1;
populate_compact_attribute(outTupDesc, spgKeyColumn);
+ TupleDescFinalize(outTupDesc);
}
return outTupDesc;
}
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index e4340b59640..7f4ed02a6b9 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -744,6 +744,7 @@ pg_prepared_xact(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 5, "dbid",
OIDOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
/*
diff --git a/src/backend/access/transam/xlogfuncs.c b/src/backend/access/transam/xlogfuncs.c
index 2efe4105efb..b6bc616c74c 100644
--- a/src/backend/access/transam/xlogfuncs.c
+++ b/src/backend/access/transam/xlogfuncs.c
@@ -400,6 +400,7 @@ pg_walfile_name_offset(PG_FUNCTION_ARGS)
TupleDescInitEntry(resultTupleDesc, (AttrNumber) 2, "file_offset",
INT4OID, -1, 0);
+ TupleDescFinalize(resultTupleDesc);
resultTupleDesc = BlessTupleDesc(resultTupleDesc);
/*
diff --git a/src/backend/backup/basebackup_copy.c b/src/backend/backup/basebackup_copy.c
index fecfad9ab7b..29dbd0cb32f 100644
--- a/src/backend/backup/basebackup_copy.c
+++ b/src/backend/backup/basebackup_copy.c
@@ -357,6 +357,8 @@ SendXlogRecPtrResult(XLogRecPtr ptr, TimeLineID tli)
*/
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "tli", INT8OID, -1, 0);
+ TupleDescFinalize(tupdesc);
+
/* send RowDescription */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -388,6 +390,7 @@ SendTablespaceList(List *tablespaces)
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 1, "spcoid", OIDOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "spclocation", TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 3, "size", INT8OID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* send RowDescription */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 43de42ce39e..75e97fb394a 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -481,6 +481,8 @@ ConstructTupleDescriptor(Relation heapRelation,
populate_compact_attribute(indexTupDesc, i);
}
+ TupleDescFinalize(indexTupDesc);
+
return indexTupDesc;
}
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index 9a4791c573e..fa353a0dd37 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -1230,6 +1230,7 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 4, "qual",
PG_NODE_TREEOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
funcctx->user_fctx = table_infos;
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index c78dcea98c1..078a1cf5127 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -229,6 +229,12 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attcompression = InvalidCompressionMethod;
TupleDescAttr(tupdesc, 2)->attcompression = InvalidCompressionMethod;
+ populate_compact_attribute(tupdesc, 0);
+ populate_compact_attribute(tupdesc, 1);
+ populate_compact_attribute(tupdesc, 2);
+
+ TupleDescFinalize(tupdesc);
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 93918a223b8..5f922c3f5c2 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -281,6 +281,7 @@ ExplainResultDesc(ExplainStmt *stmt)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "QUERY PLAN",
result_type, -1, 0);
+ TupleDescFinalize(tupdesc);
return tupdesc;
}
diff --git a/src/backend/commands/functioncmds.c b/src/backend/commands/functioncmds.c
index 242372b1e68..3afd762e9dc 100644
--- a/src/backend/commands/functioncmds.c
+++ b/src/backend/commands/functioncmds.c
@@ -2424,6 +2424,7 @@ CallStmtResultDesc(CallStmt *stmt)
-1,
0);
}
+ TupleDescFinalize(tupdesc);
}
return tupdesc;
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index e1b808bbb60..551667650ba 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -1808,6 +1808,7 @@ pg_get_sequence_data(PG_FUNCTION_ARGS)
BOOLOID, -1, 0);
TupleDescInitEntry(resultTupleDesc, (AttrNumber) 3, "page_lsn",
LSNOID, -1, 0);
+ TupleDescFinalize(resultTupleDesc);
resultTupleDesc = BlessTupleDesc(resultTupleDesc);
seqrel = try_relation_open(relid, AccessShareLock);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index df1ba112b35..5794d421cae 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -1030,6 +1030,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
}
}
+ TupleDescFinalize(descriptor);
+
/*
* For relations with table AM and partitioned tables, select access
* method to use: an explicitly indicated one, or (in the case of a
@@ -1458,6 +1460,8 @@ BuildDescForRelation(const List *columns)
populate_compact_attribute(desc, attnum - 1);
}
+ TupleDescFinalize(desc);
+
return desc;
}
diff --git a/src/backend/commands/wait.c b/src/backend/commands/wait.c
index 1290df10c6f..8e920a72372 100644
--- a/src/backend/commands/wait.c
+++ b/src/backend/commands/wait.c
@@ -338,5 +338,6 @@ WaitStmtResultDesc(WaitStmt *stmt)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "status",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
return tupdesc;
}
diff --git a/src/backend/executor/execSRF.c b/src/backend/executor/execSRF.c
index a0b111dc0e4..b481e50acfb 100644
--- a/src/backend/executor/execSRF.c
+++ b/src/backend/executor/execSRF.c
@@ -272,6 +272,7 @@ ExecMakeTableFunctionResult(SetExprState *setexpr,
funcrettype,
-1,
0);
+ TupleDescFinalize(tupdesc);
rsinfo.setDesc = tupdesc;
}
MemoryContextSwitchTo(oldcontext);
@@ -776,6 +777,7 @@ init_sexpr(Oid foid, Oid input_collation, Expr *node,
funcrettype,
-1,
0);
+ TupleDescFinalize(tupdesc);
sexpr->funcResultDesc = tupdesc;
sexpr->funcReturnsTuple = false;
}
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index b768eae9e53..e6ab51e6404 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -2173,6 +2173,8 @@ ExecTypeFromTLInternal(List *targetList, bool skipjunk)
cur_resno++;
}
+ TupleDescFinalize(typeInfo);
+
return typeInfo;
}
@@ -2207,6 +2209,8 @@ ExecTypeFromExprList(List *exprList)
cur_resno++;
}
+ TupleDescFinalize(typeInfo);
+
return typeInfo;
}
diff --git a/src/backend/executor/nodeFunctionscan.c b/src/backend/executor/nodeFunctionscan.c
index 63e605e1f81..feb82d64967 100644
--- a/src/backend/executor/nodeFunctionscan.c
+++ b/src/backend/executor/nodeFunctionscan.c
@@ -414,6 +414,7 @@ ExecInitFunctionScan(FunctionScan *node, EState *estate, int eflags)
TupleDescInitEntryCollation(tupdesc,
(AttrNumber) 1,
exprCollation(funcexpr));
+ TupleDescFinalize(tupdesc);
}
else
{
@@ -485,6 +486,7 @@ ExecInitFunctionScan(FunctionScan *node, EState *estate, int eflags)
0);
}
+ TupleDescFinalize(scan_tupdesc);
Assert(attno == natts);
}
diff --git a/src/backend/parser/parse_relation.c b/src/backend/parser/parse_relation.c
index e003db520de..9c415e166ee 100644
--- a/src/backend/parser/parse_relation.c
+++ b/src/backend/parser/parse_relation.c
@@ -1883,6 +1883,7 @@ addRangeTableEntryForFunction(ParseState *pstate,
TupleDescInitEntryCollation(tupdesc,
(AttrNumber) 1,
exprCollation(funcexpr));
+ TupleDescFinalize(tupdesc);
}
else if (functypclass == TYPEFUNC_RECORD)
{
@@ -1940,6 +1941,7 @@ addRangeTableEntryForFunction(ParseState *pstate,
i++;
}
+ TupleDescFinalize(tupdesc);
/*
* Ensure that the coldeflist defines a legal set of names (no
@@ -2008,7 +2010,7 @@ addRangeTableEntryForFunction(ParseState *pstate,
0);
/* no need to set collation */
}
-
+ TupleDescFinalize(tupdesc);
Assert(natts == totalatts);
}
else
diff --git a/src/backend/parser/parse_target.c b/src/backend/parser/parse_target.c
index dbf5b2b5c01..a03d82c0540 100644
--- a/src/backend/parser/parse_target.c
+++ b/src/backend/parser/parse_target.c
@@ -1572,6 +1572,8 @@ expandRecordVariable(ParseState *pstate, Var *var, int levelsup)
}
Assert(lname == NULL && lvar == NULL); /* lists same length? */
+ TupleDescFinalize(tupleDesc);
+
return tupleDesc;
}
diff --git a/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c b/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
index 7c8639b32e9..9f04c9ed25d 100644
--- a/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
+++ b/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
@@ -1073,6 +1073,7 @@ libpqrcv_processTuples(PGresult *pgres, WalRcvExecResult *walres,
for (coln = 0; coln < nRetTypes; coln++)
TupleDescInitEntry(walres->tupledesc, (AttrNumber) coln + 1,
PQfname(pgres, coln), retTypes[coln], -1, 0);
+ TupleDescFinalize(walres->tupledesc);
attinmeta = TupleDescGetAttInMetadata(walres->tupledesc);
/* No point in doing more here if there were no tuples returned. */
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c
index 2cde8ebc729..33a9e8d7f21 100644
--- a/src/backend/replication/walsender.c
+++ b/src/backend/replication/walsender.c
@@ -451,6 +451,7 @@ IdentifySystem(void)
TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 4, "dbname",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -496,6 +497,7 @@ ReadReplicationSlot(ReadReplicationSlotCmd *cmd)
/* TimeLineID is unsigned, so int4 is not wide enough. */
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 3, "restart_tli",
INT8OID, -1, 0);
+ TupleDescFinalize(tupdesc);
memset(nulls, true, READ_REPLICATION_SLOT_COLS * sizeof(bool));
@@ -598,6 +600,7 @@ SendTimeLineHistory(TimeLineHistoryCmd *cmd)
tupdesc = CreateTemplateTupleDesc(2);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 1, "filename", TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "content", TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
TLHistoryFileName(histfname, cmd->timeline);
TLHistoryFilePath(path, cmd->timeline);
@@ -1015,6 +1018,7 @@ StartReplication(StartReplicationCmd *cmd)
INT8OID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "next_tli_startpos",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuple */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -1369,6 +1373,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd)
TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 4, "output_plugin",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
diff --git a/src/backend/utils/adt/acl.c b/src/backend/utils/adt/acl.c
index 641673f0b0e..ce07f2bc046 100644
--- a/src/backend/utils/adt/acl.c
+++ b/src/backend/utils/adt/acl.c
@@ -1819,6 +1819,7 @@ aclexplode(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 4, "is_grantable",
BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
/* allocate memory for user context */
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index c083608b1d5..bfb949401d0 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -454,6 +454,7 @@ pg_stat_file(PG_FUNCTION_ARGS)
"creation", TIMESTAMPTZOID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 6,
"isdir", BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
memset(isnull, false, sizeof(isnull));
diff --git a/src/backend/utils/adt/lockfuncs.c b/src/backend/utils/adt/lockfuncs.c
index 9dadd6da672..4481c354fd6 100644
--- a/src/backend/utils/adt/lockfuncs.c
+++ b/src/backend/utils/adt/lockfuncs.c
@@ -146,6 +146,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 16, "waitstart",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
/*
diff --git a/src/backend/utils/adt/orderedsetaggs.c b/src/backend/utils/adt/orderedsetaggs.c
index 3b6da8e36ac..fd8b8676470 100644
--- a/src/backend/utils/adt/orderedsetaggs.c
+++ b/src/backend/utils/adt/orderedsetaggs.c
@@ -233,6 +233,7 @@ ordered_set_startup(FunctionCallInfo fcinfo, bool use_tuples)
-1,
0);
+ TupleDescFinalize(newdesc);
FreeTupleDesc(qstate->tupdesc);
qstate->tupdesc = newdesc;
}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index b1df96e7b0b..0b10da3b180 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -769,6 +769,7 @@ pg_stat_get_backend_subxact(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 2, "subxact_overflow",
BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
if ((local_beentry = pgstat_get_local_beentry_by_proc_number(procNumber)) != NULL)
@@ -1670,6 +1671,7 @@ pg_stat_wal_build_tuple(PgStat_WalCounters wal_counters,
TupleDescInitEntry(tupdesc, (AttrNumber) 6, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
/* Fill values and NULLs */
@@ -2097,6 +2099,7 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 7, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
/* Get statistics about the archiver process */
@@ -2178,6 +2181,7 @@ pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
TIMESTAMPTZOID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 13, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
namestrcpy(&slotname, text_to_cstring(slotname_text));
@@ -2265,6 +2269,7 @@ pg_stat_get_subscription_stats(PG_FUNCTION_ARGS)
INT8OID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 13, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
if (!subentry)
diff --git a/src/backend/utils/adt/tsvector_op.c b/src/backend/utils/adt/tsvector_op.c
index 71c7c7d3b3c..d8dece42b9b 100644
--- a/src/backend/utils/adt/tsvector_op.c
+++ b/src/backend/utils/adt/tsvector_op.c
@@ -651,6 +651,7 @@ tsvector_unnest(PG_FUNCTION_ARGS)
TEXTARRAYOID, -1, 0);
if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
elog(ERROR, "return type must be a row type");
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = tupdesc;
funcctx->user_fctx = PG_GETARG_TSVECTOR_COPY(0);
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 6b634c9fff1..770edb34e08 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -729,6 +729,8 @@ RelationBuildTupleDesc(Relation relation)
pfree(constr);
relation->rd_att->constr = NULL;
}
+
+ TupleDescFinalize(relation->rd_att);
}
/*
@@ -1985,6 +1987,7 @@ formrdesc(const char *relationName, Oid relationReltype,
/* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
TupleDescCompactAttr(relation->rd_att, 0)->attcacheoff = 0;
+ TupleDescFinalize(relation->rd_att);
/* mark not-null status */
if (has_not_null)
@@ -3688,6 +3691,8 @@ RelationBuildLocalRelation(const char *relname,
for (i = 0; i < natts; i++)
TupleDescAttr(rel->rd_att, i)->attrelid = relid;
+ TupleDescFinalize(rel->rd_att);
+
rel->rd_rel->reltablespace = reltablespace;
if (mapped_relation)
@@ -4443,6 +4448,7 @@ BuildHardcodedDescriptor(int natts, const FormData_pg_attribute *attrs)
/* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
TupleDescCompactAttr(result, 0)->attcacheoff = 0;
+ TupleDescFinalize(result);
/* Note: we don't bother to set up a TupleConstr entry */
@@ -6268,6 +6274,8 @@ load_relcache_init_file(bool shared)
populate_compact_attribute(rel->rd_att, i);
}
+ TupleDescFinalize(rel->rd_att);
+
/* next read the access method specific field */
if (fread(&len, 1, sizeof(len), fp) != sizeof(len))
goto read_failed;
diff --git a/src/backend/utils/fmgr/funcapi.c b/src/backend/utils/fmgr/funcapi.c
index 8a934ea8dca..516d02cfb82 100644
--- a/src/backend/utils/fmgr/funcapi.c
+++ b/src/backend/utils/fmgr/funcapi.c
@@ -340,6 +340,8 @@ get_expr_result_type(Node *expr,
exprCollation(col));
i++;
}
+ TupleDescFinalize(tupdesc);
+
if (resultTypeId)
*resultTypeId = rexpr->row_typeid;
if (resultTupleDesc)
@@ -1044,6 +1046,7 @@ resolve_polymorphic_tupdesc(TupleDesc tupdesc, oidvector *declared_args,
}
}
+ TupleDescFinalize(tupdesc);
return true;
}
@@ -1853,6 +1856,8 @@ build_function_result_tupdesc_d(char prokind,
0);
}
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -1970,6 +1975,7 @@ TypeGetTupleDesc(Oid typeoid, List *colaliases)
typeoid,
-1,
0);
+ TupleDescFinalize(tupdesc);
}
else if (functypclass == TYPEFUNC_RECORD)
{
diff --git a/src/backend/utils/misc/guc_funcs.c b/src/backend/utils/misc/guc_funcs.c
index 8524dd3a981..472cb5393ce 100644
--- a/src/backend/utils/misc/guc_funcs.c
+++ b/src/backend/utils/misc/guc_funcs.c
@@ -444,6 +444,7 @@ GetPGVariableResultDesc(const char *name)
TupleDescInitEntry(tupdesc, (AttrNumber) 1, varname,
TEXTOID, -1, 0);
}
+ TupleDescFinalize(tupdesc);
return tupdesc;
}
@@ -465,6 +466,7 @@ ShowGUCConfigOption(const char *name, DestReceiver *dest)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 1, varname,
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -499,6 +501,7 @@ ShowAllGUCConfig(DestReceiver *dest)
TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 3, "description",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -934,6 +937,8 @@ show_all_settings(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 17, "pending_restart",
BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+
/*
* Generate attribute metadata needed later to produce tuples from raw
* C strings
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index d46cdbf7a3c..595413dbbc5 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -195,6 +195,7 @@ extern TupleDesc CreateTupleDescTruncatedCopy(TupleDesc tupdesc, int natts);
extern TupleDesc CreateTupleDescCopyConstr(TupleDesc tupdesc);
+#define TupleDescFinalize(d) ((void) 0)
#define TupleDescSize(src) \
(offsetof(struct TupleDescData, compact_attrs) + \
(src)->natts * sizeof(CompactAttribute) + \
diff --git a/src/pl/plpgsql/src/pl_comp.c b/src/pl/plpgsql/src/pl_comp.c
index 5ecc7766757..b72c963b3be 100644
--- a/src/pl/plpgsql/src/pl_comp.c
+++ b/src/pl/plpgsql/src/pl_comp.c
@@ -1912,6 +1912,8 @@ build_row_from_vars(PLpgSQL_variable **vars, int numvars)
TupleDescInitEntryCollation(row->rowtupdesc, i + 1, typcoll);
}
+ TupleDescFinalize(row->rowtupdesc);
+
return row;
}
diff --git a/src/test/modules/test_custom_stats/test_custom_fixed_stats.c b/src/test/modules/test_custom_stats/test_custom_fixed_stats.c
index 485e08e5c19..f9e7c717280 100644
--- a/src/test/modules/test_custom_stats/test_custom_fixed_stats.c
+++ b/src/test/modules/test_custom_stats/test_custom_fixed_stats.c
@@ -206,6 +206,7 @@ test_custom_stats_fixed_report(PG_FUNCTION_ARGS)
INT8OID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 2, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
values[0] = Int64GetDatum(stats->numcalls);
diff --git a/src/test/modules/test_predtest/test_predtest.c b/src/test/modules/test_predtest/test_predtest.c
index 679a5de456d..48ca2a4ea70 100644
--- a/src/test/modules/test_predtest/test_predtest.c
+++ b/src/test/modules/test_predtest/test_predtest.c
@@ -230,6 +230,7 @@ test_predtest(PG_FUNCTION_ARGS)
"s_r_holds", BOOLOID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 8,
"w_r_holds", BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
tupdesc = BlessTupleDesc(tupdesc);
values[0] = BoolGetDatum(strong_implied_by);
--
2.51.0
[text/plain] v9-0003-Optimize-tuple-deformation.patch (59.9K, 4-v9-0003-Optimize-tuple-deformation.patch)
download | inline diff:
From 9ada415253bf5c36c48145cdda2548f25588d59b Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Tue, 31 Dec 2024 09:19:24 +1300
Subject: [PATCH v9 3/5] Optimize tuple deformation
This commit includes various optimizations to improve the performance of
tuple deformation.
We now precalculate CompactAttribute's attcacheoff, which allows us to
remove the code from the deform routines which was setting the
attcacheoff. Setting the attcacheoff is handled by TupleDescFinalize(),
which must be called before the TupleDesc is used for anything. Having
this TupleDescFinalize() function means we can store the first
attribute in the TupleDesc which does not have an offset cached. That
allows us to add a dedicated deforming loop to deform all attributes up
to the final one with a attcacheoff set, or up to the first NULL
attribute, whichever comes first.
We also record the maximum attribute number which is guaranteed to exist
in the tuple, that is, has a NOT NULL constraint and isn't an
atthasmissing attribute. When deforming only attributes prior to the
guaranteed attnum, we've no need to access the tuple's natt count. As an
additional optimization, we only count fixed-width columns when
calculating the maximum guaranteed column as this eliminates the need to
emit code to fetch byref types in the deformation loop for guaranteed
attributes.
Some locations in the code deform tuples that have yet to go through NOT
NULL constraint validation. We're unable to perform the guaranteed
attribute optimization when that's the case. The optimization is opt-in
via the TupleTableSlot using the TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS
flag.
This commit also adds a more efficient way of populating the isnull
array by using a bit-wise trick which performs multiplication on the
inverse of the tuple's bitmap byte and masking out all but the lower bit
of each of the boolean's byte. This results in much more optimal code
when compared to determining the NULLness via att_isnull(). 8 isnull
elements are processed at once using this method, which means we need to
round the tts_isnull array size up to the next 8 bytes. The palloc code
does this anyway, but the round-up needed to be formalized so as not to
overwrite the sentinel byte in debug builds.
---
src/backend/access/common/heaptuple.c | 362 ++++++++----------
src/backend/access/common/indextuple.c | 371 ++++++++-----------
src/backend/access/common/tupdesc.c | 53 +++
src/backend/access/spgist/spgutils.c | 3 -
src/backend/executor/execTuples.c | 360 +++++++++---------
src/backend/executor/nodeSeqscan.c | 2 +
src/backend/jit/llvm/llvmjit_deform.c | 6 -
src/backend/utils/cache/relcache.c | 12 -
src/include/access/tupdesc.h | 19 +-
src/include/access/tupmacs.h | 191 +++++++++-
src/include/executor/tuptable.h | 16 +-
src/test/modules/deform_bench/deform_bench.c | 1 +
12 files changed, 773 insertions(+), 623 deletions(-)
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 11bec20e82e..94d11bdc4fd 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -498,19 +498,7 @@ heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc)
* nocachegetattr
*
* This only gets called from fastgetattr(), in cases where we
- * can't use a cacheoffset and the value is not null.
- *
- * This caches attribute offsets in the attribute descriptor.
- *
- * An alternative way to speed things up would be to cache offsets
- * with the tuple, but that seems more difficult unless you take
- * the storage hit of actually putting those offsets into the
- * tuple you send to disk. Yuck.
- *
- * This scheme will be slightly slower than that, but should
- * perform well for queries which hit large #'s of tuples. After
- * you cache the offsets once, examining all the other tuples using
- * the same attribute descriptor will go much quicker. -cim 5/4/91
+ * can't use the attcacheoff and the value is not null.
*
* NOTE: if you need to change this code, see also heap_deform_tuple.
* Also see nocache_index_getattr, which is the same code for index
@@ -522,194 +510,123 @@ nocachegetattr(HeapTuple tup,
int attnum,
TupleDesc tupleDesc)
{
+ CompactAttribute *cattr;
HeapTupleHeader td = tup->t_data;
char *tp; /* ptr to data part of tuple */
bits8 *bp = td->t_bits; /* ptr to null bitmap in tuple */
- bool slow = false; /* do we have to walk attrs? */
int off; /* current offset within data */
+ int startAttr;
+ int firstNullAttr;
+ int i;
+ bool hasnulls = HeapTupleHasNulls(tup);
- /* ----------------
- * Three cases:
- *
- * 1: No nulls and no variable-width attributes.
- * 2: Has a null or a var-width AFTER att.
- * 3: Has nulls or var-widths BEFORE att.
- * ----------------
- */
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
attnum--;
- if (!HeapTupleNoNulls(tup))
- {
- /*
- * there's a null somewhere in the tuple
- *
- * check to see if any preceding bits are null...
- */
- int byte = attnum >> 3;
- int finalbit = attnum & 0x07;
-
- /* check for nulls "before" final bit of last byte */
- if ((~bp[byte]) & ((1 << finalbit) - 1))
- slow = true;
- else
- {
- /* check for nulls in any "earlier" bytes */
- int i;
+ /*
+ * To reduce the number of attributes we need to look at, we start at the
+ * highest attribute that we can which has a cached offset. Since the
+ * attcacheoff for an attribute is only valid if there are no NULLs in
+ * prior attribute, we must look for NULLs to determine the start attr.
+ */
+ if (hasnulls)
+ firstNullAttr = first_null_attr(bp, attnum);
+ else
+ firstNullAttr = attnum;
- for (i = 0; i < byte; i++)
- {
- if (bp[i] != 0xFF)
- {
- slow = true;
- break;
- }
- }
- }
+ if (tupleDesc->firstNonCachedOffsetAttr > 0)
+ {
+ startAttr = Min(tupleDesc->firstNonCachedOffsetAttr - 1, firstNullAttr);
+ off = TupleDescCompactAttr(tupleDesc, startAttr)->attcacheoff;
+ }
+ else
+ {
+ startAttr = 0;
+ off = 0;
}
tp = (char *) td + td->t_hoff;
- if (!slow)
+ /*
+ * Calculate 'off' up to the first NULL attr. We use two cheaper loops
+ * when the tuple has no variable-width columns. When variable-width
+ * columns exist we use att_addlength_pointer() to move the offset beyond
+ * the current attribute.
+ */
+ if (!HeapTupleHasVarWidth(tup))
{
- CompactAttribute *att;
-
- /*
- * If we get here, there are no nulls up to and including the target
- * attribute. If we have a cached offset, we can use it.
- */
- att = TupleDescCompactAttr(tupleDesc, attnum);
- if (att->attcacheoff >= 0)
- return fetchatt(att, tp + att->attcacheoff);
-
- /*
- * Otherwise, check for non-fixed-length attrs up to and including
- * target. If there aren't any, it's safe to cheaply initialize the
- * cached offsets for these attrs.
- */
- if (HeapTupleHasVarWidth(tup))
+ for (i = startAttr; i < firstNullAttr; i++)
{
- int j;
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- for (j = 0; j <= attnum; j++)
- {
- if (TupleDescCompactAttr(tupleDesc, j)->attlen <= 0)
- {
- slow = true;
- break;
- }
- }
+ off = att_nominal_alignby(off, cattr->attalignby);
+ off += cattr->attlen;
}
- }
-
- if (!slow)
- {
- int natts = tupleDesc->natts;
- int j = 1;
-
- /*
- * If we get here, we have a tuple with no nulls or var-widths up to
- * and including the target attribute, so we can use the cached offset
- * ... only we don't have it yet, or we'd not have got here. Since
- * it's cheap to compute offsets for fixed-width columns, we take the
- * opportunity to initialize the cached offsets for *all* the leading
- * fixed-width columns, in hope of avoiding future visits to this
- * routine.
- */
- TupleDescCompactAttr(tupleDesc, 0)->attcacheoff = 0;
-
- /* we might have set some offsets in the slow path previously */
- while (j < natts && TupleDescCompactAttr(tupleDesc, j)->attcacheoff > 0)
- j++;
- off = TupleDescCompactAttr(tupleDesc, j - 1)->attcacheoff +
- TupleDescCompactAttr(tupleDesc, j - 1)->attlen;
-
- for (; j < natts; j++)
+ for (; i < attnum; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, j);
-
- if (att->attlen <= 0)
- break;
-
- off = att_nominal_alignby(off, att->attalignby);
+ if (att_isnull(i, bp))
+ continue;
- att->attcacheoff = off;
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- off += att->attlen;
+ off = att_pointer_alignby(off, cattr->attalignby, cattr->attlen,
+ tp + off);
+ off += cattr->attlen;
}
-
- Assert(j > attnum);
-
- off = TupleDescCompactAttr(tupleDesc, attnum)->attcacheoff;
}
else
{
- bool usecache = true;
- int i;
-
- /*
- * Now we know that we have to walk the tuple CAREFULLY. But we still
- * might be able to cache some offsets for next time.
- *
- * Note - This loop is a little tricky. For each non-null attribute,
- * we have to first account for alignment padding before the attr,
- * then advance over the attr based on its length. Nulls have no
- * storage and no alignment padding either. We can use/set
- * attcacheoff until we reach either a null or a var-width attribute.
- */
- off = 0;
- for (i = 0;; i++) /* loop exit is at "break" */
+ for (i = startAttr; i < firstNullAttr; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, i);
+ int attlen;
- if (HeapTupleHasNulls(tup) && att_isnull(i, bp))
- {
- usecache = false;
- continue; /* this cannot be the target att */
- }
+ cattr = TupleDescCompactAttr(tupleDesc, i);
+ attlen = cattr->attlen;
- /* If we know the next offset, we can skip the rest */
- if (usecache && att->attcacheoff >= 0)
- off = att->attcacheoff;
- else if (att->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be
- * no pad bytes in any case: then the offset will be valid for
- * either an aligned or unaligned value.
- */
- if (usecache &&
- off == att_nominal_alignby(off, att->attalignby))
- att->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, att->attalignby, -1,
- tp + off);
- usecache = false;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, att->attalignby);
+ /*
+ * cstrings don't exist in heap tuples. Use pg_assume to instruct
+ * the compiler not to emit the cstring related code in
+ * att_addlength_pointer().
+ */
+ pg_assume(attlen > 0 || attlen == -1);
- if (usecache)
- att->attcacheoff = off;
- }
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ attlen,
+ tp + off);
+ off = att_addlength_pointer(off, attlen, tp + off);
+ }
- if (i == attnum)
- break;
+ for (; i < attnum; i++)
+ {
+ int attlen;
- off = att_addlength_pointer(off, att->attlen, tp + off);
+ if (att_isnull(i, bp))
+ continue;
- if (usecache && att->attlen <= 0)
- usecache = false;
+ cattr = TupleDescCompactAttr(tupleDesc, i);
+ attlen = cattr->attlen;
+
+ /* As above, heaptuples have no cstrings */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ off = att_pointer_alignby(off, cattr->attalignby, attlen,
+ tp + off);
+ off = att_addlength_pointer(off, attlen, tp + off);
}
+
}
- return fetchatt(TupleDescCompactAttr(tupleDesc, attnum), tp + off);
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+
+ return fetchatt(cattr, tp + off);
}
/* ----------------
@@ -1347,6 +1264,7 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,
Datum *values, bool *isnull)
{
HeapTupleHeader tup = tuple->t_data;
+ CompactAttribute *cattr;
bool hasnulls = HeapTupleHasNulls(tuple);
int tdesc_natts = tupleDesc->natts;
int natts; /* number of atts to extract */
@@ -1354,70 +1272,98 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,
char *tp; /* ptr to tuple data */
uint32 off; /* offset in tuple data */
bits8 *bp = tup->t_bits; /* ptr to null bitmap in tuple */
- bool slow = false; /* can we use/set attcacheoff? */
+ int firstNonCacheOffsetAttr;
+ int firstNullAttr;
natts = HeapTupleHeaderGetNatts(tup);
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
+
/*
* In inheritance situations, it is possible that the given tuple actually
* has more fields than the caller is expecting. Don't run off the end of
* the caller's arrays.
*/
natts = Min(natts, tdesc_natts);
+ firstNonCacheOffsetAttr = Min(tupleDesc->firstNonCachedOffsetAttr, natts);
+
+ if (hasnulls)
+ {
+ firstNullAttr = first_null_attr(bp, natts);
+
+ /*
+ * XXX: it'd be nice to use populate_isnull_array() here, but that
+ * requires that the isnull array's size is rounded up to the next 8
+ * elements. Doing that would require adjusting many location that
+ * allocate the array.
+ */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, firstNullAttr);
+ }
+ else
+ firstNullAttr = natts;
tp = (char *) tup + tup->t_hoff;
+ attnum = 0;
- off = 0;
+ if (firstNonCacheOffsetAttr > 0)
+ {
+#ifdef USE_ASSERT_CHECKING
+ /* In Assert enabled builds, verify attcacheoff is correct */
+ int offcheck = 0;
+#endif
+ do
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ off = cattr->attcacheoff;
- for (attnum = 0; attnum < natts; attnum++)
+#ifdef USE_ASSERT_CHECKING
+ offcheck = att_nominal_alignby(offcheck, cattr->attalignby);
+ Assert(offcheck == cattr->attcacheoff);
+ offcheck += cattr->attlen;
+#endif
+
+ values[attnum] = fetch_att_noerr(tp + off,
+ cattr->attbyval,
+ cattr->attlen);
+ } while (++attnum < firstNonCacheOffsetAttr);
+ off += cattr->attlen;
+ }
+ else
+ off = 0;
+
+ for (; attnum < firstNullAttr; attnum++)
{
- CompactAttribute *thisatt = TupleDescCompactAttr(tupleDesc, attnum);
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
+ }
+
+ for (; attnum < natts; attnum++)
+ {
+ Assert(hasnulls);
- if (hasnulls && att_isnull(attnum, bp))
+ if (att_isnull(attnum, bp))
{
values[attnum] = (Datum) 0;
isnull[attnum] = true;
- slow = true; /* can't use attcacheoff anymore */
continue;
}
isnull[attnum] = false;
-
- if (!slow && thisatt->attcacheoff >= 0)
- off = thisatt->attcacheoff;
- else if (thisatt->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be no
- * pad bytes in any case: then the offset will be valid for either
- * an aligned or unaligned value.
- */
- if (!slow &&
- off == att_nominal_alignby(off, thisatt->attalignby))
- thisatt->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, thisatt->attalignby, -1,
- tp + off);
- slow = true;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, thisatt->attalignby);
-
- if (!slow)
- thisatt->attcacheoff = off;
- }
-
- values[attnum] = fetchatt(thisatt, tp + off);
-
- off = att_addlength_pointer(off, thisatt->attlen, tp + off);
-
- if (thisatt->attlen <= 0)
- slow = true; /* can't use attcacheoff anymore */
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
}
/*
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index d6350201e01..92282039671 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -223,18 +223,6 @@ index_form_tuple_context(TupleDesc tupleDescriptor,
*
* This gets called from index_getattr() macro, and only in cases
* where we can't use cacheoffset and the value is not null.
- *
- * This caches attribute offsets in the attribute descriptor.
- *
- * An alternative way to speed things up would be to cache offsets
- * with the tuple, but that seems more difficult unless you take
- * the storage hit of actually putting those offsets into the
- * tuple you send to disk. Yuck.
- *
- * This scheme will be slightly slower than that, but should
- * perform well for queries which hit large #'s of tuples. After
- * you cache the offsets once, examining all the other tuples using
- * the same attribute descriptor will go much quicker. -cim 5/4/91
* ----------------
*/
Datum
@@ -242,205 +230,126 @@ nocache_index_getattr(IndexTuple tup,
int attnum,
TupleDesc tupleDesc)
{
+ CompactAttribute *cattr;
char *tp; /* ptr to data part of tuple */
bits8 *bp = NULL; /* ptr to null bitmap in tuple */
- bool slow = false; /* do we have to walk attrs? */
int data_off; /* tuple data offset */
int off; /* current offset within data */
+ int startAttr;
+ int firstNullAttr;
+ bool hasnulls = IndexTupleHasNulls(tup);
+ int i;
- /* ----------------
- * Three cases:
- *
- * 1: No nulls and no variable-width attributes.
- * 2: Has a null or a var-width AFTER att.
- * 3: Has nulls or var-widths BEFORE att.
- * ----------------
- */
-
- data_off = IndexInfoFindDataOffset(tup->t_info);
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
attnum--;
- if (IndexTupleHasNulls(tup))
- {
- /*
- * there's a null somewhere in the tuple
- *
- * check to see if desired att is null
- */
+ data_off = IndexInfoFindDataOffset(tup->t_info);
+ tp = (char *) tup + data_off;
- /* XXX "knows" t_bits are just after fixed tuple header! */
+ /*
+ * To reduce the number of attributes we need to look at, we start at the
+ * highest attribute that we can which has a cached offset. Since the
+ * attcacheoff for an attribute is only valid if there are no NULLs in
+ * prior attribute, we must look for NULLs to determine the start attr.
+ */
+ if (hasnulls)
+ {
bp = (bits8 *) ((char *) tup + sizeof(IndexTupleData));
-
- /*
- * Now check to see if any preceding bits are null...
- */
- {
- int byte = attnum >> 3;
- int finalbit = attnum & 0x07;
-
- /* check for nulls "before" final bit of last byte */
- if ((~bp[byte]) & ((1 << finalbit) - 1))
- slow = true;
- else
- {
- /* check for nulls in any "earlier" bytes */
- int i;
-
- for (i = 0; i < byte; i++)
- {
- if (bp[i] != 0xFF)
- {
- slow = true;
- break;
- }
- }
- }
- }
+ firstNullAttr = first_null_attr(bp, attnum);
}
+ else
+ firstNullAttr = attnum;
- tp = (char *) tup + data_off;
-
- if (!slow)
+ if (tupleDesc->firstNonCachedOffsetAttr > 0)
{
- CompactAttribute *att;
-
- /*
- * If we get here, there are no nulls up to and including the target
- * attribute. If we have a cached offset, we can use it.
- */
- att = TupleDescCompactAttr(tupleDesc, attnum);
- if (att->attcacheoff >= 0)
- return fetchatt(att, tp + att->attcacheoff);
-
- /*
- * Otherwise, check for non-fixed-length attrs up to and including
- * target. If there aren't any, it's safe to cheaply initialize the
- * cached offsets for these attrs.
- */
- if (IndexTupleHasVarwidths(tup))
- {
- int j;
-
- for (j = 0; j <= attnum; j++)
- {
- if (TupleDescCompactAttr(tupleDesc, j)->attlen <= 0)
- {
- slow = true;
- break;
- }
- }
- }
+ startAttr = Min(tupleDesc->firstNonCachedOffsetAttr - 1, firstNullAttr);
+ off = TupleDescCompactAttr(tupleDesc, startAttr)->attcacheoff;
}
-
- if (!slow)
+ else
{
- int natts = tupleDesc->natts;
- int j = 1;
-
- /*
- * If we get here, we have a tuple with no nulls or var-widths up to
- * and including the target attribute, so we can use the cached offset
- * ... only we don't have it yet, or we'd not have got here. Since
- * it's cheap to compute offsets for fixed-width columns, we take the
- * opportunity to initialize the cached offsets for *all* the leading
- * fixed-width columns, in hope of avoiding future visits to this
- * routine.
- */
- TupleDescCompactAttr(tupleDesc, 0)->attcacheoff = 0;
+ startAttr = 0;
+ off = 0;
+ }
- /* we might have set some offsets in the slow path previously */
- while (j < natts && TupleDescCompactAttr(tupleDesc, j)->attcacheoff > 0)
- j++;
+ /*
+ * Calculate 'off' up to the first NULL attr. We use two cheaper loops
+ * when the tuple has no variable-width columns. When variable-width
+ * columns exist we use att_addlength_pointer() to move the offset beyond
+ * the current attribute.
+ */
+ if (IndexTupleHasVarwidths(tup))
+ {
+ /* Calculate the offset up until the first NULL */
+ for (i = startAttr; i < firstNullAttr; i++)
+ {
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- off = TupleDescCompactAttr(tupleDesc, j - 1)->attcacheoff +
- TupleDescCompactAttr(tupleDesc, j - 1)->attlen;
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+ off = att_addlength_pointer(off, cattr->attlen, tp + off);
+ }
- for (; j < natts; j++)
+ /* Calculate the offset for any remaining columns. */
+ for (; i < attnum; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, j);
+ Assert(hasnulls);
- if (att->attlen <= 0)
- break;
+ if (att_isnull(i, bp))
+ continue;
- off = att_nominal_alignby(off, att->attalignby);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- att->attcacheoff = off;
-
- off += att->attlen;
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+ off = att_addlength_pointer(off, cattr->attlen, tp + off);
}
-
- Assert(j > attnum);
-
- off = TupleDescCompactAttr(tupleDesc, attnum)->attcacheoff;
}
else
{
- bool usecache = true;
- int i;
+ /* Handle tuples with only fixed-width attributes */
- /*
- * Now we know that we have to walk the tuple CAREFULLY. But we still
- * might be able to cache some offsets for next time.
- *
- * Note - This loop is a little tricky. For each non-null attribute,
- * we have to first account for alignment padding before the attr,
- * then advance over the attr based on its length. Nulls have no
- * storage and no alignment padding either. We can use/set
- * attcacheoff until we reach either a null or a var-width attribute.
- */
- off = 0;
- for (i = 0;; i++) /* loop exit is at "break" */
+ /* Calculate the offset up until the first NULL */
+ for (i = startAttr; i < firstNullAttr; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, i);
-
- if (IndexTupleHasNulls(tup) && att_isnull(i, bp))
- {
- usecache = false;
- continue; /* this cannot be the target att */
- }
-
- /* If we know the next offset, we can skip the rest */
- if (usecache && att->attcacheoff >= 0)
- off = att->attcacheoff;
- else if (att->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be
- * no pad bytes in any case: then the offset will be valid for
- * either an aligned or unaligned value.
- */
- if (usecache &&
- off == att_nominal_alignby(off, att->attalignby))
- att->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, att->attalignby, -1,
- tp + off);
- usecache = false;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, att->attalignby);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
+
+ Assert(cattr->attlen > 0);
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+ off += cattr->attlen;
+ }
- if (usecache)
- att->attcacheoff = off;
- }
+ /* Calculate the offset for any remaining columns. */
+ for (; i < attnum; i++)
+ {
+ Assert(hasnulls);
- if (i == attnum)
- break;
+ if (att_isnull(i, bp))
+ continue;
- off = att_addlength_pointer(off, att->attlen, tp + off);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- if (usecache && att->attlen <= 0)
- usecache = false;
+ Assert(cattr->attlen > 0);
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+ off += cattr->attlen;
}
}
- return fetchatt(TupleDescCompactAttr(tupleDesc, attnum), tp + off);
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ off = att_pointer_alignby(off, cattr->attalignby,
+ cattr->attlen, tp + off);
+ return fetchatt(cattr, tp + off);
}
/*
@@ -480,63 +389,87 @@ index_deform_tuple_internal(TupleDesc tupleDescriptor,
Datum *values, bool *isnull,
char *tp, bits8 *bp, int hasnulls)
{
+ CompactAttribute *cattr;
int natts = tupleDescriptor->natts; /* number of atts to extract */
- int attnum;
- int off = 0; /* offset in tuple data */
- bool slow = false; /* can we use/set attcacheoff? */
+ int attnum = 0;
+ uint32 off = 0; /* offset in tuple data */
+ int firstNonCacheOffsetAttr;
+ int firstNullAttr;
/* Assert to protect callers who allocate fixed-size arrays */
Assert(natts <= INDEX_MAX_KEYS);
- for (attnum = 0; attnum < natts; attnum++)
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDescriptor->firstNonCachedOffsetAttr >= 0);
+
+ firstNonCacheOffsetAttr = Min(tupleDescriptor->firstNonCachedOffsetAttr, natts);
+
+ if (hasnulls)
+ {
+ firstNullAttr = first_null_attr(bp, natts);
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, firstNullAttr);
+ }
+ else
+ firstNullAttr = natts;
+
+ if (firstNonCacheOffsetAttr > 0)
+ {
+#ifdef USE_ASSERT_CHECKING
+ /* In Assert enabled builds, verify attcacheoff is correct */
+ off = 0;
+#endif
+
+ do
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDescriptor, attnum);
+
+#ifdef USE_ASSERT_CHECKING
+ off = att_nominal_alignby(off, cattr->attalignby);
+ Assert(off == cattr->attcacheoff);
+ off += cattr->attlen;
+#endif
+
+ values[attnum] = fetch_att_noerr(tp + cattr->attcacheoff, cattr->attbyval,
+ cattr->attlen);
+ } while (++attnum < firstNonCacheOffsetAttr);
+
+ off = cattr->attcacheoff + cattr->attlen;
+ }
+
+ for (; attnum < firstNullAttr; attnum++)
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDescriptor, attnum);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
+ }
+
+ for (; attnum < natts; attnum++)
{
- CompactAttribute *thisatt = TupleDescCompactAttr(tupleDescriptor, attnum);
+ Assert(hasnulls);
- if (hasnulls && att_isnull(attnum, bp))
+ if (att_isnull(attnum, bp))
{
values[attnum] = (Datum) 0;
isnull[attnum] = true;
- slow = true; /* can't use attcacheoff anymore */
continue;
}
isnull[attnum] = false;
-
- if (!slow && thisatt->attcacheoff >= 0)
- off = thisatt->attcacheoff;
- else if (thisatt->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be no
- * pad bytes in any case: then the offset will be valid for either
- * an aligned or unaligned value.
- */
- if (!slow &&
- off == att_nominal_alignby(off, thisatt->attalignby))
- thisatt->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, thisatt->attalignby, -1,
- tp + off);
- slow = true;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, thisatt->attalignby);
-
- if (!slow)
- thisatt->attcacheoff = off;
- }
-
- values[attnum] = fetchatt(thisatt, tp + off);
-
- off = att_addlength_pointer(off, thisatt->attlen, tp + off);
-
- if (thisatt->attlen <= 0)
- slow = true; /* can't use attcacheoff anymore */
+ cattr = TupleDescCompactAttr(tupleDescriptor, attnum);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
}
}
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 2137385a833..028fcd53734 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -197,6 +197,10 @@ CreateTemplateTupleDesc(int natts)
desc->tdtypmod = -1;
desc->tdrefcount = -1; /* assume not reference-counted */
+ /* This will be set to the correct value by TupleDescFinalize() */
+ desc->firstNonCachedOffsetAttr = -1;
+ desc->firstNonGuaranteedAttr = -1;
+
return desc;
}
@@ -457,6 +461,9 @@ TupleDescCopy(TupleDesc dst, TupleDesc src)
* descriptor to another.
*
* !!! Constraints and defaults are not copied !!!
+ *
+ * The caller must take care of calling TupleDescFinalize() on once all
+ * TupleDesc changes have been made.
*/
void
TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
@@ -489,6 +496,52 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
populate_compact_attribute(dst, dstAttno - 1);
}
+/*
+ * TupleDescFinalize
+ * Finalize the given TupleDesc. This must be called after the
+ * attributes arrays have been populated or adjusted by any code.
+ *
+ * Must be called after populate_compact_attribute() and before
+ * BlessTupleDesc().
+ */
+void
+TupleDescFinalize(TupleDesc tupdesc)
+{
+ int firstNonCachedOffsetAttr = 0;
+ int firstNonGuaranteedAttr = tupdesc->natts;
+ int off = 0;
+
+ for (int i = 0; i < tupdesc->natts; i++)
+ {
+ CompactAttribute *cattr = TupleDescCompactAttr(tupdesc, i);
+
+ if (firstNonGuaranteedAttr == tupdesc->natts &&
+ (cattr->attnullability != ATTNULLABLE_VALID || !cattr->attbyval ||
+ cattr->atthasmissing || cattr->attisdropped || cattr->attlen <= 0))
+ firstNonGuaranteedAttr = i;
+
+ if (cattr->attlen <= 0)
+ break;
+
+ off = att_nominal_alignby(off, cattr->attalignby);
+
+ /*
+ * attcacheoff is an int16, so don't try and cache any offsets larger
+ * than will fit in that type.
+ */
+ if (off > PG_INT16_MAX)
+ break;
+
+ cattr->attcacheoff = off;
+
+ off += cattr->attlen;
+ firstNonCachedOffsetAttr = i + 1;
+ }
+
+ tupdesc->firstNonCachedOffsetAttr = firstNonCachedOffsetAttr;
+ tupdesc->firstNonGuaranteedAttr = firstNonGuaranteedAttr;
+}
+
/*
* Free a TupleDesc including all substructure
*/
diff --git a/src/backend/access/spgist/spgutils.c b/src/backend/access/spgist/spgutils.c
index b246e8127db..a4694bd8065 100644
--- a/src/backend/access/spgist/spgutils.c
+++ b/src/backend/access/spgist/spgutils.c
@@ -335,9 +335,6 @@ getSpGistTupleDesc(Relation index, SpGistTypeDesc *keyType)
/* We shouldn't need to bother with making these valid: */
att->attcompression = InvalidCompressionMethod;
att->attcollation = InvalidOid;
- /* In case we changed typlen, we'd better reset following offsets */
- for (int i = spgFirstIncludeColumn; i < outTupDesc->natts; i++)
- TupleDescCompactAttr(outTupDesc, i)->attcacheoff = -1;
populate_compact_attribute(outTupDesc, spgKeyColumn);
TupleDescFinalize(outTupDesc);
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index e6ab51e6404..80faf29b797 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -993,218 +993,242 @@ tts_buffer_heap_store_tuple(TupleTableSlot *slot, HeapTuple tuple,
}
/*
- * slot_deform_heap_tuple_internal
- * An always inline helper function for use in slot_deform_heap_tuple to
- * allow the compiler to emit specialized versions of this function for
- * various combinations of "slow" and "hasnulls". For example, if a
- * given tuple has no nulls, then we needn't check "hasnulls" for every
- * attribute that we're deforming. The caller can just call this
- * function with hasnulls set to constant-false and have the compiler
- * remove the constant-false branches and emit more optimal code.
- *
- * Returns the next attnum to deform, which can be equal to natts when the
- * function manages to deform all requested attributes. *offp is an input and
- * output parameter which is the byte offset within the tuple to start deforming
- * from which, on return, gets set to the offset where the next attribute
- * should be deformed from. *slowp is set to true when subsequent deforming
- * of this tuple must use a version of this function with "slow" passed as
- * true.
- *
- * Callers cannot assume when we return "attnum" (i.e. all requested
- * attributes have been deformed) that slow mode isn't required for any
- * additional deforming as the final attribute may have caused a switch to
- * slow mode.
+ * slot_deform_heap_tuple
+ * Given a TupleTableSlot, extract data from the slot's physical tuple
+ * into its Datum/isnull arrays. Data is extracted up through the
+ * natts'th column (caller must ensure this is a legal column number).
+ *
+ * This is essentially an incremental version of heap_deform_tuple:
+ * on each call we extract attributes up to the one needed, without
+ * re-computing information about previously extracted attributes.
+ * slot->tts_nvalid is the number of attributes already extracted.
+ *
+ * This is marked as always inline, so the different offp for different types
+ * of slots gets optimized away.
*/
-static pg_attribute_always_inline int
-slot_deform_heap_tuple_internal(TupleTableSlot *slot, HeapTuple tuple,
- int attnum, int natts, bool slow,
- bool hasnulls, uint32 *offp, bool *slowp)
+static pg_attribute_always_inline void
+slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
+ int natts)
{
+ CompactAttribute *cattr;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
- Datum *values = slot->tts_values;
- bool *isnull = slot->tts_isnull;
HeapTupleHeader tup = tuple->t_data;
+ int attnum;
+ int firstNonCacheOffsetAttr;
+ int firstNonGuaranteedAttr;
+ int firstNullAttr;
+ Datum *values;
+ bool *isnull;
char *tp; /* ptr to tuple data */
- bits8 *bp = tup->t_bits; /* ptr to null bitmap in tuple */
- bool slownext = false;
+ uint32 off; /* offset in tuple data */
- tp = (char *) tup + tup->t_hoff;
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
- for (; attnum < natts; attnum++)
+ isnull = slot->tts_isnull;
+
+ /*
+ * Some callers may form and deform tuples prior to NOT NULL constraints
+ * being checked. Here we'd like to optimize the case where we only need
+ * to fetch attributes before or up to the point where the attribute is
+ * guaranteed to exist in the tuple. We rely on the slot flag being set
+ * correctly to only enable this optimization when it's valid to do so.
+ * This optimization allows us to save fetching the number of attributes
+ * from the tuple and saves the additional cost of handling non-byval
+ * attrs.
+ */
+ if (TTS_OBEYS_NOT_NULL_CONSTRAINTS(slot))
+ firstNonGuaranteedAttr = Min(natts, tupleDesc->firstNonGuaranteedAttr);
+ else
+ firstNonGuaranteedAttr = 0;
+
+ firstNonCacheOffsetAttr = tupleDesc->firstNonCachedOffsetAttr;
+
+ if (HeapTupleHasNulls(tuple))
{
- CompactAttribute *thisatt = TupleDescCompactAttr(tupleDesc, attnum);
+ int tupnatts = HeapTupleHeaderGetNatts(tup);
- if (hasnulls && att_isnull(attnum, bp))
+ tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits) +
+ BITMAPLEN(tupnatts));
+
+ natts = Min(tupnatts, natts);
+ if (natts > firstNonGuaranteedAttr)
{
- values[attnum] = (Datum) 0;
- isnull[attnum] = true;
- if (!slow)
- {
- *slowp = true;
- return attnum + 1;
- }
- else
- continue;
- }
+ bits8 *bp = tup->t_bits;
- isnull[attnum] = false;
+ /* Find the first NULL attr */
+ firstNullAttr = first_null_attr(bp, natts);
- /* calculate the offset of this attribute */
- if (!slow && thisatt->attcacheoff >= 0)
- *offp = thisatt->attcacheoff;
- else if (thisatt->attlen == -1)
- {
/*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be no
- * pad bytes in any case: then the offset will be valid for either
- * an aligned or unaligned value.
+ * And populate the isnull array for all attributes being fetched
+ * from the tuple.
*/
- if (!slow && *offp == att_nominal_alignby(*offp, thisatt->attalignby))
- thisatt->attcacheoff = *offp;
- else
- {
- *offp = att_pointer_alignby(*offp,
- thisatt->attalignby,
- -1,
- tp + *offp);
+ populate_isnull_array(bp, natts, isnull);
- if (!slow)
- slownext = true;
- }
+ /* We can only use any cached offsets until the first NULL attr */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, firstNullAttr);
}
else
{
- /* not varlena, so safe to use att_nominal_alignby */
- *offp = att_nominal_alignby(*offp, thisatt->attalignby);
-
- if (!slow)
- thisatt->attcacheoff = *offp;
+ /* Otherwise all required columns are guaranteed to exist */
+ firstNullAttr = natts;
}
+ }
+ else
+ {
+ tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits));
+
+ /*
+ * We only need to look at the tuple's natts if we need more than the
+ * guaranteed number of columns
+ */
+ if (natts > firstNonGuaranteedAttr)
+ natts = Min(HeapTupleHeaderGetNatts(tup), natts);
+
+ /* All attrs can be fetched without checking for NULLs */
+ firstNullAttr = natts;
+ }
- values[attnum] = fetchatt(thisatt, tp + *offp);
+ attnum = slot->tts_nvalid;
+ values = slot->tts_values;
+ slot->tts_nvalid = natts;
- *offp = att_addlength_pointer(*offp, thisatt->attlen, tp + *offp);
+ /* Ensure we calculated tp correctly */
+ Assert(tp == (char *) tup + tup->t_hoff);
- /* check if we need to switch to slow mode */
- if (!slow)
+ if (attnum < firstNonGuaranteedAttr)
+ {
+ do
{
+ int attlen;
+
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ attlen = cattr->attlen;
+
+ /* We don't expect any non-byval types */
+ pg_assume(attlen > 0);
+
/*
- * We're unable to deform any further if the above code set
- * 'slownext', or if this isn't a fixed-width attribute.
+ * Technically we could support non-byval fixed-width types, but
+ * not doing so allows us to pass true to fetch_att_noerr() which
+ * eliminates the !attbyval branch.
*/
- if (slownext || thisatt->attlen <= 0)
- {
- *slowp = true;
- return attnum + 1;
- }
- }
- }
+ Assert(cattr->attbyval == true);
- return natts;
-}
-
-/*
- * slot_deform_heap_tuple
- * Given a TupleTableSlot, extract data from the slot's physical tuple
- * into its Datum/isnull arrays. Data is extracted up through the
- * natts'th column (caller must ensure this is a legal column number).
- *
- * This is essentially an incremental version of heap_deform_tuple:
- * on each call we extract attributes up to the one needed, without
- * re-computing information about previously extracted attributes.
- * slot->tts_nvalid is the number of attributes already extracted.
- *
- * This is marked as always inline, so the different offp for different types
- * of slots gets optimized away.
- */
-static pg_attribute_always_inline void
-slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
- int natts)
-{
- bool hasnulls = HeapTupleHasNulls(tuple);
- int attnum;
- uint32 off; /* offset in tuple data */
- bool slow; /* can we use/set attcacheoff? */
+ off = cattr->attcacheoff;
+ values[attnum] = fetch_att_noerr(tp + off, true, attlen);
+ attnum++;
+ } while (attnum < firstNonGuaranteedAttr);
- /* We can only fetch as many attributes as the tuple has. */
- natts = Min(HeapTupleHeaderGetNatts(tuple->t_data), natts);
+ off += cattr->attlen;
- /*
- * Check whether the first call for this tuple, and initialize or restore
- * loop state.
- */
- attnum = slot->tts_nvalid;
- if (attnum == 0)
- {
- /* Start from the first attribute */
- off = 0;
- slow = false;
+ if (attnum == natts)
+ goto done;
}
else
{
/* Restore state from previous execution */
off = *offp;
- slow = TTS_SLOW(slot);
+
+ /* We expect *offp to be set to 0 when attnum == 0 */
+ Assert(off == 0 || attnum > 0);
}
+ /* We can only fetch as many attributes as the tuple has. */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, natts);
+
/*
- * If 'slow' isn't set, try deforming using deforming code that does not
- * contain any of the extra checks required for non-fixed offset
- * deforming. During deforming, if or when we find a NULL or a variable
- * length attribute, we'll switch to a deforming method which includes the
- * extra code required for non-fixed offset deforming, a.k.a slow mode.
- * Because this is performance critical, we inline
- * slot_deform_heap_tuple_internal passing the 'slow' and 'hasnull'
- * parameters as constants to allow the compiler to emit specialized code
- * with the known-const false comparisons and subsequent branches removed.
+ * Handle the portion of the tuple that we have cached the offset for up
+ * to the first NULL attribute. The offset is effectively fixed for these
+ * so we can use the CompactAttribute's attcacheoff.
*/
- if (!slow)
+ if (attnum < firstNonCacheOffsetAttr)
{
- /* Tuple without any NULLs? We can skip doing any NULL checking */
- if (!hasnulls)
- attnum = slot_deform_heap_tuple_internal(slot,
- tuple,
- attnum,
- natts,
- false, /* slow */
- false, /* hasnulls */
- &off,
- &slow);
- else
- attnum = slot_deform_heap_tuple_internal(slot,
- tuple,
- attnum,
- natts,
- false, /* slow */
- true, /* hasnulls */
- &off,
- &slow);
+ do
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+
+ off = cattr->attcacheoff;
+ values[attnum] = fetch_att_noerr(tp + off,
+ cattr->attbyval,
+ cattr->attlen);
+ } while (++attnum < firstNonCacheOffsetAttr);
+
+ /*
+ * Point the offset after the end of the last attribute with a cached
+ * offset. We expect the final cached offset attribute to have a
+ * fixed width, so just add the attlen to the attcacheoff
+ */
+ Assert(cattr->attlen > 0);
+ off += cattr->attlen;
+ }
+
+ /*
+ * Handle any portion of the tuple that doesn't have a fixed offset up
+ * until the first NULL attribute. This loops only differs from the one
+ * after it by the NULL checks.
+ */
+ for (; attnum < firstNullAttr; attnum++)
+ {
+ int attlen;
+
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ attlen = cattr->attlen;
+
+ /*
+ * cstrings don't exist in heap tuples. Use pg_assume to instruct the
+ * compiler not to emit the cstring related code in
+ * align_fetch_then_add().
+ */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ attlen,
+ cattr->attalignby);
}
- /* If there's still work to do then we must be in slow mode */
- if (attnum < natts)
+ /*
+ * Now handle any remaining attributes in the tuple up to the requested
+ * attnum. This time, include NULL checks as we're now at the first NULL
+ * attribute.
+ */
+ for (; attnum < natts; attnum++)
{
- /* XXX is it worth adding a separate call when hasnulls is false? */
- attnum = slot_deform_heap_tuple_internal(slot,
- tuple,
- attnum,
- natts,
- true, /* slow */
- hasnulls,
- &off,
- &slow);
+ int attlen;
+
+ if (isnull[attnum])
+ {
+ values[attnum] = (Datum) 0;
+ continue;
+ }
+
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ attlen = cattr->attlen;
+
+ /* As above, we don't expect cstrings */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ attlen,
+ cattr->attalignby);
}
+done:
+
/*
* Save state for next execution
*/
slot->tts_nvalid = attnum;
*offp = off;
- if (slow)
- slot->tts_flags |= TTS_FLAG_SLOW;
- else
- slot->tts_flags &= ~TTS_FLAG_SLOW;
}
const TupleTableSlotOps TTSOpsVirtual = {
@@ -1508,7 +1532,7 @@ ExecSetSlotDescriptor(TupleTableSlot *slot, /* slot to change */
slot->tts_values = (Datum *)
MemoryContextAlloc(slot->tts_mcxt, tupdesc->natts * sizeof(Datum));
slot->tts_isnull = (bool *)
- MemoryContextAlloc(slot->tts_mcxt, tupdesc->natts * sizeof(bool));
+ MemoryContextAlloc(slot->tts_mcxt, MAXALIGN(tupdesc->natts * sizeof(bool)));
}
/* --------------------------------
@@ -2259,10 +2283,16 @@ ExecTypeSetColNames(TupleDesc typeInfo, List *namesList)
* This happens "for free" if the tupdesc came from a relcache entry, but
* not if we have manufactured a tupdesc for a transient RECORD datatype.
* In that case we have to notify typcache.c of the existence of the type.
+ *
+ * TupleDescFinalize() must be called on the TupleDesc before calling this
+ * function.
*/
TupleDesc
BlessTupleDesc(TupleDesc tupdesc)
{
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupdesc->firstNonCachedOffsetAttr >= 0);
+
if (tupdesc->tdtypeid == RECORDOID &&
tupdesc->tdtypmod < 0)
assign_record_type_typmod(tupdesc);
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index af3c788ce8b..7f74a8ddcb2 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -246,6 +246,8 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags)
RelationGetDescr(scanstate->ss.ss_currentRelation),
table_slot_callbacks(scanstate->ss.ss_currentRelation));
+ scanstate->ss.ss_ScanTupleSlot->tts_flags |= TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
+
/*
* Initialize result type and projection.
*/
diff --git a/src/backend/jit/llvm/llvmjit_deform.c b/src/backend/jit/llvm/llvmjit_deform.c
index 3eb087eb56b..12521e3e46a 100644
--- a/src/backend/jit/llvm/llvmjit_deform.c
+++ b/src/backend/jit/llvm/llvmjit_deform.c
@@ -62,7 +62,6 @@ slot_compile_deform(LLVMJitContext *context, TupleDesc desc,
LLVMValueRef v_tts_values;
LLVMValueRef v_tts_nulls;
LLVMValueRef v_slotoffp;
- LLVMValueRef v_flagsp;
LLVMValueRef v_nvalidp;
LLVMValueRef v_nvalid;
LLVMValueRef v_maxatt;
@@ -178,7 +177,6 @@ slot_compile_deform(LLVMJitContext *context, TupleDesc desc,
v_tts_nulls =
l_load_struct_gep(b, StructTupleTableSlot, v_slot, FIELDNO_TUPLETABLESLOT_ISNULL,
"tts_ISNULL");
- v_flagsp = l_struct_gep(b, StructTupleTableSlot, v_slot, FIELDNO_TUPLETABLESLOT_FLAGS, "");
v_nvalidp = l_struct_gep(b, StructTupleTableSlot, v_slot, FIELDNO_TUPLETABLESLOT_NVALID, "");
if (ops == &TTSOpsHeapTuple || ops == &TTSOpsBufferHeapTuple)
@@ -747,14 +745,10 @@ slot_compile_deform(LLVMJitContext *context, TupleDesc desc,
{
LLVMValueRef v_off = l_load(b, TypeSizeT, v_offp, "");
- LLVMValueRef v_flags;
LLVMBuildStore(b, l_int16_const(lc, natts), v_nvalidp);
v_off = LLVMBuildTrunc(b, v_off, LLVMInt32TypeInContext(lc), "");
LLVMBuildStore(b, v_off, v_slotoffp);
- v_flags = l_load(b, LLVMInt16TypeInContext(lc), v_flagsp, "tts_flags");
- v_flags = LLVMBuildOr(b, v_flags, l_int16_const(lc, TTS_FLAG_SLOW), "");
- LLVMBuildStore(b, v_flags, v_flagsp);
LLVMBuildRetVoid(b);
}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 770edb34e08..998be24ac41 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -666,14 +666,6 @@ RelationBuildTupleDesc(Relation relation)
elog(ERROR, "pg_attribute catalog is missing %d attribute(s) for relation OID %u",
need, RelationGetRelid(relation));
- /*
- * We can easily set the attcacheoff value for the first attribute: it
- * must be zero. This eliminates the need for special cases for attnum=1
- * that used to exist in fastgetattr() and index_getattr().
- */
- if (RelationGetNumberOfAttributes(relation) > 0)
- TupleDescCompactAttr(relation->rd_att, 0)->attcacheoff = 0;
-
/*
* Set up constraint/default info
*/
@@ -1985,8 +1977,6 @@ formrdesc(const char *relationName, Oid relationReltype,
populate_compact_attribute(relation->rd_att, i);
}
- /* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
- TupleDescCompactAttr(relation->rd_att, 0)->attcacheoff = 0;
TupleDescFinalize(relation->rd_att);
/* mark not-null status */
@@ -4446,8 +4436,6 @@ BuildHardcodedDescriptor(int natts, const FormData_pg_attribute *attrs)
populate_compact_attribute(result, i);
}
- /* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
- TupleDescCompactAttr(result, 0)->attcacheoff = 0;
TupleDescFinalize(result);
/* Note: we don't bother to set up a TupleConstr entry */
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 595413dbbc5..d4c3a749558 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -131,6 +131,19 @@ typedef struct CompactAttribute
* Any code making changes manually to and fields in the FormData_pg_attribute
* array must subsequently call populate_compact_attribute() to flush the
* changes out to the corresponding 'compact_attrs' element.
+ *
+ * firstNonCachedOffsetAttr stores the index into the compact_attrs array for
+ * the first attribute that we don't have a known attcacheoff for.
+ *
+ * firstNonGuaranteedAttr stores the index to info the compact_attrs array for
+ * the first attribute that is either NULLable, missing, or !attbyval. This
+ * can be used in locations as a guarantee that attributes before this will
+ * always exist in tuples. The !attbyval part isn't required for this, but
+ * including this allows various tuple deforming routines to forego any checks
+ * for !attbyval.
+ *
+ * Once a TupleDesc has been populated, before it is used for any purpose
+ * TupleDescFinalize() must be called on it.
*/
typedef struct TupleDescData
{
@@ -138,6 +151,10 @@ typedef struct TupleDescData
Oid tdtypeid; /* composite type ID for tuple type */
int32 tdtypmod; /* typmod for tuple type */
int tdrefcount; /* reference count, or -1 if not counting */
+ int firstNonCachedOffsetAttr; /* index of the first att without
+ * an attcacheoff */
+ int firstNonGuaranteedAttr; /* index of the first nullable,
+ * missing or !attbyval attribute. */
TupleConstr *constr; /* constraints, or NULL if none */
/* compact_attrs[N] is the compact metadata of Attribute Number N+1 */
CompactAttribute compact_attrs[FLEXIBLE_ARRAY_MEMBER];
@@ -195,7 +212,6 @@ extern TupleDesc CreateTupleDescTruncatedCopy(TupleDesc tupdesc, int natts);
extern TupleDesc CreateTupleDescCopyConstr(TupleDesc tupdesc);
-#define TupleDescFinalize(d) ((void) 0)
#define TupleDescSize(src) \
(offsetof(struct TupleDescData, compact_attrs) + \
(src)->natts * sizeof(CompactAttribute) + \
@@ -206,6 +222,7 @@ extern void TupleDescCopy(TupleDesc dst, TupleDesc src);
extern void TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
TupleDesc src, AttrNumber srcAttno);
+extern void TupleDescFinalize(TupleDesc tupdesc);
extern void FreeTupleDesc(TupleDesc tupdesc);
extern void IncrTupleDescRefCount(TupleDesc tupdesc);
diff --git a/src/include/access/tupmacs.h b/src/include/access/tupmacs.h
index d64c18b950b..c9587a1adc6 100644
--- a/src/include/access/tupmacs.h
+++ b/src/include/access/tupmacs.h
@@ -15,7 +15,8 @@
#define TUPMACS_H
#include "catalog/pg_type_d.h" /* for TYPALIGN macros */
-
+#include "port/pg_bitutils.h"
+#include "varatt.h"
/*
* Check a tuple's null bitmap to determine whether the attribute is null.
@@ -28,6 +29,49 @@ att_isnull(int ATT, const bits8 *BITS)
return !(BITS[ATT >> 3] & (1 << (ATT & 0x07)));
}
+/*
+ * populate_isnull_array
+ * Transform a tuple's null bitmap into a boolean array.
+ *
+ * Caller must ensure that the isnull array is sized so it contains
+ * at least as many elements as there are bits in the 'bits' array.
+ * This is required because we always round 'natts' up to the next multiple
+ * of 8.
+ */
+static inline void
+populate_isnull_array(const bits8 *bits, int natts, bool *isnull)
+{
+ int nbytes = (natts + 7) >> 3;
+
+ /*
+ * Multiplying an inverted NULL bitmap byte by this value results in the
+ * lowest bit in each byte being set the same as each bit of the inverted
+ * byte. We perform this as 2 32-bit operations rather than a single
+ * 64-bit operation as multiplying by the required value to do this in
+ * 64-bits would result in overflowing a uint64 in some cases.
+ */
+#define SPREAD_BITS_MULTIPLIER_32 0x204081U
+
+ for (int i = 0; i < nbytes; i++, isnull += 8)
+ {
+ uint64 isnull_8;
+ bits8 nullbyte = ~bits[i];
+
+ /* convert the lower 4 bits of null bitmap word into 32 bit int */
+ isnull_8 = (nullbyte & 0xf) * SPREAD_BITS_MULTIPLIER_32;
+
+ /*
+ * convert the upper 4 bits of null bitmap word into 32 bit int, shift
+ * into the upper 32 bit
+ */
+ isnull_8 |= ((uint64) ((nullbyte >> 4) * SPREAD_BITS_MULTIPLIER_32)) << 32;
+
+ /* mask out all other bits apart from the lowest bit of each byte */
+ isnull_8 &= UINT64CONST(0x0101010101010101);
+ memcpy(isnull, &isnull_8, sizeof(uint64));
+ }
+}
+
#ifndef FRONTEND
/*
* Given an attbyval and an attlen from either a Form_pg_attribute or
@@ -69,6 +113,151 @@ fetch_att(const void *T, bool attbyval, int attlen)
else
return PointerGetDatum(T);
}
+
+/*
+ * Same, but no error checking for invalid attlens for byval types. This
+ * is safe to use when attlen comes from CompactAttribute as we validate the
+ * length when populating that struct.
+ */
+static inline Datum
+fetch_att_noerr(const void *T, bool attbyval, int attlen)
+{
+ if (attbyval)
+ {
+ switch (attlen)
+ {
+ case sizeof(int32):
+ return Int32GetDatum(*((const int32 *) T));
+ case sizeof(int16):
+ return Int16GetDatum(*((const int16 *) T));
+ case sizeof(char):
+ return CharGetDatum(*((const char *) T));
+ default:
+ Assert(attlen == sizeof(int64));
+ return Int64GetDatum(*((const int64 *) T));
+ }
+ }
+ else
+ return PointerGetDatum(T);
+}
+
+
+/*
+ * align_fetch_then_add
+ * Applies all the functionality of att_pointer_alignby(),
+ * fetch_att_noerr() and att_addlength_pointer() resulting in *off
+ * pointer to the perhaps unaligned number of bytes into 'tupptr', ready
+ * to deform the next attribute.
+ *
+ * tupptr: pointer to the beginning of the tuple, after the header and any
+ * NULL bitmask.
+ * off: offset in bytes for reading tuple data, possibly unaligned.
+ * attbyval, attlen, attalignby are values from CompactAttribute.
+ */
+static inline Datum
+align_fetch_then_add(const char *tupptr, uint32 *off, bool attbyval, int attlen,
+ uint8 attalignby)
+{
+ Datum res;
+
+ if (attlen > 0)
+ {
+ const char *offset_ptr;
+
+ *off = TYPEALIGN(attalignby, *off);
+ offset_ptr = tupptr + *off;
+ *off += attlen;
+ if (attbyval)
+ {
+ switch (attlen)
+ {
+ case sizeof(char):
+ return CharGetDatum(*((const char *) offset_ptr));
+ case sizeof(int16):
+ return Int16GetDatum(*((const int16 *) offset_ptr));
+ case sizeof(int32):
+ return Int32GetDatum(*((const int32 *) offset_ptr));
+ default:
+
+ /*
+ * populate_compact_attribute_internal() should have
+ * checked
+ */
+ Assert(attlen == sizeof(int64));
+ return Int64GetDatum(*((const int64 *) offset_ptr));
+ }
+ }
+ return PointerGetDatum(offset_ptr);
+ }
+ else if (attlen == -1)
+ {
+ if (!VARATT_IS_SHORT(tupptr + *off))
+ *off = TYPEALIGN(attalignby, *off);
+
+ res = PointerGetDatum(tupptr + *off);
+ *off += VARSIZE_ANY(DatumGetPointer(res));
+ return res;
+ }
+ else
+ {
+ Assert(attlen == -2);
+ *off = TYPEALIGN(attalignby, *off);
+ res = PointerGetDatum(tupptr + *off);
+ *off += strlen(tupptr + *off) + 1;
+ return res;
+ }
+}
+
+/*
+ * first_null_attr
+ * Inspect a NULL bitmask from a tuple and return the 0-based attnum of the
+ * first NULL attribute. Returns natts if no NULLs were found.
+ *
+ * We expect that 'bits' contains at least one 0 bit somewhere in the mask,
+ * not necessarily < natts.
+ */
+static inline int
+first_null_attr(const bits8 *bits, int natts)
+{
+ int lastByte = natts >> 3;
+ int bytenum;
+ int res;
+
+#ifdef USE_ASSERT_CHECKING
+ int firstnull_check = natts;
+
+ /* Do it the slow way and check we get the same answer. */
+ for (int i = 0; i < natts; i++)
+ {
+ if (att_isnull(i, bits))
+ {
+ firstnull_check = i;
+ break;
+ }
+ }
+#endif
+
+ /* Process all bytes up to just before the byte for the natts index */
+ for (bytenum = 0; bytenum < lastByte; bytenum++)
+ {
+ /* break if there's any NULL attrs (a 0 bit) */
+ if (bits[bytenum] != 0xFF)
+ break;
+ }
+
+ res = bytenum << 3;
+ res += pg_rightmost_one_pos32(~bits[bytenum]);
+
+ /*
+ * Since we did no masking to mask out bits beyond natts, we may have
+ * found a bit higher than natts, so we must cap to natts
+ */
+ res = Min(res, natts);
+
+ Assert(res == firstnull_check);
+
+ return res;
+}
#endif /* FRONTEND */
/*
diff --git a/src/include/executor/tuptable.h b/src/include/executor/tuptable.h
index a2dfd707e78..8346be77302 100644
--- a/src/include/executor/tuptable.h
+++ b/src/include/executor/tuptable.h
@@ -84,9 +84,6 @@
* tts_values/tts_isnull are allocated either when the slot is created (when
* the descriptor is provided), or when a descriptor is assigned to the slot;
* they are of length equal to the descriptor's natts.
- *
- * The TTS_FLAG_SLOW flag is saved state for
- * slot_deform_heap_tuple, and should not be touched by any other code.
*----------
*/
@@ -98,12 +95,13 @@
#define TTS_FLAG_SHOULDFREE (1 << 2)
#define TTS_SHOULDFREE(slot) (((slot)->tts_flags & TTS_FLAG_SHOULDFREE) != 0)
-/* saved state for slot_deform_heap_tuple */
-#define TTS_FLAG_SLOW (1 << 3)
-#define TTS_SLOW(slot) (((slot)->tts_flags & TTS_FLAG_SLOW) != 0)
+/* true = formed tuple guaranteed to not have NULLs in NOT NULLable columns */
+#define TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS (1 << 3)
+#define TTS_OBEYS_NOT_NULL_CONSTRAINTS(slot) \
+ (((slot)->tts_flags & TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS) != 0)
/* fixed tuple descriptor */
-#define TTS_FLAG_FIXED (1 << 4)
+#define TTS_FLAG_FIXED (1 << 4) /* XXX change to #3? */
#define TTS_FIXED(slot) (((slot)->tts_flags & TTS_FLAG_FIXED) != 0)
struct TupleTableSlotOps;
@@ -123,7 +121,9 @@ typedef struct TupleTableSlot
#define FIELDNO_TUPLETABLESLOT_VALUES 5
Datum *tts_values; /* current per-attribute values */
#define FIELDNO_TUPLETABLESLOT_ISNULL 6
- bool *tts_isnull; /* current per-attribute isnull flags */
+ bool *tts_isnull; /* current per-attribute isnull flags. Array
+ * size must always be rounded up to the next
+ * multiple of 8 elements. */
MemoryContext tts_mcxt; /* slot itself is in this context */
ItemPointerData tts_tid; /* stored tuple's tid */
Oid tts_tableOid; /* table oid of tuple */
diff --git a/src/test/modules/deform_bench/deform_bench.c b/src/test/modules/deform_bench/deform_bench.c
index 7838f639bef..de39fecf8fd 100644
--- a/src/test/modules/deform_bench/deform_bench.c
+++ b/src/test/modules/deform_bench/deform_bench.c
@@ -49,6 +49,7 @@ deform_bench(PG_FUNCTION_ARGS)
tupdesc = RelationGetDescr(rel);
slot = MakeTupleTableSlot(tupdesc, &TTSOpsBufferHeapTuple);
+ slot->tts_flags |= TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
scan = table_beginscan_strat(rel, GetActiveSnapshot(), 0, NULL, true, false);
/*
--
2.51.0
[text/plain] v9-0004-Allow-sibling-call-optimization-in-slot_getsomeat.patch (8.5K, 5-v9-0004-Allow-sibling-call-optimization-in-slot_getsomeat.patch)
download | inline diff:
From 29ddc46737a5180786b36daba3c723d910a44bc1 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Mon, 16 Feb 2026 14:20:19 +1300
Subject: [PATCH v9 4/5] Allow sibling call optimization in
slot_getsomeattrs_int()
This changes the TupleTableSlotOps contract to make it so the
getsomeattrs() function is in charge of calling
slot_getmissingattrs().
Since this removes all code from slot_getsomeattrs_int() aside from the
getsomeattrs() call itself, we may as well adjust slot_getsomeattrs() so
that it calls getsomeattrs() directly. We leave slot_getsomeattrs_int()
intact as this is still called from the JIT code.
---
src/backend/executor/execTuples.c | 79 ++++++++++++++++---------------
src/include/executor/tuptable.h | 7 +--
2 files changed, 44 insertions(+), 42 deletions(-)
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 80faf29b797..2070c665d2f 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -73,7 +73,7 @@
static TupleDesc ExecTypeFromTLInternal(List *targetList,
bool skipjunk);
static pg_attribute_always_inline void slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
- int natts);
+ int reqnatts);
static inline void tts_buffer_heap_store_tuple(TupleTableSlot *slot,
HeapTuple tuple,
Buffer buffer,
@@ -996,7 +996,10 @@ tts_buffer_heap_store_tuple(TupleTableSlot *slot, HeapTuple tuple,
* slot_deform_heap_tuple
* Given a TupleTableSlot, extract data from the slot's physical tuple
* into its Datum/isnull arrays. Data is extracted up through the
- * natts'th column (caller must ensure this is a legal column number).
+ * reqnatts'th column. If there are insufficient attributes in the given
+ * tuple, then slot_getmissingattrs() is called to populate the
+ * remainder. If reqnatts is above the number of attributes in the
+ * slot's TupleDesc, an error is raised.
*
* This is essentially an incremental version of heap_deform_tuple:
* on each call we extract attributes up to the one needed, without
@@ -1008,7 +1011,7 @@ tts_buffer_heap_store_tuple(TupleTableSlot *slot, HeapTuple tuple,
*/
static pg_attribute_always_inline void
slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
- int natts)
+ int reqnatts)
{
CompactAttribute *cattr;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
@@ -1017,6 +1020,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int firstNonCacheOffsetAttr;
int firstNonGuaranteedAttr;
int firstNullAttr;
+ int natts;
Datum *values;
bool *isnull;
char *tp; /* ptr to tuple data */
@@ -1038,7 +1042,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
* attrs.
*/
if (TTS_OBEYS_NOT_NULL_CONSTRAINTS(slot))
- firstNonGuaranteedAttr = Min(natts, tupleDesc->firstNonGuaranteedAttr);
+ firstNonGuaranteedAttr = Min(reqnatts, tupleDesc->firstNonGuaranteedAttr);
else
firstNonGuaranteedAttr = 0;
@@ -1046,12 +1050,11 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
if (HeapTupleHasNulls(tuple))
{
- int tupnatts = HeapTupleHeaderGetNatts(tup);
-
+ natts = HeapTupleHeaderGetNatts(tup);
tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits) +
- BITMAPLEN(tupnatts));
+ BITMAPLEN(natts));
- natts = Min(tupnatts, natts);
+ natts = Min(natts, reqnatts);
if (natts > firstNonGuaranteedAttr)
{
bits8 *bp = tup->t_bits;
@@ -1082,8 +1085,13 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
* We only need to look at the tuple's natts if we need more than the
* guaranteed number of columns
*/
- if (natts > firstNonGuaranteedAttr)
- natts = Min(HeapTupleHeaderGetNatts(tup), natts);
+ if (reqnatts > firstNonGuaranteedAttr)
+ natts = Min(HeapTupleHeaderGetNatts(tup), reqnatts);
+ else
+ {
+ /* No need to access the number of attributes in the tuple */
+ natts = reqnatts;
+ }
/* All attrs can be fetched without checking for NULLs */
firstNullAttr = natts;
@@ -1091,7 +1099,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
attnum = slot->tts_nvalid;
values = slot->tts_values;
- slot->tts_nvalid = natts;
+ slot->tts_nvalid = reqnatts;
/* Ensure we calculated tp correctly */
Assert(tp == (char *) tup + tup->t_hoff);
@@ -1123,7 +1131,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
off += cattr->attlen;
- if (attnum == natts)
+ if (attnum == reqnatts)
goto done;
}
else
@@ -1222,12 +1230,12 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
cattr->attalignby);
}
+ /* Fetch any missing attrs and raise an error if reqnatts is invalid */
+ if (unlikely(attnum < reqnatts))
+ slot_getmissingattrs(slot, attnum, reqnatts);
done:
- /*
- * Save state for next execution
- */
- slot->tts_nvalid = attnum;
+ /* Save current offset for next execution */
*offp = off;
}
@@ -2088,28 +2096,29 @@ slot_getmissingattrs(TupleTableSlot *slot, int startAttNum, int lastAttNum)
if (!attrmiss)
{
/* no missing values array at all, so just fill everything in as NULL */
- memset(slot->tts_values + startAttNum, 0,
- (lastAttNum - startAttNum) * sizeof(Datum));
- memset(slot->tts_isnull + startAttNum, 1,
- (lastAttNum - startAttNum) * sizeof(bool));
+ for (int attnum = startAttNum; attnum < lastAttNum; attnum++)
+ {
+ slot->tts_values[attnum] = (Datum) 0;
+ slot->tts_isnull[attnum] = true;
+ }
}
else
{
- int missattnum;
-
- /* if there is a missing values array we must process them one by one */
- for (missattnum = startAttNum;
- missattnum < lastAttNum;
- missattnum++)
+ /* use attrmiss to set the missing values */
+ for (int attnum = startAttNum; attnum < lastAttNum; attnum++)
{
- slot->tts_values[missattnum] = attrmiss[missattnum].am_value;
- slot->tts_isnull[missattnum] = !attrmiss[missattnum].am_present;
+ slot->tts_values[attnum] = attrmiss[attnum].am_value;
+ slot->tts_isnull[attnum] = !attrmiss[attnum].am_present;
}
}
+
+ if (unlikely(lastAttNum > slot->tts_tupleDescriptor->natts))
+ elog(ERROR, "invalid attribute number %d", lastAttNum);
}
/*
- * slot_getsomeattrs_int - workhorse for slot_getsomeattrs()
+ * slot_getsomeattrs_int
+ * external function to call getsomeattrs() for use in JIT
*/
void
slot_getsomeattrs_int(TupleTableSlot *slot, int attnum)
@@ -2118,21 +2127,13 @@ slot_getsomeattrs_int(TupleTableSlot *slot, int attnum)
Assert(slot->tts_nvalid < attnum); /* checked in slot_getsomeattrs */
Assert(attnum > 0);
- if (unlikely(attnum > slot->tts_tupleDescriptor->natts))
- elog(ERROR, "invalid attribute number %d", attnum);
-
/* Fetch as many attributes as possible from the underlying tuple. */
slot->tts_ops->getsomeattrs(slot, attnum);
/*
- * If the underlying tuple doesn't have enough attributes, tuple
- * descriptor must have the missing attributes.
+ * Avoid putting new code here as that would prevent the compiler from
+ * using the sibling call optimization for the above function.
*/
- if (unlikely(slot->tts_nvalid < attnum))
- {
- slot_getmissingattrs(slot, slot->tts_nvalid, attnum);
- slot->tts_nvalid = attnum;
- }
}
/* ----------------------------------------------------------------
diff --git a/src/include/executor/tuptable.h b/src/include/executor/tuptable.h
index 8346be77302..1922c912089 100644
--- a/src/include/executor/tuptable.h
+++ b/src/include/executor/tuptable.h
@@ -153,8 +153,8 @@ struct TupleTableSlotOps
* Fill up first natts entries of tts_values and tts_isnull arrays with
* values from the tuple contained in the slot. The function may be called
* with natts more than the number of attributes available in the tuple,
- * in which case it should set tts_nvalid to the number of returned
- * columns.
+ * in which case the function must call slot_getmissingattrs() to populate
+ * the remaining attributes.
*/
void (*getsomeattrs) (TupleTableSlot *slot, int natts);
@@ -357,8 +357,9 @@ extern void slot_getsomeattrs_int(TupleTableSlot *slot, int attnum);
static inline void
slot_getsomeattrs(TupleTableSlot *slot, int attnum)
{
+ /* Populate slot with attributes up to 'attnum', if it's not already */
if (slot->tts_nvalid < attnum)
- slot_getsomeattrs_int(slot, attnum);
+ slot->tts_ops->getsomeattrs(slot, attnum);
}
/*
--
2.51.0
[text/plain] v9-0005-Reduce-size-of-CompactAttribute-struct-to-8-bytes.patch (5.3K, 6-v9-0005-Reduce-size-of-CompactAttribute-struct-to-8-bytes.patch)
download | inline diff:
From db8daebf6a46aef82a0f8de70344165d3c171fb4 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Mon, 23 Feb 2026 09:39:37 +1300
Subject: [PATCH v9 5/5] Reduce size of CompactAttribute struct to 8 bytes
Previously, this was 16 bytes. With the use of some bitflags and by
reducing the attcacheoff field size to a 16-bit type, we can halve the
size of the struct.
It's unlikely that caching the offsets for offsets larger than what will
fit in a 16-bit int will help much as the tuple is very likely to have
some non-fixed-width types anyway, the offsets of which we cannot cache.
---
src/backend/access/common/tupdesc.c | 9 +++++++++
src/backend/executor/execTuples.c | 16 ++++++++++++----
src/include/access/tupdesc.h | 16 ++++++++--------
3 files changed, 29 insertions(+), 12 deletions(-)
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 028fcd53734..590dd8ec331 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -534,6 +534,15 @@ TupleDescFinalize(TupleDesc tupdesc)
cattr->attcacheoff = off;
+ /*
+ * attcacheoff is an int16, so don't try and cache any offsets larger
+ * than will fit in that type.
+ */
+ if (off > PG_INT16_MAX)
+ break;
+
+ cattr->attcacheoff = off;
+
off += cattr->attlen;
firstNonCachedOffsetAttr = i + 1;
}
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 2070c665d2f..b3699b77649 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -1013,6 +1013,7 @@ static pg_attribute_always_inline void
slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int reqnatts)
{
+ CompactAttribute *cattrs;
CompactAttribute *cattr;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
HeapTupleHeader tup = tuple->t_data;
@@ -1101,6 +1102,13 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
values = slot->tts_values;
slot->tts_nvalid = reqnatts;
+ /*
+ * We store the tupleDesc's CompactAttribute array in 'cattrs' as gcc
+ * seems to be unwilling to optimize accessing the CompactAttribute
+ * element efficiently when accessing it via TupleDescCompactAttr().
+ */
+ cattrs = tupleDesc->compact_attrs;
+
/* Ensure we calculated tp correctly */
Assert(tp == (char *) tup + tup->t_hoff);
@@ -1111,7 +1119,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int attlen;
isnull[attnum] = false;
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
attlen = cattr->attlen;
/* We don't expect any non-byval types */
@@ -1156,7 +1164,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
do
{
isnull[attnum] = false;
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
off = cattr->attcacheoff;
values[attnum] = fetch_att_noerr(tp + off,
@@ -1183,7 +1191,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int attlen;
isnull[attnum] = false;
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
attlen = cattr->attlen;
/*
@@ -1216,7 +1224,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
continue;
}
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
attlen = cattr->attlen;
/* As above, we don't expect cstrings */
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index d4c3a749558..5e54b82c296 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -55,7 +55,7 @@ typedef struct TupleConstr
* directly after the FormData_pg_attribute struct is populated or
* altered in any way.
*
- * Currently, this struct is 16 bytes. Any code changes which enlarge this
+ * Currently, this struct is 8 bytes. Any code changes which enlarge this
* struct should be considered very carefully.
*
* Code which must access a TupleDesc's attribute data should always make use
@@ -67,17 +67,17 @@ typedef struct TupleConstr
*/
typedef struct CompactAttribute
{
- int32 attcacheoff; /* fixed offset into tuple, if known, or -1 */
+ int16 attcacheoff; /* fixed offset into tuple, if known, or -1 */
int16 attlen; /* attr len in bytes or -1 = varlen, -2 =
* cstring */
bool attbyval; /* as FormData_pg_attribute.attbyval */
- bool attispackable; /* FormData_pg_attribute.attstorage !=
- * TYPSTORAGE_PLAIN */
- bool atthasmissing; /* as FormData_pg_attribute.atthasmissing */
- bool attisdropped; /* as FormData_pg_attribute.attisdropped */
- bool attgenerated; /* FormData_pg_attribute.attgenerated != '\0' */
- char attnullability; /* status of not-null constraint, see below */
uint8 attalignby; /* alignment requirement in bytes */
+ bool attispackable:1; /* FormData_pg_attribute.attstorage !=
+ * TYPSTORAGE_PLAIN */
+ bool atthasmissing:1; /* as FormData_pg_attribute.atthasmissing */
+ bool attisdropped:1; /* as FormData_pg_attribute.attisdropped */
+ bool attgenerated:1; /* FormData_pg_attribute.attgenerated != '\0' */
+ char attnullability; /* status of not-null constraint, see below */
} CompactAttribute;
/* Valid values for CompactAttribute->attnullability */
--
2.51.0
[text/csv] deform_results_v9.csv (25.7K, 7-deform_results_v9.csv)
download | inline:
amd3990x gcc,,,,,,,,,,,,,,,,,,
branch,test_id,extra_columns,avg_ms,stddev,,branch,test_id,extra_columns,avg_ms,stddev,,branch,test_id,extra_columns,avg_ms,stddev,,0004 vs 0005
master,1,0,26.38,0.946317278471701,,v9-0001-0004,1,0,22.95,1.01254019233266,,v9-0001-0005,1,0,24.42,1.15770914119802,,106.41%
master,2,0,27.46,0.962607682372061,,v9-0001-0004,2,0,24.81,1.11498690372496,,v9-0001-0005,2,0,25.37,0.949288461246058,,102.26%
master,3,0,28.16,0.874407253873551,,v9-0001-0004,3,0,26.61,1.0844714514911,,v9-0001-0005,3,0,26.94,1.23215039249768,,101.24%
master,4,0,28.16,0.920472838974753,,v9-0001-0004,4,0,26.73,1.11572151170829,,v9-0001-0005,4,0,26.97,1.12984892181276,,100.90%
master,5,0,29.23,1.05317250068246,,v9-0001-0004,5,0,23.57,0.956481782320206,,v9-0001-0005,5,0,24.89,1.0600533309864,,105.60%
master,6,0,30.04,0.970902150932396,,v9-0001-0004,6,0,27.99,1.0610685315294,,v9-0001-0005,6,0,28.81,1.12950611806855,,102.93%
master,7,0,27.32,0.827410655505258,,v9-0001-0004,7,0,26.17,0.724231822710408,,v9-0001-0005,7,0,26.41,1.05086034156783,,100.92%
master,8,0,27.19,0.773807310909803,,v9-0001-0004,8,0,26.18,0.734205032636779,,v9-0001-0005,8,0,26.43,0.938978647202271,,100.95%
master,1,10,46.52,1.52384626826671,,v9-0001-0004,1,10,38.46,1.97881944036803,,v9-0001-0005,1,10,39.25,2.02414223422785,,102.05%
master,2,10,48.94,1.89683668457345,,v9-0001-0004,2,10,45.4,1.9941903305029,,v9-0001-0005,2,10,46.28,2.03897490278123,,101.94%
master,3,10,65.06,3.45311633628704,,v9-0001-0004,3,10,56.02,2.07839877369679,,v9-0001-0005,3,10,55.29,2.17988487708601,,98.70%
master,4,10,66.24,2.59583465494898,,v9-0001-0004,4,10,56.03,2.04341374892415,,v9-0001-0005,4,10,55.67,2.10155993379674,,99.36%
master,5,10,66.93,3.04084832560644,,v9-0001-0004,5,10,43.19,2.04375224202236,,v9-0001-0005,5,10,47.25,1.86179222589273,,109.40%
master,6,10,70.64,3.20513994015903,,v9-0001-0004,6,10,60.35,2.01272091901355,,v9-0001-0005,6,10,61.16,2.37229042987981,,101.34%
master,7,10,56.19,1.88472675708813,,v9-0001-0004,7,10,49.69,1.77765551900937,,v9-0001-0005,7,10,49.51,2.20594635581326,,99.64%
master,8,10,56.35,1.75701047437666,,v9-0001-0004,8,10,49.7,1.88264982988009,,v9-0001-0005,8,10,49.62,2.22874149999161,,99.84%
master,1,20,77.33,2.79382026057385,,v9-0001-0004,1,20,67.29,2.61907908235788,,v9-0001-0005,1,20,65.16,2.85819979848355,,96.83%
master,2,20,78.03,2.65837310977883,,v9-0001-0004,2,20,76.47,3.39150776422804,,v9-0001-0005,2,20,83.96,5.08224424370883,,109.79%
master,3,20,101.58,3.81080599686697,,v9-0001-0004,3,20,84.71,3.22566322441146,,v9-0001-0005,3,20,84.29,3.88227232083066,,99.50%
master,4,20,101.42,2.95422425507498,,v9-0001-0004,4,20,84.52,2.40601090379074,,v9-0001-0005,4,20,84.29,3.7868779764136,,99.73%
master,5,20,96.36,3.80488812913808,,v9-0001-0004,5,20,72.16,2.9530884867158,,v9-0001-0005,5,20,71.8,2.75814375628848,,99.50%
master,6,20,102.08,4.67714692927063,,v9-0001-0004,6,20,85.63,3.04468248570984,,v9-0001-0005,6,20,84.76,4.67652069354098,,98.98%
master,7,20,99.7,3.96315204531553,,v9-0001-0004,7,20,83.12,2.11943761342848,,v9-0001-0005,7,20,85.19,3.85961963969189,,102.49%
master,8,20,98.83,3.49527643688871,,v9-0001-0004,8,20,82.37,4.12755084137939,,v9-0001-0005,8,20,85.95,3.8098752330834,,104.35%
master,1,30,115.51,3.12872784909124,,v9-0001-0004,1,30,92.73,2.50165366786477,,v9-0001-0005,1,30,91.48,2.49708473954606,,98.65%
master,2,30,117.55,3.18906039360697,,v9-0001-0004,2,30,112.85,2.87334722995031,,v9-0001-0005,2,30,118.38,3.00988048582645,,104.90%
master,3,30,138.31,6.65532037908513,,v9-0001-0004,3,30,121.69,3.25943918677012,,v9-0001-0005,3,30,122.1,3.11842602453165,,100.34%
master,4,30,138.33,6.62762747393962,,v9-0001-0004,4,30,119.52,3.24089468710302,,v9-0001-0005,4,30,122.17,3.18436467392523,,102.22%
master,5,30,139.41,7.59055743670967,,v9-0001-0004,5,30,93.37,2.65864691962686,,v9-0001-0005,5,30,93.77,2.99946303132071,,100.43%
master,6,30,136.58,5.6360146169056,,v9-0001-0004,6,30,124.23,4.2409151936336,,v9-0001-0005,6,30,126.12,3.06922111964438,,101.52%
master,7,30,140.41,8.44170814338531,,v9-0001-0004,7,30,122.85,2.27530773636863,,v9-0001-0005,7,30,124.84,3.46921348188014,,101.62%
master,8,30,138.59,7.14470633970422,,v9-0001-0004,8,30,122.36,2.40729522804363,,v9-0001-0005,8,30,124.75,3.39853321220385,,101.95%
master,1,40,136.63,3.08603292350244,,v9-0001-0004,1,40,108.13,3.73920931986643,,v9-0001-0005,1,40,107.88,3.60627753741342,,99.77%
master,2,40,142.77,3.36854802262854,,v9-0001-0004,2,40,149.74,3.39671249243142,,v9-0001-0005,2,40,143.45,3.96006788246075,,95.80%
master,3,40,168.41,2.19101685919139,,v9-0001-0004,3,40,141.7,3.72144458178272,,v9-0001-0005,3,40,142.61,2.8402023688235,,100.64%
master,4,40,168.26,2.56270271353712,,v9-0001-0004,4,40,142.88,2.99366762713737,,v9-0001-0005,4,40,142.91,3.18784928845543,,100.02%
master,5,40,171.08,1.9322942240684,,v9-0001-0004,5,40,109.87,3.36169167679084,,v9-0001-0005,5,40,111.74,4.13348369083801,,101.70%
master,6,40,168.14,2.70908047693678,,v9-0001-0004,6,40,142.51,2.82135216762058,,v9-0001-0005,6,40,142.92,3.72154930897298,,100.29%
master,7,40,166.24,3.41910692444872,,v9-0001-0004,7,40,157.16,2.5134752782676,,v9-0001-0005,7,40,148.53,3.71817627394301,,94.51%
master,8,40,167.25,3.38498031726905,,v9-0001-0004,8,40,156.01,3.5569598859386,,v9-0001-0005,8,40,149.39,2.82339365943524,,95.76%
,,,,,,,,,,,,,,,,,,
amd3990x clang,,,,,,,,,,,,,,,,,,
branch,test_id,extra_columns,avg_ms,stddev,,branch,test_id,extra_columns,avg_ms,stddev,,branch,test_id,extra_columns,avg_ms,stddev,,0004 vs 0005
master,1,0,33.23,1.15378281955897,,v9-0001-0004,1,0,25.89,1.24568301120552,,v9-0001-0005,1,0,25.62,1.09371751484013,,98.96%
master,2,0,35.71,0.98817164402953,,v9-0001-0004,2,0,30.71,1.13441858230101,,v9-0001-0005,2,0,29.66,1.11655570360439,,96.58%
master,3,0,34.98,0.98954542350486,,v9-0001-0004,3,0,30.36,1.18434865429361,,v9-0001-0005,3,0,31.16,1.36944186721736,,102.64%
master,4,0,34.99,1.00214150292942,,v9-0001-0004,4,0,30.45,1.21414912058289,,v9-0001-0005,4,0,31.6,1.38781725501562,,103.78%
master,5,0,34.12,0.998631179629829,,v9-0001-0004,5,0,26.76,1.13688680394824,,v9-0001-0005,5,0,26.66,1.07038805072255,,99.63%
master,6,0,37.23,1.04360375414345,,v9-0001-0004,6,0,33.04,1.11917357416053,,v9-0001-0005,6,0,32.65,1.05615053824211,,98.82%
master,7,0,34.6,0.78617869463983,,v9-0001-0004,7,0,30.17,0.794660887672213,,v9-0001-0005,7,0,30.41,1.03963518824562,,100.80%
master,8,0,34.36,0.835537086414169,,v9-0001-0004,8,0,30.14,0.788085007840768,,v9-0001-0005,8,0,30.64,1.16205919788224,,101.66%
master,1,10,63.86,1.78648409857556,,v9-0001-0004,1,10,41.81,1.61444134861999,,v9-0001-0005,1,10,41.13,1.84910640340971,,98.37%
master,2,10,66.24,2.03738109312779,,v9-0001-0004,2,10,50.91,1.89742002450441,,v9-0001-0005,2,10,49.33,2.06320760369845,,96.90%
master,3,10,75.45,2.63740816058162,,v9-0001-0004,3,10,52.05,2.13047563287479,,v9-0001-0005,3,10,67.38,1.74360548975292,,129.45%
master,4,10,76.26,2.36060644983628,,v9-0001-0004,4,10,52.08,2.13347471457218,,v9-0001-0005,4,10,66.94,1.90911405658176,,128.53%
master,5,10,74.22,2.94660907446393,,v9-0001-0004,5,10,47.91,2.15291199906207,,v9-0001-0005,5,10,52.1,2.38435334198739,,108.75%
master,6,10,83.61,1.96054014334828,,v9-0001-0004,6,10,55.89,2.04555398360647,,v9-0001-0005,6,10,60.57,2.34542381216232,,108.37%
master,7,10,66.95,1.84715234885692,,v9-0001-0004,7,10,51.53,1.62539012724123,,v9-0001-0005,7,10,52.44,1.96187242494181,,101.77%
master,8,10,67.02,1.95160108394108,,v9-0001-0004,8,10,51.61,1.60199492766036,,v9-0001-0005,8,10,52.53,1.94306392493594,,101.78%
master,1,20,104.52,3.25040233897318,,v9-0001-0004,1,20,70.25,3.37839625357999,,v9-0001-0005,1,20,67.75,1.86612795724253,,96.44%
master,2,20,110.07,3.46642158551138,,v9-0001-0004,2,20,85.55,3.19406461427093,,v9-0001-0005,2,20,82.08,2.75072949837551,,95.94%
master,3,20,129.04,4.30780357935643,,v9-0001-0004,3,20,85.09,2.85911912824489,,v9-0001-0005,3,20,86.08,3.01712870219814,,101.16%
master,4,20,129.61,3.85565503205434,,v9-0001-0004,4,20,85.51,2.84659592889443,,v9-0001-0005,4,20,85.11,3.31671003237554,,99.53%
master,5,20,115.9,8.48765542914855,,v9-0001-0004,5,20,73.82,3.35529370560727,,v9-0001-0005,5,20,71.11,3.7974690801151,,96.33%
master,6,20,129.68,3.19093231034027,,v9-0001-0004,6,20,89.9,3.29873549277434,,v9-0001-0005,6,20,84.38,4.52886069265003,,93.86%
master,7,20,120.77,6.53704506955606,,v9-0001-0004,7,20,85.82,3.96110691064917,,v9-0001-0005,7,20,84.31,4.57695873957363,,98.24%
master,8,20,119.5,6.66580455837972,,v9-0001-0004,8,20,85.39,2.91260415605282,,v9-0001-0005,8,20,87.19,3.98543346187924,,102.11%
master,1,30,146.79,3.58183674200152,,v9-0001-0004,1,30,91.63,2.64001075264408,,v9-0001-0005,1,30,93.71,3.14440856569093,,102.27%
master,2,30,150.38,3.24841388996625,,v9-0001-0004,2,30,105.55,3.09353771439775,,v9-0001-0005,2,30,113.12,3.43090430235263,,107.17%
master,3,30,162.65,2.38795005304703,,v9-0001-0004,3,30,111.43,1.7714299906626,,v9-0001-0005,3,30,117.48,3.21545946475555,,105.43%
master,4,30,157.76,7.97241189025478,,v9-0001-0004,4,30,111.03,2.97435018088143,,v9-0001-0005,4,30,117.07,1.66634301783109,,105.44%
master,5,30,160.56,10.9197836659985,,v9-0001-0004,5,30,94.6,2.49789643700445,,v9-0001-0005,5,30,95.96,3.42693509815196,,101.44%
master,6,30,160.55,2.80582085752507,,v9-0001-0004,6,30,123.54,2.21394219291831,,v9-0001-0005,6,30,121.16,4.18099356636743,,98.07%
master,7,30,153.64,2.35806846097994,,v9-0001-0004,7,30,106.13,3.06328356084187,,v9-0001-0005,7,30,119.27,2.9676653179617,,112.38%
master,8,30,152.57,1.86241645731867,,v9-0001-0004,8,30,105.85,3.08516470116208,,v9-0001-0005,8,30,120.72,2.47356971345911,,114.05%
master,1,40,183,3.29670723537347,,v9-0001-0004,1,40,110.24,4.36865581674346,,v9-0001-0005,1,40,114.18,2.90297430945888,,103.57%
master,2,40,187.63,4.36055772824419,,v9-0001-0004,2,40,143.6,4.17528868856475,,v9-0001-0005,2,40,136.33,5.99584731012183,,94.94%
master,3,40,193.64,5.19229879395822,,v9-0001-0004,3,40,130.41,3.59090798676275,,v9-0001-0005,3,40,138.45,3.18878103134302,,106.17%
master,4,40,193,5.56498607216762,,v9-0001-0004,4,40,131.55,4.7722392619422,,v9-0001-0005,4,40,138.67,2.98081165268001,,105.41%
master,5,40,186.3,10.8093754592099,,v9-0001-0004,5,40,111.36,3.4667119090846,,v9-0001-0005,5,40,114.84,2.86063441671146,,103.13%
master,6,40,190.63,3.73658755206839,,v9-0001-0004,6,40,138.76,5.24986885332138,,v9-0001-0005,6,40,138.48,2.73368237508955,,99.80%
master,7,40,191.7,3.34774883008989,,v9-0001-0004,7,40,143.66,4.56370981428579,,v9-0001-0005,7,40,149.57,4.06948914547464,,104.11%
master,8,40,192.61,2.81010616770404,,v9-0001-0004,8,40,142.14,6.11216701427283,,v9-0001-0005,8,40,148.94,4.09808413193939,,104.78%
,,,,,,,,,,,,,,,,,,
amd7945hx gcc,,,,,,,,,,,,,,,,,,
branch,test_id,extra_columns,avg_ms,stddev,,branch,test_id,extra_columns,avg_ms,stddev,,branch,test_id,extra_columns,avg_ms,stddev,,0004 vs 0005
master,1,0,16.22,0.934703972956798,,v9-0001-0004,1,0,14.02,0.90719782913524,,v9-0001-0005,1,0,14.75,0.787824719215584,,105.21%
master,2,0,16.2,0.769879038879482,,v9-0001-0004,2,0,15.64,0.718560991928661,,v9-0001-0005,2,0,16.44,0.711001692319881,,105.12%
master,3,0,16.29,0.721728098996736,,v9-0001-0004,3,0,17.54,0.948153285170856,,v9-0001-0005,3,0,18.46,0.957999996252194,,105.25%
master,4,0,17.47,0.881628161047112,,v9-0001-0004,4,0,17.35,0.922180595314137,,v9-0001-0005,4,0,17.73,0.611498468156277,,102.19%
master,5,0,16.58,0.951359351135622,,v9-0001-0004,5,0,14.47,1.0080925699917,,v9-0001-0005,5,0,15.38,1.06512110900743,,106.29%
master,6,0,18.77,0.907591966234102,,v9-0001-0004,6,0,19.51,0.741374840493426,,v9-0001-0005,6,0,18.9,1.03441457524483,,96.87%
master,7,0,15.67,0.641756055111027,,v9-0001-0004,7,0,17.32,0.754999792052224,,v9-0001-0005,7,0,17.16,0.813292949524169,,99.08%
master,8,0,16.01,0.676662160660782,,v9-0001-0004,8,0,17.45,0.635709960365472,,v9-0001-0005,8,0,17.14,0.750189621725823,,98.22%
master,1,10,27.83,1.58461890458174,,v9-0001-0004,1,10,23.41,1.55297621077735,,v9-0001-0005,1,10,25.14,1.48053733018298,,107.39%
master,2,10,28.21,1.63205033571433,,v9-0001-0004,2,10,29.46,1.70879090871579,,v9-0001-0005,2,10,30.91,1.84396868065102,,104.92%
master,3,10,34.48,1.95439026319707,,v9-0001-0004,3,10,32.72,1.85913371935077,,v9-0001-0005,3,10,34.41,1.78490089514427,,105.17%
master,4,10,31.86,1.82153143387714,,v9-0001-0004,4,10,32.67,1.88023383440683,,v9-0001-0005,4,10,32.73,1.78025245209716,,100.18%
master,5,10,34.51,1.78791449424814,,v9-0001-0004,5,10,24.87,1.90022746534559,,v9-0001-0005,5,10,25.82,1.7344463445763,,103.82%
master,6,10,33.04,1.77825567970686,,v9-0001-0004,6,10,33.1,1.89054480648096,,v9-0001-0005,6,10,34.93,2.05298511548224,,105.53%
master,7,10,31.95,1.62782043021697,,v9-0001-0004,7,10,30.56,1.44508809181964,,v9-0001-0005,7,10,34.27,1.89893413416505,,112.14%
master,8,10,32.01,1.69317391514311,,v9-0001-0004,8,10,30.64,1.55235301472806,,v9-0001-0005,8,10,34.11,1.86300340228238,,111.33%
master,1,20,41.04,2.36591953046371,,v9-0001-0004,1,20,32.74,2.29794060984905,,v9-0001-0005,1,20,33.87,2.31937496743699,,103.45%
master,2,20,44.53,2.60943599636804,,v9-0001-0004,2,20,43.84,2.4002223202266,,v9-0001-0005,2,20,45.6,2.19821772273009,,104.01%
master,3,20,51.51,2.72067536888197,,v9-0001-0004,3,20,48.36,2.55873792831095,,v9-0001-0005,3,20,50.63,2.33451947387413,,104.69%
master,4,20,50,4.01246566580572,,v9-0001-0004,4,20,46.26,2.75136044117219,,v9-0001-0005,4,20,51.81,2.63142322457972,,112.00%
master,5,20,56.58,3.33970008246636,,v9-0001-0004,5,20,33.46,2.42273727886,,v9-0001-0005,5,20,36.12,2.79970847422005,,107.95%
master,6,20,52.6,4.02037222898871,,v9-0001-0004,6,20,49.67,2.36985899331677,,v9-0001-0005,6,20,51.36,2.63544778221671,,103.40%
master,7,20,48.73,4.0931085152649,,v9-0001-0004,7,20,47.84,2.19882191621266,,v9-0001-0005,7,20,47.31,2.40187554210949,,98.89%
master,8,20,50.75,2.65310543887458,,v9-0001-0004,8,20,48.72,2.301951598094,,v9-0001-0005,8,20,51.35,2.34788214341792,,105.40%
master,1,30,58.23,2.79599826219327,,v9-0001-0004,1,30,46.45,2.19761248332913,,v9-0001-0005,1,30,50.29,2.09327716792791,,108.27%
master,2,30,58.09,2.07572803109201,,v9-0001-0004,2,30,66.86,2.64616643511885,,v9-0001-0005,2,30,69.96,4.34805778567684,,104.64%
master,3,30,76.84,2.73873407264624,,v9-0001-0004,3,30,66.75,2.31161069674014,,v9-0001-0005,3,30,73.2,3.08213254219932,,109.66%
master,4,30,82.84,2.62503992264209,,v9-0001-0004,4,30,66.97,3.90611708040358,,v9-0001-0005,4,30,67.22,2.48183671931114,,100.37%
master,5,30,85.08,3.28260115010679,,v9-0001-0004,5,30,51.23,1.80886322925297,,v9-0001-0005,5,30,49.61,2.41660520743414,,96.84%
master,6,30,83.4,2.78893547832034,,v9-0001-0004,6,30,76.53,2.78678816185458,,v9-0001-0005,6,30,76.58,2.04154893311111,,100.07%
master,7,30,72.24,4.53391240977746,,v9-0001-0004,7,30,68.3,2.29433703111237,,v9-0001-0005,7,30,72.09,1.25522958152123,,105.55%
master,8,30,74.55,4.92937915966494,,v9-0001-0004,8,30,69.47,2.0922200056741,,v9-0001-0005,8,30,75.67,2.15513994350255,,108.92%
master,1,40,81.72,2.2348382953491,,v9-0001-0004,1,40,57.82,2.05044978945637,,v9-0001-0005,1,40,59.96,2.63360752188109,,103.70%
master,2,40,91.9,4.1253516634732,,v9-0001-0004,2,40,87.38,4.01803117857276,,v9-0001-0005,2,40,84.94,2.89775991192866,,97.21%
master,3,40,91.76,2.37978053296315,,v9-0001-0004,3,40,88.35,2.49762999315813,,v9-0001-0005,3,40,90.71,2.74422858815616,,102.67%
master,4,40,98.15,2.75758997705264,,v9-0001-0004,4,40,89.95,2.67351993693024,,v9-0001-0005,4,40,90.39,2.87713356945607,,100.49%
master,5,40,95.95,2.56320966840298,,v9-0001-0004,5,40,55.99,2.55618640098219,,v9-0001-0005,5,40,60.37,2.81557858888567,,107.82%
master,6,40,99.19,2.86712528158762,,v9-0001-0004,6,40,89.7,2.93078963684158,,v9-0001-0005,6,40,94.42,2.42779134634778,,105.26%
master,7,40,92.34,2.34398602449254,,v9-0001-0004,7,40,94.42,3.00090660438185,,v9-0001-0005,7,40,90.78,2.41701405523677,,96.14%
master,8,40,93.92,2.74468685994991,,v9-0001-0004,8,40,96.9,3.03830786441204,,v9-0001-0005,8,40,90.61,2.22103836278569,,93.51%
,,,,,,,,,,,,,,,,,,
amd7945hx clang,,,,,,,,,,,,,,,,,,
branch,test_id,extra_columns,avg_ms,stddev,,branch,test_id,extra_columns,avg_ms,stddev,,branch,test_id,extra_columns,avg_ms,stddev,,
master,1,0,17.57,0.517599848202015,,v9-0001-0004,1,0,14.58,0.681105489885279,,v9-0001-0005,1,0,14.27,0.887196695940377,,
master,2,0,19.34,0.949601843626375,,v9-0001-0004,2,0,17.07,0.763391281860839,,v9-0001-0005,2,0,18.13,0.899183421334613,,
master,3,0,17.72,0.615349528782653,,v9-0001-0004,3,0,19.23,1.67425697646681,,v9-0001-0005,3,0,18.94,0.69496127759868,,
master,4,0,18.16,0.724468308937089,,v9-0001-0004,4,0,19.39,2.79602896506638,,v9-0001-0005,4,0,19.2,0.997666214365339,,
master,5,0,17.23,0.597429065588238,,v9-0001-0004,5,0,15.06,0.831088161539079,,v9-0001-0005,5,0,16.6,1.38821277506246,,
master,6,0,18.95,0.668502813349074,,v9-0001-0004,6,0,19.48,0.621328507032421,,v9-0001-0005,6,0,19.67,0.825038471249763,,
master,7,0,17.98,0.773797899323962,,v9-0001-0004,7,0,19.5,0.538534506441429,,v9-0001-0005,7,0,17.42,0.771518164950746,,
master,8,0,17.45,0.572540059559517,,v9-0001-0004,8,0,17.89,0.587815694457329,,v9-0001-0005,8,0,17.6,0.634782954521909,,
master,1,10,33.45,1.29230052545548,,v9-0001-0004,1,10,23.26,1.33217871123082,,v9-0001-0005,1,10,24.2,2.10825592866149,,
master,2,10,38.24,1.28215093573649,,v9-0001-0004,2,10,28.73,1.21217058795287,,v9-0001-0005,2,10,30.78,1.62504723338276,,
master,3,10,40.37,1.71966814066814,,v9-0001-0004,3,10,31.74,1.44106225672749,,v9-0001-0005,3,10,34.29,2.30903701516967,,
master,4,10,37.98,1.38654823307379,,v9-0001-0004,4,10,32.09,1.78747637418055,,v9-0001-0005,4,10,33.28,1.75765379259534,,
master,5,10,33.98,1.35055560224457,,v9-0001-0004,5,10,24.03,1.54152549233114,,v9-0001-0005,5,10,24.29,1.96217345472491,,
master,6,10,38.62,1.33200158967078,,v9-0001-0004,6,10,33.59,1.43736684450194,,v9-0001-0005,6,10,34.63,1.89379465228116,,
master,7,10,37.45,1.11191084360471,,v9-0001-0004,7,10,31.4,1.33798920365116,,v9-0001-0005,7,10,31.03,1.6164720956554,,
master,8,10,37.41,1.1516352414251,,v9-0001-0004,8,10,31.19,1.38703679686967,,v9-0001-0005,8,10,33.26,1.56599145210531,,
master,1,20,48.51,2.7471609846647,,v9-0001-0004,1,20,31.68,1.864978591978,,v9-0001-0005,1,20,34.58,2.36879640752413,,
master,2,20,57.31,1.78748922197336,,v9-0001-0004,2,20,45.02,1.74835567347482,,v9-0001-0005,2,20,44.76,2.56032635177198,,
master,3,20,57.44,1.67560418590827,,v9-0001-0004,3,20,46.77,1.88962592050486,,v9-0001-0005,3,20,49.19,2.82451090878125,,
master,4,20,57.5,1.92198496932962,,v9-0001-0004,4,20,49.72,2.12322839700701,,v9-0001-0005,4,20,48.96,2.71457527420255,,
master,5,20,51.21,1.55748253895893,,v9-0001-0004,5,20,34.34,2.217721216767,,v9-0001-0005,5,20,36.99,2.64723066928196,,
master,6,20,58.52,1.80142983514857,,v9-0001-0004,6,20,46.31,2.07703531691954,,v9-0001-0005,6,20,51.45,2.91087115906043,,
master,7,20,56.66,1.66571609571276,,v9-0001-0004,7,20,49.72,2.26894217613076,,v9-0001-0005,7,20,49.24,2.39549459610851,,
master,8,20,56.11,1.72064626636195,,v9-0001-0004,8,20,46.78,1.83378816603887,,v9-0001-0005,8,20,47.87,2.23968589431011,,
master,1,30,71.46,1.67335691382039,,v9-0001-0004,1,30,46.45,1.87561183249883,,v9-0001-0005,1,30,48.69,3.16613165464413,,
master,2,30,78.5,1.59876091682031,,v9-0001-0004,2,30,65.89,2.16434198319616,,v9-0001-0005,2,30,65.61,4.08447912745675,,
master,3,30,84.06,2.37386837301948,,v9-0001-0004,3,30,65.6,1.83149347070611,,v9-0001-0005,3,30,70.45,2.66325368158459,,
master,4,30,89.91,2.38140650602552,,v9-0001-0004,4,30,66.73,1.61972906889793,,v9-0001-0005,4,30,66.64,3.54839256760591,,
master,5,30,79.54,1.80818072030604,,v9-0001-0004,5,30,47.87,1.79710660499725,,v9-0001-0005,5,30,55.21,2.48745838245475,,
master,6,30,87.76,1.93188125802343,,v9-0001-0004,6,30,65.23,2.48233732613471,,v9-0001-0005,6,30,68.9,3.33784523556887,,
master,7,30,85.08,1.61970071755296,,v9-0001-0004,7,30,62.18,2.51574376842314,,v9-0001-0005,7,30,69.48,2.70907215187956,,
master,8,30,82.33,1.74931766669544,,v9-0001-0004,8,30,62.25,2.51811701991961,,v9-0001-0005,8,30,70.95,2.59054202047016,,
master,1,40,93.27,1.87359480560949,,v9-0001-0004,1,40,59.08,1.93703296260374,,v9-0001-0005,1,40,61.18,3.070880772436,,
master,2,40,112.09,5.94610528043952,,v9-0001-0004,2,40,89.27,8.09981004307861,,v9-0001-0005,2,40,85.39,3.45085522401948,,
master,3,40,106.14,2.0498729807112,,v9-0001-0004,3,40,83.05,3.71602656574676,,v9-0001-0005,3,40,84.84,2.8502645075293,,
master,4,40,106.77,2.62106064077092,,v9-0001-0004,4,40,80.65,1.92644584485106,,v9-0001-0005,4,40,96.27,8.43686257029907,,
master,5,40,99.25,2.43071022734983,,v9-0001-0004,5,40,57.84,2.04088362166694,,v9-0001-0005,5,40,62.54,6.20459328435797,,
master,6,40,109.07,1.90711449392452,,v9-0001-0004,6,40,87.33,1.97791981079275,,v9-0001-0005,6,40,86.08,2.74340271114625,,
master,7,40,108.81,1.71798477132794,,v9-0001-0004,7,40,85.56,2.69871932809969,,v9-0001-0005,7,40,85.17,2.38052148011951,,
master,8,40,109.01,1.65016461703773,,v9-0001-0004,8,40,84.55,2.87508045458298,,v9-0001-0005,8,40,89.71,3.15620007934651,,
,,,,,,,,,,,,,,,,,,
m2 clang,,,,,,,,,,,,,,,,,,
branch,test_id,extra_columns,avg_ms,stddev,,branch,test_id,extra_columns,avg_ms,stddev,,branch,test_id,extra_columns,avg_ms,stddev,,0004 vs 0005
master,1,0,21.34,3.20106480047817,,v9-0001-0004,1,0,15.49,3.42334479390494,,v9-0001-0005,1,0,15.01,3.86037916386822,,96.90%
master,2,0,17.76,1.31212842264806,,v9-0001-0004,2,0,24.6,0.928434706810789,,v9-0001-0005,2,0,25.12,0.976904142814707,,102.11%
master,3,0,16.88,1.19948545742509,,v9-0001-0004,3,0,15.65,1.19114942285051,,v9-0001-0005,3,0,16.03,1.27850511271598,,102.43%
master,4,0,16.86,1.00806011386338,,v9-0001-0004,4,0,15.93,1.35754725017371,,v9-0001-0005,4,0,16.06,1.36658022866727,,100.82%
master,5,0,18.72,1.13434379179424,,v9-0001-0004,5,0,13.84,1.46427960961682,,v9-0001-0005,5,0,13.27,1.24484860333573,,95.88%
master,6,0,17.89,1.13001776691006,,v9-0001-0004,6,0,19.22,1.2815063114086,,v9-0001-0005,6,0,19.12,1.23608748920448,,99.48%
master,7,0,16.78,0.756122930461646,,v9-0001-0004,7,0,15.29,0.919640337195801,,v9-0001-0005,7,0,15.39,0.892204794172116,,100.65%
master,8,0,16.5,0.679380758391071,,v9-0001-0004,8,0,15.29,0.936867678923857,,v9-0001-0005,8,0,15.5,1.14738001691845,,101.37%
master,1,10,43.52,2.29988708636144,,v9-0001-0004,1,10,22.05,3.60238428508943,,v9-0001-0005,1,10,19.65,2.4676779001029,,89.12%
master,2,10,45.1,2.14627632346491,,v9-0001-0004,2,10,26.3,2.59546849947441,,v9-0001-0005,2,10,26.42,2.64613738870351,,100.46%
master,3,10,46.13,2.4269008032083,,v9-0001-0004,3,10,29.13,2.58307150940474,,v9-0001-0005,3,10,29.19,2.82511967552611,,100.21%
master,4,10,45.68,2.21605337788719,,v9-0001-0004,4,10,29.03,2.47920687351031,,v9-0001-0005,4,10,29.15,2.64723817724369,,100.41%
master,5,10,43.74,2.59793424311346,,v9-0001-0004,5,10,24.55,2.80429367871108,,v9-0001-0005,5,10,22.16,1.97990562181935,,90.26%
master,6,10,42.33,3.17655652789789,,v9-0001-0004,6,10,30.63,2.71527069850766,,v9-0001-0005,6,10,30.38,3.1274111953385,,99.18%
master,7,10,45.63,2.47648213328043,,v9-0001-0004,7,10,27.3,3.11033568178863,,v9-0001-0005,7,10,27.53,2.98988748234637,,100.84%
master,8,10,45.02,2.05564282277562,,v9-0001-0004,8,10,27.19,2.73787983428422,,v9-0001-0005,8,10,27.56,3.14015012386661,,101.36%
master,1,20,65.86,4.68398408872982,,v9-0001-0004,1,20,33.04,3.12521243695067,,v9-0001-0005,1,20,31.65,2.36809721818229,,95.79%
master,2,20,72.34,3.45318844682657,,v9-0001-0004,2,20,39.79,2.75822287344297,,v9-0001-0005,2,20,40.5,2.88999473363,,101.78%
master,3,20,71.98,3.87113108473948,,v9-0001-0004,3,20,40.75,4.70153404043622,,v9-0001-0005,3,20,40.13,4.62105009704158,,98.48%
master,4,20,72.47,5.543296911208,,v9-0001-0004,4,20,41.08,5.66899503837348,,v9-0001-0005,4,20,40.81,6.16047296119971,,99.34%
master,5,20,66.05,4.02246206321803,,v9-0001-0004,5,20,31.53,5.57323002631299,,v9-0001-0005,5,20,29.22,4.4627237465736,,92.67%
master,6,20,70.89,4.28319631950092,,v9-0001-0004,6,20,41.91,4.81947021011578,,v9-0001-0005,6,20,40.72,4.56823662262306,,97.16%
master,7,20,71,5.2039470188671,,v9-0001-0004,7,20,42.83,3.76689315535313,,v9-0001-0005,7,20,42.36,4.41743622710967,,98.90%
master,8,20,70.73,3.48986975565595,,v9-0001-0004,8,20,42.6,2.81074136960705,,v9-0001-0005,8,20,41.76,3.35761253332597,,98.03%
master,1,30,81.14,4.61258200180692,,v9-0001-0004,1,30,37.35,4.80043244593014,,v9-0001-0005,1,30,36.76,3.82172830685852,,98.42%
master,2,30,90.27,5.39912211587677,,v9-0001-0004,2,30,52.21,3.47453522553101,,v9-0001-0005,2,30,50.8,4.02119985499198,,97.30%
master,3,30,90.95,4.61608007337523,,v9-0001-0004,3,30,52.75,4.67872966685551,,v9-0001-0005,3,30,52.7,4.68680743508728,,99.91%
master,4,30,90.63,4.3110891384496,,v9-0001-0004,4,30,53.59,4.42289934418508,,v9-0001-0005,4,30,53.14,4.69436461792718,,99.16%
master,5,30,82.74,4.37497288748148,,v9-0001-0004,5,30,42.52,5.40595477298203,,v9-0001-0005,5,30,41.5,4.15188338431965,,97.60%
master,6,30,91.29,4.41019479015886,,v9-0001-0004,6,30,61.14,5.9913426988853,,v9-0001-0005,6,30,58.18,6.76509706043758,,95.16%
master,7,30,90.07,4.58486077552714,,v9-0001-0004,7,30,51.6,4.61446099317718,,v9-0001-0005,7,30,51.84,4.27573559764628,,100.47%
master,8,30,90.56,4.37858397670393,,v9-0001-0004,8,30,53.41,5.33383976570604,,v9-0001-0005,8,30,52.63,4.41860648542732,,98.54%
master,1,40,97.28,5.84086627240208,,v9-0001-0004,1,40,44.59,5.4278641075599,,v9-0001-0005,1,40,46.56,4.15857398010055,,104.42%
master,2,40,110.42,7.33623131319917,,v9-0001-0004,2,40,65.89,5.26320258680814,,v9-0001-0005,2,40,63.04,6.22315529023537,,95.67%
master,3,40,107.28,5.79151367127886,,v9-0001-0004,3,40,66.62,5.34725333892706,,v9-0001-0005,3,40,66.64,5.45956016025952,,100.03%
master,4,40,107.78,5.70389276607127,,v9-0001-0004,4,40,65.7,5.44134685062334,,v9-0001-0005,4,40,66.34,5.46923658698649,,100.97%
master,5,40,98.09,8.01977568017454,,v9-0001-0004,5,40,49.2,7.96545911800713,,v9-0001-0005,5,40,50.4,6.82955343797532,,102.44%
master,6,40,108.54,5.61546891682218,,v9-0001-0004,6,40,65.72,6.00575190970115,,v9-0001-0005,6,40,65.78,6.30252745396546,,100.09%
master,7,40,106.99,5.13680752859861,,v9-0001-0004,7,40,70.84,3.99764421654093,,v9-0001-0005,7,40,67.32,5.35767621698308,,95.03%
master,8,40,109.04,8.40571939725749,,v9-0001-0004,8,40,72.15,6.53619513275077,,v9-0001-0005,8,40,68.88,6.9088586010267,,95.47%
^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: More speedups for tuple deformation
@ 2026-02-24 08:45 Amit Langote <[email protected]>
parent: David Rowley <[email protected]>
2 siblings, 1 reply; 30+ messages in thread
From: Amit Langote @ 2026-02-24 08:45 UTC (permalink / raw)
To: David Rowley <[email protected]>; +Cc: Andres Freund <[email protected]>; John Naylor <[email protected]>; Chao Li <[email protected]>; PostgreSQL Developers <[email protected]>
Hi David,
On Tue, Feb 24, 2026 at 11:23 AM David Rowley <[email protected]> wrote:
> I've attached an updated version of the patch (v9).
I noticed what looks like some rebase noise in TupleDescFinalize():
off = att_nominal_alignby(off, cattr->attalignby);
/*
* attcacheoff is an int16, so don't try and cache any offsets larger
* than will fit in that type.
*/
if (off > PG_INT16_MAX)
break;
cattr->attcacheoff = off;
/*
* attcacheoff is an int16, so don't try and cache any offsets larger
* than will fit in that type.
*/
if (off > PG_INT16_MAX)
break;
cattr->attcacheoff = off;
--
Thanks, Amit Langote
^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: More speedups for tuple deformation
@ 2026-02-24 12:33 David Rowley <[email protected]>
parent: Amit Langote <[email protected]>
0 siblings, 0 replies; 30+ messages in thread
From: David Rowley @ 2026-02-24 12:33 UTC (permalink / raw)
To: Amit Langote <[email protected]>; +Cc: Andres Freund <[email protected]>; John Naylor <[email protected]>; Chao Li <[email protected]>; PostgreSQL Developers <[email protected]>
On Tue, 24 Feb 2026 at 21:46, Amit Langote <[email protected]> wrote:
> I noticed what looks like some rebase noise in TupleDescFinalize():
Thanks. Fixed locally. That must have happened during a rebase when I
renamed offp to off.
David
^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: More speedups for tuple deformation
@ 2026-02-24 14:39 Andres Freund <[email protected]>
parent: David Rowley <[email protected]>
2 siblings, 1 reply; 30+ messages in thread
From: Andres Freund @ 2026-02-24 14:39 UTC (permalink / raw)
To: David Rowley <[email protected]>; +Cc: John Naylor <[email protected]>; Chao Li <[email protected]>; PostgreSQL Developers <[email protected]>
Hi,
On 2026-02-24 15:23:17 +1300, David Rowley wrote:
> The changes in 0004 and 0005 are new. 0004 makes calling
> slot_getmissingattrs() the responsibility of the
> TupleTableSlotOps.getsomeattrs() function. Doing this allows
> getsomeattrs() to be called with the sibling call optimisation in
> slot_getsomeattrs_int() and since slot_getsomeattrs_int() is such a
> trivial function now, I ended up just modifying slot_getsomeattrs() to
> call getsomeattrs() in a way that allows the compiler to apply the
> sibling call optimisation. This seems to help reduce some overheads
> and makes the 0 extra column tests look better.
ISTM we should just merge 0004. In my testing it's a very clear win, without,
afaict, any downsides.
> 0005 reduces the size of CompactAttribute. It shrinks the struct down
> to 8 bytes from 16 by using some bitflags for some lesser-used
> booleans and by shrinking attcacheoff down to int16. The idea is that
> we just don't cache any offsets larger than 2^15. It's likely if we
> get a tuple that big that there's a variable-length attribute anyway,
> which caching the offset of isn't possible.
>
> I'm not getting great results from benchmarking the 0005 patch. I
> verified that gcc does access the array without calculating the
> element address from scratch each time and calculates it once, then
> increments the pointer by sizeof(CompactAttribute). See the attached
> .csv for the results on the 3 machines I tested on.
FWIW, where I had seen that be rather beneficial is the TupleDescCompactAttr()
at the start of the various loops, where the compiler has little choice to
compute the address of the tupdesc->compact_attrs[firstNeededCol]. That
matters only when only deforming a small number of columns, of course.
> I've also resequenced the patches to make the deform_bench test module
> part of the 0001 patch. This makes it easier to test the performance
> of master.
What are your thoughts about merging the deform_bench tooling? I wonder if we
should have src/test/modules/benchmark_tools or such, so we can add a few more
micro-benchmarky tools over time?
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: More speedups for tuple deformation
@ 2026-02-24 18:33 Zsolt Parragi <[email protected]>
parent: David Rowley <[email protected]>
2 siblings, 1 reply; 30+ messages in thread
From: Zsolt Parragi @ 2026-02-24 18:33 UTC (permalink / raw)
To: David Rowley <[email protected]>; +Cc: Andres Freund <[email protected]>; John Naylor <[email protected]>; Chao Li <[email protected]>; PostgreSQL Developers <[email protected]>
Hello
+ * We expect that 'bits' contains at least one 0 bit somewhere in the mask,
+ * not necessarily < natts.
+ */
Is this precondition really enough?
Let's say we have 20 attributes, only attribute 20 is NULL
Caller requests natts=8
That sets lastByte = 1, loop only checks bits[0], which is 0xFF, exits
with bytenum=1, bits[1] is also 0xFF
Then we execute
+ res += pg_rightmost_one_pos32(~bits[bytenum]);
where ~0xFF = 0
+ /* convert the lower 4 bits of null bitmap word into 32 bit int */
+ isnull_8 = (nullbyte & 0xf) * SPREAD_BITS_MULTIPLIER_32;
+
+ /*
+ * convert the upper 4 bits of null bitmap word into 32 bit int, shift
+ * into the upper 32 bit
+ */
+ isnull_8 |= ((uint64) ((nullbyte >> 4) * SPREAD_BITS_MULTIPLIER_32)) << 32;
+
+ /* mask out all other bits apart from the lowest bit of each byte */
+ isnull_8 &= UINT64CONST(0x0101010101010101);
+ memcpy(isnull, &isnull_8, sizeof(uint64));
Won't this mix up column numbers on big-endian systems?
Subject: [PATCH v9 1/5] Introduce deform_bench test module
For benchmaring tuple deformation.
---
Typo: should be benchmarking
+ * firstNonGuaranteedAttr stores the index to info the compact_attrs array for
to info should be "into"?
^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: More speedups for tuple deformation
@ 2026-02-25 00:39 David Rowley <[email protected]>
parent: Zsolt Parragi <[email protected]>
0 siblings, 1 reply; 30+ messages in thread
From: David Rowley @ 2026-02-25 00:39 UTC (permalink / raw)
To: Zsolt Parragi <[email protected]>; +Cc: Andres Freund <[email protected]>; John Naylor <[email protected]>; Chao Li <[email protected]>; PostgreSQL Developers <[email protected]>
Thanks for having a look at this.
On Wed, 25 Feb 2026 at 07:33, Zsolt Parragi <[email protected]> wrote:
> + * We expect that 'bits' contains at least one 0 bit somewhere in the mask,
> + * not necessarily < natts.
> + */
>
> Is this precondition really enough?
>
> Let's say we have 20 attributes, only attribute 20 is NULL
> Caller requests natts=8
>
> That sets lastByte = 1, loop only checks bits[0], which is 0xFF, exits
> with bytenum=1, bits[1] is also 0xFF
>
> Then we execute
>
> + res += pg_rightmost_one_pos32(~bits[bytenum]);
>
> where ~0xFF = 0
I think this works ok in v9, but in v10 I've added a cast so that line becomes:
res += pg_rightmost_one_pos32(~((uint32) bits[bytenum]));
For the case you describe, 0xFF becomes 0xFFFFFF00 and
pg_rightmost_one_pos32() returns 8, the lowest bit of the 2nd byte.
> + /* convert the lower 4 bits of null bitmap word into 32 bit int */
> + isnull_8 = (nullbyte & 0xf) * SPREAD_BITS_MULTIPLIER_32;
> +
> + /*
> + * convert the upper 4 bits of null bitmap word into 32 bit int, shift
> + * into the upper 32 bit
> + */
> + isnull_8 |= ((uint64) ((nullbyte >> 4) * SPREAD_BITS_MULTIPLIER_32)) << 32;
> +
> + /* mask out all other bits apart from the lowest bit of each byte */
> + isnull_8 &= UINT64CONST(0x0101010101010101);
> + memcpy(isnull, &isnull_8, sizeof(uint64));
>
>
> Won't this mix up column numbers on big-endian systems?
Yes, I believe you're right. I've added the following before the memcpy().
#ifdef WORDS_BIGENDIAN
/*
* Fix byte order on big-endian machines before copying to the array.
*/
isnull_8 = pg_bswap64(isnull_8);
#endif
>
> Subject: [PATCH v9 1/5] Introduce deform_bench test module
>
> For benchmaring tuple deformation.
> ---
>
> Typo: should be benchmarking
Fixed.
> + * firstNonGuaranteedAttr stores the index to info the compact_attrs array for
>
> to info should be "into"?
Also fixed.
I've attached a revised set of patches.
David
From b9e6d0cbe19c2ea5506edef13d0ad960f824eae0 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Tue, 27 Jan 2026 15:08:09 +1300
Subject: [PATCH v10 1/5] Introduce deform_bench test module
For benchmarking tuple deformation.
---
src/test/modules/deform_bench/.gitignore | 4 +
src/test/modules/deform_bench/Makefile | 21 ++++
.../deform_bench/deform_bench--1.0.sql | 8 ++
src/test/modules/deform_bench/deform_bench.c | 107 ++++++++++++++++++
.../modules/deform_bench/deform_bench.control | 4 +
src/test/modules/deform_bench/meson.build | 22 ++++
src/test/modules/meson.build | 1 +
7 files changed, 167 insertions(+)
create mode 100644 src/test/modules/deform_bench/.gitignore
create mode 100644 src/test/modules/deform_bench/Makefile
create mode 100644 src/test/modules/deform_bench/deform_bench--1.0.sql
create mode 100644 src/test/modules/deform_bench/deform_bench.c
create mode 100644 src/test/modules/deform_bench/deform_bench.control
create mode 100644 src/test/modules/deform_bench/meson.build
diff --git a/src/test/modules/deform_bench/.gitignore b/src/test/modules/deform_bench/.gitignore
new file mode 100644
index 00000000000..5dcb3ff9723
--- /dev/null
+++ b/src/test/modules/deform_bench/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/src/test/modules/deform_bench/Makefile b/src/test/modules/deform_bench/Makefile
new file mode 100644
index 00000000000..b5fc0f7a583
--- /dev/null
+++ b/src/test/modules/deform_bench/Makefile
@@ -0,0 +1,21 @@
+# src/test/modules/deform_bench/Makefile
+
+MODULE_big = deform_bench
+OBJS = deform_bench.o
+
+EXTENSION = deform_bench
+DATA = deform_bench--1.0.sql
+PGFILEDESC = "deform_bench - tuple deform benchmarking"
+
+REGRESS = deform_bench
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = src/test/modules/deform_bench
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/src/test/modules/deform_bench/deform_bench--1.0.sql b/src/test/modules/deform_bench/deform_bench--1.0.sql
new file mode 100644
index 00000000000..492b71dba3b
--- /dev/null
+++ b/src/test/modules/deform_bench/deform_bench--1.0.sql
@@ -0,0 +1,8 @@
+/* deform_bench--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION deform_bench" to load this file. \quit
+
+CREATE FUNCTION deform_bench(tableoid Oid, attnum int[]) RETURNS FLOAT
+AS 'MODULE_PATHNAME', 'deform_bench'
+LANGUAGE C VOLATILE STRICT;
diff --git a/src/test/modules/deform_bench/deform_bench.c b/src/test/modules/deform_bench/deform_bench.c
new file mode 100644
index 00000000000..7838f639bef
--- /dev/null
+++ b/src/test/modules/deform_bench/deform_bench.c
@@ -0,0 +1,107 @@
+/*-------------------------------------------------------------------------
+ *
+ * deform_bench.c
+ *
+ * for benchmarking tuple deformation routines
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include <time.h>
+#include <sys/time.h>
+
+#include "access/heapam.h"
+#include "access/relscan.h"
+#include "catalog/pg_am_d.h"
+#include "catalog/pg_type_d.h"
+#include "funcapi.h"
+#include "miscadmin.h"
+#include "utils/array.h"
+#include "utils/arrayaccess.h"
+#include "utils/builtins.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(deform_bench);
+
+Datum
+deform_bench(PG_FUNCTION_ARGS)
+{
+ Oid tableoid = PG_GETARG_OID(0);
+ ArrayType *array = PG_GETARG_ARRAYTYPE_P(1);
+ TableScanDesc scan;
+ Relation rel;
+ TupleDesc tupdesc;
+ TupleTableSlot *slot;
+ Datum *elem_datums = NULL;
+ bool *elem_nulls = NULL;
+ int elem_count;
+ int *attnums;
+ clock_t start,
+ end;
+
+ rel = relation_open(tableoid, AccessShareLock);
+
+ if (rel->rd_rel->relam != HEAP_TABLE_AM_OID)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("only heap AM is supported")));
+
+ tupdesc = RelationGetDescr(rel);
+ slot = MakeTupleTableSlot(tupdesc, &TTSOpsBufferHeapTuple);
+ scan = table_beginscan_strat(rel, GetActiveSnapshot(), 0, NULL, true, false);
+
+ /*
+ * The array is used to allow callers to define how many atts to deform.
+ * e.g: '{1,10}'::int[] would deform attnum=1, then in a 2nd pass deform
+ * the remainder up to attnum=10. Passing an element as NULL means all
+ * attnums. This allows simulation of incremental deformation. Generally
+ * if you're passing an array with more than 1 element, then the array
+ * should be in ascending order. Doing something like '{10,1}' would mean
+ * we've already deformed 10 attributes and on the 2nd pass there's
+ * nothing to do since attnum=1 was already deformed in the first pass.
+ *
+ * You'll get an ERROR if you pass a number higher than the number of
+ * attributes in the table.
+ */
+ deconstruct_array(array,
+ INT4OID,
+ sizeof(int32),
+ true,
+ 'i',
+ &elem_datums,
+ &elem_nulls,
+ &elem_count);
+
+ attnums = palloc_array(int, elem_count);
+
+ for (int i = 0; i < elem_count; i++)
+ {
+ /* Make a NULL element mean all attributes */
+ if (elem_nulls[i])
+ attnums[i] = tupdesc->natts;
+ else
+ attnums[i] = DatumGetInt32(elem_datums[i]);
+ }
+
+ start = clock();
+
+ while (heap_getnextslot(scan, ForwardScanDirection, slot))
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ /* Deform in stages according to the attnums array */
+ for (int i = 0; i < elem_count; i++)
+ slot_getsomeattrs(slot, attnums[i]);
+ }
+
+ end = clock();
+
+ ExecDropSingleTupleTableSlot(slot);
+ table_endscan(scan);
+ relation_close(rel, AccessShareLock);
+
+
+ /* Returns the number of milliseconds to run the test */
+ PG_RETURN_FLOAT8((double) (end - start) / (CLOCKS_PER_SEC / 1000));
+}
diff --git a/src/test/modules/deform_bench/deform_bench.control b/src/test/modules/deform_bench/deform_bench.control
new file mode 100644
index 00000000000..a2023f9d738
--- /dev/null
+++ b/src/test/modules/deform_bench/deform_bench.control
@@ -0,0 +1,4 @@
+# deform_bench extension
+comment = 'functions for benchmarking tuple deformation'
+default_version = '1.0'
+module_pathname = '$libdir/deform_bench'
diff --git a/src/test/modules/deform_bench/meson.build b/src/test/modules/deform_bench/meson.build
new file mode 100644
index 00000000000..82049585244
--- /dev/null
+++ b/src/test/modules/deform_bench/meson.build
@@ -0,0 +1,22 @@
+# Copyright (c) 2026, PostgreSQL Global Development Group
+
+deform_bench_sources = files(
+ 'deform_bench.c',
+)
+
+if host_system == 'windows'
+ deform_bench_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+ '--NAME', 'deform_bench',
+ '--FILEDESC', 'deform_bench - benchmarking tuple deformation',])
+endif
+
+deform_bench = shared_module('deform_bench',
+ deform_bench_sources,
+ kwargs: pg_test_mod_args,
+)
+test_install_libs += deform_bench
+
+test_install_data += files(
+ 'deform_bench--1.0.sql',
+ 'deform_bench.control',
+)
diff --git a/src/test/modules/meson.build b/src/test/modules/meson.build
index 2634a519935..ef2b0af4581 100644
--- a/src/test/modules/meson.build
+++ b/src/test/modules/meson.build
@@ -2,6 +2,7 @@
subdir('brin')
subdir('commit_ts')
+subdir('deform_bench')
subdir('delay_execution')
subdir('dummy_index_am')
subdir('dummy_seclabel')
--
2.51.0
From 04de30bdc8833148513fe07a0b9c7eb1e06c6b81 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Wed, 21 Jan 2026 15:41:37 +1300
Subject: [PATCH v10 2/5] Add empty TupleDescFinalize() function
Currently does nothing, but will in a future commit.
---
contrib/dblink/dblink.c | 4 ++++
contrib/pg_buffercache/pg_buffercache_pages.c | 2 ++
contrib/pg_visibility/pg_visibility.c | 2 ++
src/backend/access/brin/brin_tuple.c | 1 +
src/backend/access/common/tupdesc.c | 13 +++++++++++++
src/backend/access/gin/ginutil.c | 1 +
src/backend/access/gist/gistscan.c | 1 +
src/backend/access/spgist/spgutils.c | 1 +
src/backend/access/transam/twophase.c | 1 +
src/backend/access/transam/xlogfuncs.c | 1 +
src/backend/backup/basebackup_copy.c | 3 +++
src/backend/catalog/index.c | 2 ++
src/backend/catalog/pg_publication.c | 1 +
src/backend/catalog/toasting.c | 6 ++++++
src/backend/commands/explain.c | 1 +
src/backend/commands/functioncmds.c | 1 +
src/backend/commands/sequence.c | 1 +
src/backend/commands/tablecmds.c | 4 ++++
src/backend/commands/wait.c | 1 +
src/backend/executor/execSRF.c | 2 ++
src/backend/executor/execTuples.c | 4 ++++
src/backend/executor/nodeFunctionscan.c | 2 ++
src/backend/parser/parse_relation.c | 4 +++-
src/backend/parser/parse_target.c | 2 ++
.../replication/libpqwalreceiver/libpqwalreceiver.c | 1 +
src/backend/replication/walsender.c | 5 +++++
src/backend/utils/adt/acl.c | 1 +
src/backend/utils/adt/genfile.c | 1 +
src/backend/utils/adt/lockfuncs.c | 1 +
src/backend/utils/adt/orderedsetaggs.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 5 +++++
src/backend/utils/adt/tsvector_op.c | 1 +
src/backend/utils/cache/relcache.c | 8 ++++++++
src/backend/utils/fmgr/funcapi.c | 6 ++++++
src/backend/utils/misc/guc_funcs.c | 5 +++++
src/include/access/tupdesc.h | 1 +
src/pl/plpgsql/src/pl_comp.c | 2 ++
.../test_custom_stats/test_custom_fixed_stats.c | 1 +
src/test/modules/test_predtest/test_predtest.c | 1 +
39 files changed, 100 insertions(+), 1 deletion(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 2498d80c8e7..4038950a6ef 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -881,6 +881,7 @@ materializeResult(FunctionCallInfo fcinfo, PGconn *conn, PGresult *res)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "status",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
ntuples = 1;
nfields = 1;
}
@@ -1044,6 +1045,7 @@ materializeQueryResult(FunctionCallInfo fcinfo,
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "status",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
attinmeta = TupleDescGetAttInMetadata(tupdesc);
oldcontext = MemoryContextSwitchTo(rsinfo->econtext->ecxt_per_query_memory);
@@ -1529,6 +1531,8 @@ dblink_get_pkey(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 2, "colname",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+
/*
* Generate attribute metadata needed later to produce tuples from raw
* C strings
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 89b86855243..a6b4fb5252b 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -174,6 +174,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
INT4OID, -1, 0);
+ TupleDescFinalize(tupledesc);
fctx->tupdesc = BlessTupleDesc(tupledesc);
/* Allocate NBuffers worth of BufferCachePagesRec records. */
@@ -442,6 +443,7 @@ pg_buffercache_os_pages_internal(FunctionCallInfo fcinfo, bool include_numa)
TupleDescInitEntry(tupledesc, (AttrNumber) 3, "numa_node",
INT4OID, -1, 0);
+ TupleDescFinalize(tupledesc);
fctx->tupdesc = BlessTupleDesc(tupledesc);
fctx->include_numa = include_numa;
diff --git a/contrib/pg_visibility/pg_visibility.c b/contrib/pg_visibility/pg_visibility.c
index 9bc3a784bf7..dfab0b64cf5 100644
--- a/contrib/pg_visibility/pg_visibility.c
+++ b/contrib/pg_visibility/pg_visibility.c
@@ -469,6 +469,8 @@ pg_visibility_tupdesc(bool include_blkno, bool include_pd)
TupleDescInitEntry(tupdesc, ++a, "pd_all_visible", BOOLOID, -1, 0);
Assert(a == maxattr);
+ TupleDescFinalize(tupdesc);
+
return BlessTupleDesc(tupdesc);
}
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 69c233c62eb..742ac089a28 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -84,6 +84,7 @@ brtuple_disk_tupdesc(BrinDesc *brdesc)
MemoryContextSwitchTo(oldcxt);
+ TupleDescFinalize(tupdesc);
brdesc->bd_disktdesc = tupdesc;
}
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index b69d10f0a45..2137385a833 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -221,6 +221,9 @@ CreateTupleDesc(int natts, Form_pg_attribute *attrs)
memcpy(TupleDescAttr(desc, i), attrs[i], ATTRIBUTE_FIXED_PART_SIZE);
populate_compact_attribute(desc, i);
}
+
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -265,6 +268,8 @@ CreateTupleDescCopy(TupleDesc tupdesc)
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -311,6 +316,8 @@ CreateTupleDescTruncatedCopy(TupleDesc tupdesc, int natts)
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -396,6 +403,8 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -438,6 +447,8 @@ TupleDescCopy(TupleDesc dst, TupleDesc src)
* source's refcount would be wrong in any case.)
*/
dst->tdrefcount = -1;
+
+ TupleDescFinalize(dst);
}
/*
@@ -1065,6 +1076,8 @@ BuildDescFromLists(const List *names, const List *types, const List *typmods, co
TupleDescInitEntryCollation(desc, attnum, attcollation);
}
+ TupleDescFinalize(desc);
+
return desc;
}
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index ff927279cc3..fe7b984ff32 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -129,6 +129,7 @@ initGinState(GinState *state, Relation index)
attr->attndims);
TupleDescInitEntryCollation(state->tupdesc[i], (AttrNumber) 2,
attr->attcollation);
+ TupleDescFinalize(state->tupdesc[i]);
}
/*
diff --git a/src/backend/access/gist/gistscan.c b/src/backend/access/gist/gistscan.c
index f23bc4a6757..c65f93abdae 100644
--- a/src/backend/access/gist/gistscan.c
+++ b/src/backend/access/gist/gistscan.c
@@ -201,6 +201,7 @@ gistrescan(IndexScanDesc scan, ScanKey key, int nkeys,
attno - 1)->atttypid,
-1, 0);
}
+ TupleDescFinalize(so->giststate->fetchTupdesc);
scan->xs_hitupdesc = so->giststate->fetchTupdesc;
/* Also create a memory context that will hold the returned tuples */
diff --git a/src/backend/access/spgist/spgutils.c b/src/backend/access/spgist/spgutils.c
index 9f5379b87ac..b246e8127db 100644
--- a/src/backend/access/spgist/spgutils.c
+++ b/src/backend/access/spgist/spgutils.c
@@ -340,6 +340,7 @@ getSpGistTupleDesc(Relation index, SpGistTypeDesc *keyType)
TupleDescCompactAttr(outTupDesc, i)->attcacheoff = -1;
populate_compact_attribute(outTupDesc, spgKeyColumn);
+ TupleDescFinalize(outTupDesc);
}
return outTupDesc;
}
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index e4340b59640..7f4ed02a6b9 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -744,6 +744,7 @@ pg_prepared_xact(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 5, "dbid",
OIDOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
/*
diff --git a/src/backend/access/transam/xlogfuncs.c b/src/backend/access/transam/xlogfuncs.c
index 2efe4105efb..b6bc616c74c 100644
--- a/src/backend/access/transam/xlogfuncs.c
+++ b/src/backend/access/transam/xlogfuncs.c
@@ -400,6 +400,7 @@ pg_walfile_name_offset(PG_FUNCTION_ARGS)
TupleDescInitEntry(resultTupleDesc, (AttrNumber) 2, "file_offset",
INT4OID, -1, 0);
+ TupleDescFinalize(resultTupleDesc);
resultTupleDesc = BlessTupleDesc(resultTupleDesc);
/*
diff --git a/src/backend/backup/basebackup_copy.c b/src/backend/backup/basebackup_copy.c
index 07f58b39d8c..6c3453efd80 100644
--- a/src/backend/backup/basebackup_copy.c
+++ b/src/backend/backup/basebackup_copy.c
@@ -357,6 +357,8 @@ SendXlogRecPtrResult(XLogRecPtr ptr, TimeLineID tli)
*/
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "tli", INT8OID, -1, 0);
+ TupleDescFinalize(tupdesc);
+
/* send RowDescription */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -388,6 +390,7 @@ SendTablespaceList(List *tablespaces)
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 1, "spcoid", OIDOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "spclocation", TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 3, "size", INT8OID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* send RowDescription */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 43de42ce39e..75e97fb394a 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -481,6 +481,8 @@ ConstructTupleDescriptor(Relation heapRelation,
populate_compact_attribute(indexTupDesc, i);
}
+ TupleDescFinalize(indexTupDesc);
+
return indexTupDesc;
}
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index 9a4791c573e..fa353a0dd37 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -1230,6 +1230,7 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 4, "qual",
PG_NODE_TREEOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
funcctx->user_fctx = table_infos;
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index c78dcea98c1..078a1cf5127 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -229,6 +229,12 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attcompression = InvalidCompressionMethod;
TupleDescAttr(tupdesc, 2)->attcompression = InvalidCompressionMethod;
+ populate_compact_attribute(tupdesc, 0);
+ populate_compact_attribute(tupdesc, 1);
+ populate_compact_attribute(tupdesc, 2);
+
+ TupleDescFinalize(tupdesc);
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 93918a223b8..5f922c3f5c2 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -281,6 +281,7 @@ ExplainResultDesc(ExplainStmt *stmt)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "QUERY PLAN",
result_type, -1, 0);
+ TupleDescFinalize(tupdesc);
return tupdesc;
}
diff --git a/src/backend/commands/functioncmds.c b/src/backend/commands/functioncmds.c
index 242372b1e68..3afd762e9dc 100644
--- a/src/backend/commands/functioncmds.c
+++ b/src/backend/commands/functioncmds.c
@@ -2424,6 +2424,7 @@ CallStmtResultDesc(CallStmt *stmt)
-1,
0);
}
+ TupleDescFinalize(tupdesc);
}
return tupdesc;
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index e1b808bbb60..551667650ba 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -1808,6 +1808,7 @@ pg_get_sequence_data(PG_FUNCTION_ARGS)
BOOLOID, -1, 0);
TupleDescInitEntry(resultTupleDesc, (AttrNumber) 3, "page_lsn",
LSNOID, -1, 0);
+ TupleDescFinalize(resultTupleDesc);
resultTupleDesc = BlessTupleDesc(resultTupleDesc);
seqrel = try_relation_open(relid, AccessShareLock);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index b04b0dbd2a0..8678cecd53f 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -1030,6 +1030,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
}
}
+ TupleDescFinalize(descriptor);
+
/*
* For relations with table AM and partitioned tables, select access
* method to use: an explicitly indicated one, or (in the case of a
@@ -1458,6 +1460,8 @@ BuildDescForRelation(const List *columns)
populate_compact_attribute(desc, attnum - 1);
}
+ TupleDescFinalize(desc);
+
return desc;
}
diff --git a/src/backend/commands/wait.c b/src/backend/commands/wait.c
index 1290df10c6f..8e920a72372 100644
--- a/src/backend/commands/wait.c
+++ b/src/backend/commands/wait.c
@@ -338,5 +338,6 @@ WaitStmtResultDesc(WaitStmt *stmt)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "status",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
return tupdesc;
}
diff --git a/src/backend/executor/execSRF.c b/src/backend/executor/execSRF.c
index a0b111dc0e4..b481e50acfb 100644
--- a/src/backend/executor/execSRF.c
+++ b/src/backend/executor/execSRF.c
@@ -272,6 +272,7 @@ ExecMakeTableFunctionResult(SetExprState *setexpr,
funcrettype,
-1,
0);
+ TupleDescFinalize(tupdesc);
rsinfo.setDesc = tupdesc;
}
MemoryContextSwitchTo(oldcontext);
@@ -776,6 +777,7 @@ init_sexpr(Oid foid, Oid input_collation, Expr *node,
funcrettype,
-1,
0);
+ TupleDescFinalize(tupdesc);
sexpr->funcResultDesc = tupdesc;
sexpr->funcReturnsTuple = false;
}
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index b768eae9e53..e6ab51e6404 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -2173,6 +2173,8 @@ ExecTypeFromTLInternal(List *targetList, bool skipjunk)
cur_resno++;
}
+ TupleDescFinalize(typeInfo);
+
return typeInfo;
}
@@ -2207,6 +2209,8 @@ ExecTypeFromExprList(List *exprList)
cur_resno++;
}
+ TupleDescFinalize(typeInfo);
+
return typeInfo;
}
diff --git a/src/backend/executor/nodeFunctionscan.c b/src/backend/executor/nodeFunctionscan.c
index 63e605e1f81..feb82d64967 100644
--- a/src/backend/executor/nodeFunctionscan.c
+++ b/src/backend/executor/nodeFunctionscan.c
@@ -414,6 +414,7 @@ ExecInitFunctionScan(FunctionScan *node, EState *estate, int eflags)
TupleDescInitEntryCollation(tupdesc,
(AttrNumber) 1,
exprCollation(funcexpr));
+ TupleDescFinalize(tupdesc);
}
else
{
@@ -485,6 +486,7 @@ ExecInitFunctionScan(FunctionScan *node, EState *estate, int eflags)
0);
}
+ TupleDescFinalize(scan_tupdesc);
Assert(attno == natts);
}
diff --git a/src/backend/parser/parse_relation.c b/src/backend/parser/parse_relation.c
index e003db520de..9c415e166ee 100644
--- a/src/backend/parser/parse_relation.c
+++ b/src/backend/parser/parse_relation.c
@@ -1883,6 +1883,7 @@ addRangeTableEntryForFunction(ParseState *pstate,
TupleDescInitEntryCollation(tupdesc,
(AttrNumber) 1,
exprCollation(funcexpr));
+ TupleDescFinalize(tupdesc);
}
else if (functypclass == TYPEFUNC_RECORD)
{
@@ -1940,6 +1941,7 @@ addRangeTableEntryForFunction(ParseState *pstate,
i++;
}
+ TupleDescFinalize(tupdesc);
/*
* Ensure that the coldeflist defines a legal set of names (no
@@ -2008,7 +2010,7 @@ addRangeTableEntryForFunction(ParseState *pstate,
0);
/* no need to set collation */
}
-
+ TupleDescFinalize(tupdesc);
Assert(natts == totalatts);
}
else
diff --git a/src/backend/parser/parse_target.c b/src/backend/parser/parse_target.c
index dbf5b2b5c01..a03d82c0540 100644
--- a/src/backend/parser/parse_target.c
+++ b/src/backend/parser/parse_target.c
@@ -1572,6 +1572,8 @@ expandRecordVariable(ParseState *pstate, Var *var, int levelsup)
}
Assert(lname == NULL && lvar == NULL); /* lists same length? */
+ TupleDescFinalize(tupleDesc);
+
return tupleDesc;
}
diff --git a/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c b/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
index 7c8639b32e9..9f04c9ed25d 100644
--- a/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
+++ b/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
@@ -1073,6 +1073,7 @@ libpqrcv_processTuples(PGresult *pgres, WalRcvExecResult *walres,
for (coln = 0; coln < nRetTypes; coln++)
TupleDescInitEntry(walres->tupledesc, (AttrNumber) coln + 1,
PQfname(pgres, coln), retTypes[coln], -1, 0);
+ TupleDescFinalize(walres->tupledesc);
attinmeta = TupleDescGetAttInMetadata(walres->tupledesc);
/* No point in doing more here if there were no tuples returned. */
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c
index 2cde8ebc729..33a9e8d7f21 100644
--- a/src/backend/replication/walsender.c
+++ b/src/backend/replication/walsender.c
@@ -451,6 +451,7 @@ IdentifySystem(void)
TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 4, "dbname",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -496,6 +497,7 @@ ReadReplicationSlot(ReadReplicationSlotCmd *cmd)
/* TimeLineID is unsigned, so int4 is not wide enough. */
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 3, "restart_tli",
INT8OID, -1, 0);
+ TupleDescFinalize(tupdesc);
memset(nulls, true, READ_REPLICATION_SLOT_COLS * sizeof(bool));
@@ -598,6 +600,7 @@ SendTimeLineHistory(TimeLineHistoryCmd *cmd)
tupdesc = CreateTemplateTupleDesc(2);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 1, "filename", TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "content", TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
TLHistoryFileName(histfname, cmd->timeline);
TLHistoryFilePath(path, cmd->timeline);
@@ -1015,6 +1018,7 @@ StartReplication(StartReplicationCmd *cmd)
INT8OID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "next_tli_startpos",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuple */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -1369,6 +1373,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd)
TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 4, "output_plugin",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
diff --git a/src/backend/utils/adt/acl.c b/src/backend/utils/adt/acl.c
index 641673f0b0e..ce07f2bc046 100644
--- a/src/backend/utils/adt/acl.c
+++ b/src/backend/utils/adt/acl.c
@@ -1819,6 +1819,7 @@ aclexplode(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 4, "is_grantable",
BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
/* allocate memory for user context */
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index c083608b1d5..bfb949401d0 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -454,6 +454,7 @@ pg_stat_file(PG_FUNCTION_ARGS)
"creation", TIMESTAMPTZOID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 6,
"isdir", BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
memset(isnull, false, sizeof(isnull));
diff --git a/src/backend/utils/adt/lockfuncs.c b/src/backend/utils/adt/lockfuncs.c
index 9dadd6da672..4481c354fd6 100644
--- a/src/backend/utils/adt/lockfuncs.c
+++ b/src/backend/utils/adt/lockfuncs.c
@@ -146,6 +146,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 16, "waitstart",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
/*
diff --git a/src/backend/utils/adt/orderedsetaggs.c b/src/backend/utils/adt/orderedsetaggs.c
index 3b6da8e36ac..fd8b8676470 100644
--- a/src/backend/utils/adt/orderedsetaggs.c
+++ b/src/backend/utils/adt/orderedsetaggs.c
@@ -233,6 +233,7 @@ ordered_set_startup(FunctionCallInfo fcinfo, bool use_tuples)
-1,
0);
+ TupleDescFinalize(newdesc);
FreeTupleDesc(qstate->tupdesc);
qstate->tupdesc = newdesc;
}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index b1df96e7b0b..0b10da3b180 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -769,6 +769,7 @@ pg_stat_get_backend_subxact(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 2, "subxact_overflow",
BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
if ((local_beentry = pgstat_get_local_beentry_by_proc_number(procNumber)) != NULL)
@@ -1670,6 +1671,7 @@ pg_stat_wal_build_tuple(PgStat_WalCounters wal_counters,
TupleDescInitEntry(tupdesc, (AttrNumber) 6, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
/* Fill values and NULLs */
@@ -2097,6 +2099,7 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 7, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
/* Get statistics about the archiver process */
@@ -2178,6 +2181,7 @@ pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
TIMESTAMPTZOID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 13, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
namestrcpy(&slotname, text_to_cstring(slotname_text));
@@ -2265,6 +2269,7 @@ pg_stat_get_subscription_stats(PG_FUNCTION_ARGS)
INT8OID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 13, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
if (!subentry)
diff --git a/src/backend/utils/adt/tsvector_op.c b/src/backend/utils/adt/tsvector_op.c
index 71c7c7d3b3c..d8dece42b9b 100644
--- a/src/backend/utils/adt/tsvector_op.c
+++ b/src/backend/utils/adt/tsvector_op.c
@@ -651,6 +651,7 @@ tsvector_unnest(PG_FUNCTION_ARGS)
TEXTARRAYOID, -1, 0);
if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
elog(ERROR, "return type must be a row type");
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = tupdesc;
funcctx->user_fctx = PG_GETARG_TSVECTOR_COPY(0);
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 6b634c9fff1..770edb34e08 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -729,6 +729,8 @@ RelationBuildTupleDesc(Relation relation)
pfree(constr);
relation->rd_att->constr = NULL;
}
+
+ TupleDescFinalize(relation->rd_att);
}
/*
@@ -1985,6 +1987,7 @@ formrdesc(const char *relationName, Oid relationReltype,
/* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
TupleDescCompactAttr(relation->rd_att, 0)->attcacheoff = 0;
+ TupleDescFinalize(relation->rd_att);
/* mark not-null status */
if (has_not_null)
@@ -3688,6 +3691,8 @@ RelationBuildLocalRelation(const char *relname,
for (i = 0; i < natts; i++)
TupleDescAttr(rel->rd_att, i)->attrelid = relid;
+ TupleDescFinalize(rel->rd_att);
+
rel->rd_rel->reltablespace = reltablespace;
if (mapped_relation)
@@ -4443,6 +4448,7 @@ BuildHardcodedDescriptor(int natts, const FormData_pg_attribute *attrs)
/* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
TupleDescCompactAttr(result, 0)->attcacheoff = 0;
+ TupleDescFinalize(result);
/* Note: we don't bother to set up a TupleConstr entry */
@@ -6268,6 +6274,8 @@ load_relcache_init_file(bool shared)
populate_compact_attribute(rel->rd_att, i);
}
+ TupleDescFinalize(rel->rd_att);
+
/* next read the access method specific field */
if (fread(&len, 1, sizeof(len), fp) != sizeof(len))
goto read_failed;
diff --git a/src/backend/utils/fmgr/funcapi.c b/src/backend/utils/fmgr/funcapi.c
index 8a934ea8dca..516d02cfb82 100644
--- a/src/backend/utils/fmgr/funcapi.c
+++ b/src/backend/utils/fmgr/funcapi.c
@@ -340,6 +340,8 @@ get_expr_result_type(Node *expr,
exprCollation(col));
i++;
}
+ TupleDescFinalize(tupdesc);
+
if (resultTypeId)
*resultTypeId = rexpr->row_typeid;
if (resultTupleDesc)
@@ -1044,6 +1046,7 @@ resolve_polymorphic_tupdesc(TupleDesc tupdesc, oidvector *declared_args,
}
}
+ TupleDescFinalize(tupdesc);
return true;
}
@@ -1853,6 +1856,8 @@ build_function_result_tupdesc_d(char prokind,
0);
}
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -1970,6 +1975,7 @@ TypeGetTupleDesc(Oid typeoid, List *colaliases)
typeoid,
-1,
0);
+ TupleDescFinalize(tupdesc);
}
else if (functypclass == TYPEFUNC_RECORD)
{
diff --git a/src/backend/utils/misc/guc_funcs.c b/src/backend/utils/misc/guc_funcs.c
index 8524dd3a981..472cb5393ce 100644
--- a/src/backend/utils/misc/guc_funcs.c
+++ b/src/backend/utils/misc/guc_funcs.c
@@ -444,6 +444,7 @@ GetPGVariableResultDesc(const char *name)
TupleDescInitEntry(tupdesc, (AttrNumber) 1, varname,
TEXTOID, -1, 0);
}
+ TupleDescFinalize(tupdesc);
return tupdesc;
}
@@ -465,6 +466,7 @@ ShowGUCConfigOption(const char *name, DestReceiver *dest)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 1, varname,
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -499,6 +501,7 @@ ShowAllGUCConfig(DestReceiver *dest)
TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 3, "description",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -934,6 +937,8 @@ show_all_settings(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 17, "pending_restart",
BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+
/*
* Generate attribute metadata needed later to produce tuples from raw
* C strings
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index d46cdbf7a3c..595413dbbc5 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -195,6 +195,7 @@ extern TupleDesc CreateTupleDescTruncatedCopy(TupleDesc tupdesc, int natts);
extern TupleDesc CreateTupleDescCopyConstr(TupleDesc tupdesc);
+#define TupleDescFinalize(d) ((void) 0)
#define TupleDescSize(src) \
(offsetof(struct TupleDescData, compact_attrs) + \
(src)->natts * sizeof(CompactAttribute) + \
diff --git a/src/pl/plpgsql/src/pl_comp.c b/src/pl/plpgsql/src/pl_comp.c
index 5ecc7766757..b72c963b3be 100644
--- a/src/pl/plpgsql/src/pl_comp.c
+++ b/src/pl/plpgsql/src/pl_comp.c
@@ -1912,6 +1912,8 @@ build_row_from_vars(PLpgSQL_variable **vars, int numvars)
TupleDescInitEntryCollation(row->rowtupdesc, i + 1, typcoll);
}
+ TupleDescFinalize(row->rowtupdesc);
+
return row;
}
diff --git a/src/test/modules/test_custom_stats/test_custom_fixed_stats.c b/src/test/modules/test_custom_stats/test_custom_fixed_stats.c
index 485e08e5c19..f9e7c717280 100644
--- a/src/test/modules/test_custom_stats/test_custom_fixed_stats.c
+++ b/src/test/modules/test_custom_stats/test_custom_fixed_stats.c
@@ -206,6 +206,7 @@ test_custom_stats_fixed_report(PG_FUNCTION_ARGS)
INT8OID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 2, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
values[0] = Int64GetDatum(stats->numcalls);
diff --git a/src/test/modules/test_predtest/test_predtest.c b/src/test/modules/test_predtest/test_predtest.c
index 679a5de456d..48ca2a4ea70 100644
--- a/src/test/modules/test_predtest/test_predtest.c
+++ b/src/test/modules/test_predtest/test_predtest.c
@@ -230,6 +230,7 @@ test_predtest(PG_FUNCTION_ARGS)
"s_r_holds", BOOLOID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 8,
"w_r_holds", BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
tupdesc = BlessTupleDesc(tupdesc);
values[0] = BoolGetDatum(strong_implied_by);
--
2.51.0
From 46c7e9dd888247866b0b2931b87d8a7cfec9bb1b Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Tue, 31 Dec 2024 09:19:24 +1300
Subject: [PATCH v10 3/5] Optimize tuple deformation
This commit includes various optimizations to improve the performance of
tuple deformation.
We now precalculate CompactAttribute's attcacheoff, which allows us to
remove the code from the deform routines which was setting the
attcacheoff. Setting the attcacheoff is handled by TupleDescFinalize(),
which must be called before the TupleDesc is used for anything. Having
this TupleDescFinalize() function means we can store the first
attribute in the TupleDesc which does not have an offset cached. That
allows us to add a dedicated deforming loop to deform all attributes up
to the final one with an attcacheoff set, or up to the first NULL
attribute, whichever comes first.
We also record the maximum attribute number which is guaranteed to exist
in the tuple, that is, has a NOT NULL constraint and isn't an
atthasmissing attribute. When deforming only attributes prior to the
guaranteed attnum, we've no need to access the tuple's natt count. As an
additional optimization, we only count fixed-width columns when
calculating the maximum guaranteed column as this eliminates the need to
emit code to fetch byref types in the deformation loop for guaranteed
attributes.
Some locations in the code deform tuples that have yet to go through NOT
NULL constraint validation. We're unable to perform the guaranteed
attribute optimization when that's the case. The optimization is opt-in
via the TupleTableSlot using the TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS
flag.
This commit also adds a more efficient way of populating the isnull
array by using a bit-wise trick which performs multiplication on the
inverse of the tuple's bitmap byte and masking out all but the lower bit
of each of the boolean's byte. This results in much more optimal code
when compared to determining the NULLness via att_isnull(). 8 isnull
elements are processed at once using this method, which means we need to
round the tts_isnull array size up to the next 8 bytes. The palloc code
does this anyway, but the round-up needed to be formalized so as not to
overwrite the sentinel byte in debug builds.
---
src/backend/access/common/heaptuple.c | 362 ++++++++----------
src/backend/access/common/indextuple.c | 371 ++++++++-----------
src/backend/access/common/tupdesc.c | 46 +++
src/backend/access/spgist/spgutils.c | 3 -
src/backend/executor/execTuples.c | 360 +++++++++---------
src/backend/executor/nodeSeqscan.c | 2 +
src/backend/jit/llvm/llvmjit_deform.c | 6 -
src/backend/utils/cache/relcache.c | 12 -
src/include/access/tupdesc.h | 19 +-
src/include/access/tupmacs.h | 206 +++++++++-
src/include/executor/tuptable.h | 16 +-
src/test/modules/deform_bench/deform_bench.c | 1 +
12 files changed, 781 insertions(+), 623 deletions(-)
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 11bec20e82e..606c1f67568 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -498,19 +498,7 @@ heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc)
* nocachegetattr
*
* This only gets called from fastgetattr(), in cases where we
- * can't use a cacheoffset and the value is not null.
- *
- * This caches attribute offsets in the attribute descriptor.
- *
- * An alternative way to speed things up would be to cache offsets
- * with the tuple, but that seems more difficult unless you take
- * the storage hit of actually putting those offsets into the
- * tuple you send to disk. Yuck.
- *
- * This scheme will be slightly slower than that, but should
- * perform well for queries which hit large #'s of tuples. After
- * you cache the offsets once, examining all the other tuples using
- * the same attribute descriptor will go much quicker. -cim 5/4/91
+ * can't use the attcacheoff and the value is not null.
*
* NOTE: if you need to change this code, see also heap_deform_tuple.
* Also see nocache_index_getattr, which is the same code for index
@@ -522,194 +510,123 @@ nocachegetattr(HeapTuple tup,
int attnum,
TupleDesc tupleDesc)
{
+ CompactAttribute *cattr;
HeapTupleHeader td = tup->t_data;
char *tp; /* ptr to data part of tuple */
bits8 *bp = td->t_bits; /* ptr to null bitmap in tuple */
- bool slow = false; /* do we have to walk attrs? */
int off; /* current offset within data */
+ int startAttr;
+ int firstNullAttr;
+ int i;
+ bool hasnulls = HeapTupleHasNulls(tup);
- /* ----------------
- * Three cases:
- *
- * 1: No nulls and no variable-width attributes.
- * 2: Has a null or a var-width AFTER att.
- * 3: Has nulls or var-widths BEFORE att.
- * ----------------
- */
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
attnum--;
- if (!HeapTupleNoNulls(tup))
- {
- /*
- * there's a null somewhere in the tuple
- *
- * check to see if any preceding bits are null...
- */
- int byte = attnum >> 3;
- int finalbit = attnum & 0x07;
-
- /* check for nulls "before" final bit of last byte */
- if ((~bp[byte]) & ((1 << finalbit) - 1))
- slow = true;
- else
- {
- /* check for nulls in any "earlier" bytes */
- int i;
+ /*
+ * To reduce the number of attributes we need to look at, we start at the
+ * highest attribute that we can which has a cached offset. Since the
+ * attcacheoff for an attribute is only valid if there are no NULLs in
+ * prior attribute, we must look for NULLs to determine the start attr.
+ */
+ if (hasnulls)
+ firstNullAttr = first_null_attr(bp, attnum);
+ else
+ firstNullAttr = attnum;
- for (i = 0; i < byte; i++)
- {
- if (bp[i] != 0xFF)
- {
- slow = true;
- break;
- }
- }
- }
+ if (tupleDesc->firstNonCachedOffsetAttr > 0)
+ {
+ startAttr = Min(tupleDesc->firstNonCachedOffsetAttr - 1, firstNullAttr);
+ off = TupleDescCompactAttr(tupleDesc, startAttr)->attcacheoff;
+ }
+ else
+ {
+ startAttr = 0;
+ off = 0;
}
tp = (char *) td + td->t_hoff;
- if (!slow)
+ /*
+ * Calculate 'off' up to the first NULL attr. We use two cheaper loops
+ * when the tuple has no variable-width columns. When variable-width
+ * columns exist we use att_addlength_pointer() to move the offset beyond
+ * the current attribute.
+ */
+ if (!HeapTupleHasVarWidth(tup))
{
- CompactAttribute *att;
-
- /*
- * If we get here, there are no nulls up to and including the target
- * attribute. If we have a cached offset, we can use it.
- */
- att = TupleDescCompactAttr(tupleDesc, attnum);
- if (att->attcacheoff >= 0)
- return fetchatt(att, tp + att->attcacheoff);
-
- /*
- * Otherwise, check for non-fixed-length attrs up to and including
- * target. If there aren't any, it's safe to cheaply initialize the
- * cached offsets for these attrs.
- */
- if (HeapTupleHasVarWidth(tup))
+ for (i = startAttr; i < firstNullAttr; i++)
{
- int j;
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- for (j = 0; j <= attnum; j++)
- {
- if (TupleDescCompactAttr(tupleDesc, j)->attlen <= 0)
- {
- slow = true;
- break;
- }
- }
+ off = att_nominal_alignby(off, cattr->attalignby);
+ off += cattr->attlen;
}
- }
-
- if (!slow)
- {
- int natts = tupleDesc->natts;
- int j = 1;
-
- /*
- * If we get here, we have a tuple with no nulls or var-widths up to
- * and including the target attribute, so we can use the cached offset
- * ... only we don't have it yet, or we'd not have got here. Since
- * it's cheap to compute offsets for fixed-width columns, we take the
- * opportunity to initialize the cached offsets for *all* the leading
- * fixed-width columns, in hope of avoiding future visits to this
- * routine.
- */
- TupleDescCompactAttr(tupleDesc, 0)->attcacheoff = 0;
-
- /* we might have set some offsets in the slow path previously */
- while (j < natts && TupleDescCompactAttr(tupleDesc, j)->attcacheoff > 0)
- j++;
- off = TupleDescCompactAttr(tupleDesc, j - 1)->attcacheoff +
- TupleDescCompactAttr(tupleDesc, j - 1)->attlen;
-
- for (; j < natts; j++)
+ for (; i < attnum; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, j);
-
- if (att->attlen <= 0)
- break;
-
- off = att_nominal_alignby(off, att->attalignby);
+ if (att_isnull(i, bp))
+ continue;
- att->attcacheoff = off;
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- off += att->attlen;
+ off = att_pointer_alignby(off, cattr->attalignby, cattr->attlen,
+ tp + off);
+ off += cattr->attlen;
}
-
- Assert(j > attnum);
-
- off = TupleDescCompactAttr(tupleDesc, attnum)->attcacheoff;
}
else
{
- bool usecache = true;
- int i;
-
- /*
- * Now we know that we have to walk the tuple CAREFULLY. But we still
- * might be able to cache some offsets for next time.
- *
- * Note - This loop is a little tricky. For each non-null attribute,
- * we have to first account for alignment padding before the attr,
- * then advance over the attr based on its length. Nulls have no
- * storage and no alignment padding either. We can use/set
- * attcacheoff until we reach either a null or a var-width attribute.
- */
- off = 0;
- for (i = 0;; i++) /* loop exit is at "break" */
+ for (i = startAttr; i < firstNullAttr; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, i);
+ int attlen;
- if (HeapTupleHasNulls(tup) && att_isnull(i, bp))
- {
- usecache = false;
- continue; /* this cannot be the target att */
- }
+ cattr = TupleDescCompactAttr(tupleDesc, i);
+ attlen = cattr->attlen;
- /* If we know the next offset, we can skip the rest */
- if (usecache && att->attcacheoff >= 0)
- off = att->attcacheoff;
- else if (att->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be
- * no pad bytes in any case: then the offset will be valid for
- * either an aligned or unaligned value.
- */
- if (usecache &&
- off == att_nominal_alignby(off, att->attalignby))
- att->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, att->attalignby, -1,
- tp + off);
- usecache = false;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, att->attalignby);
+ /*
+ * cstrings don't exist in heap tuples. Use pg_assume to instruct
+ * the compiler not to emit the cstring related code in
+ * att_addlength_pointer().
+ */
+ pg_assume(attlen > 0 || attlen == -1);
- if (usecache)
- att->attcacheoff = off;
- }
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ attlen,
+ tp + off);
+ off = att_addlength_pointer(off, attlen, tp + off);
+ }
- if (i == attnum)
- break;
+ for (; i < attnum; i++)
+ {
+ int attlen;
- off = att_addlength_pointer(off, att->attlen, tp + off);
+ if (att_isnull(i, bp))
+ continue;
- if (usecache && att->attlen <= 0)
- usecache = false;
+ cattr = TupleDescCompactAttr(tupleDesc, i);
+ attlen = cattr->attlen;
+
+ /* As above, heaptuples have no cstrings */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ off = att_pointer_alignby(off, cattr->attalignby, attlen,
+ tp + off);
+ off = att_addlength_pointer(off, attlen, tp + off);
}
+
}
- return fetchatt(TupleDescCompactAttr(tupleDesc, attnum), tp + off);
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+
+ return fetchatt(cattr, tp + off);
}
/* ----------------
@@ -1347,6 +1264,7 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,
Datum *values, bool *isnull)
{
HeapTupleHeader tup = tuple->t_data;
+ CompactAttribute *cattr;
bool hasnulls = HeapTupleHasNulls(tuple);
int tdesc_natts = tupleDesc->natts;
int natts; /* number of atts to extract */
@@ -1354,70 +1272,98 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,
char *tp; /* ptr to tuple data */
uint32 off; /* offset in tuple data */
bits8 *bp = tup->t_bits; /* ptr to null bitmap in tuple */
- bool slow = false; /* can we use/set attcacheoff? */
+ int firstNonCacheOffsetAttr;
+ int firstNullAttr;
natts = HeapTupleHeaderGetNatts(tup);
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
+
/*
* In inheritance situations, it is possible that the given tuple actually
* has more fields than the caller is expecting. Don't run off the end of
* the caller's arrays.
*/
natts = Min(natts, tdesc_natts);
+ firstNonCacheOffsetAttr = Min(tupleDesc->firstNonCachedOffsetAttr, natts);
+
+ if (hasnulls)
+ {
+ firstNullAttr = first_null_attr(bp, natts);
+
+ /*
+ * XXX: it'd be nice to use populate_isnull_array() here, but that
+ * requires that the isnull array's size is rounded up to the next
+ * multiple of 8. Doing that would require adjusting many location
+ * that allocate the array.
+ */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, firstNullAttr);
+ }
+ else
+ firstNullAttr = natts;
tp = (char *) tup + tup->t_hoff;
+ attnum = 0;
- off = 0;
+ if (firstNonCacheOffsetAttr > 0)
+ {
+#ifdef USE_ASSERT_CHECKING
+ /* In Assert enabled builds, verify attcacheoff is correct */
+ int offcheck = 0;
+#endif
+ do
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ off = cattr->attcacheoff;
- for (attnum = 0; attnum < natts; attnum++)
+#ifdef USE_ASSERT_CHECKING
+ offcheck = att_nominal_alignby(offcheck, cattr->attalignby);
+ Assert(offcheck == cattr->attcacheoff);
+ offcheck += cattr->attlen;
+#endif
+
+ values[attnum] = fetch_att_noerr(tp + off,
+ cattr->attbyval,
+ cattr->attlen);
+ } while (++attnum < firstNonCacheOffsetAttr);
+ off += cattr->attlen;
+ }
+ else
+ off = 0;
+
+ for (; attnum < firstNullAttr; attnum++)
{
- CompactAttribute *thisatt = TupleDescCompactAttr(tupleDesc, attnum);
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
+ }
+
+ for (; attnum < natts; attnum++)
+ {
+ Assert(hasnulls);
- if (hasnulls && att_isnull(attnum, bp))
+ if (att_isnull(attnum, bp))
{
values[attnum] = (Datum) 0;
isnull[attnum] = true;
- slow = true; /* can't use attcacheoff anymore */
continue;
}
isnull[attnum] = false;
-
- if (!slow && thisatt->attcacheoff >= 0)
- off = thisatt->attcacheoff;
- else if (thisatt->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be no
- * pad bytes in any case: then the offset will be valid for either
- * an aligned or unaligned value.
- */
- if (!slow &&
- off == att_nominal_alignby(off, thisatt->attalignby))
- thisatt->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, thisatt->attalignby, -1,
- tp + off);
- slow = true;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, thisatt->attalignby);
-
- if (!slow)
- thisatt->attcacheoff = off;
- }
-
- values[attnum] = fetchatt(thisatt, tp + off);
-
- off = att_addlength_pointer(off, thisatt->attlen, tp + off);
-
- if (thisatt->attlen <= 0)
- slow = true; /* can't use attcacheoff anymore */
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
}
/*
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index d6350201e01..92282039671 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -223,18 +223,6 @@ index_form_tuple_context(TupleDesc tupleDescriptor,
*
* This gets called from index_getattr() macro, and only in cases
* where we can't use cacheoffset and the value is not null.
- *
- * This caches attribute offsets in the attribute descriptor.
- *
- * An alternative way to speed things up would be to cache offsets
- * with the tuple, but that seems more difficult unless you take
- * the storage hit of actually putting those offsets into the
- * tuple you send to disk. Yuck.
- *
- * This scheme will be slightly slower than that, but should
- * perform well for queries which hit large #'s of tuples. After
- * you cache the offsets once, examining all the other tuples using
- * the same attribute descriptor will go much quicker. -cim 5/4/91
* ----------------
*/
Datum
@@ -242,205 +230,126 @@ nocache_index_getattr(IndexTuple tup,
int attnum,
TupleDesc tupleDesc)
{
+ CompactAttribute *cattr;
char *tp; /* ptr to data part of tuple */
bits8 *bp = NULL; /* ptr to null bitmap in tuple */
- bool slow = false; /* do we have to walk attrs? */
int data_off; /* tuple data offset */
int off; /* current offset within data */
+ int startAttr;
+ int firstNullAttr;
+ bool hasnulls = IndexTupleHasNulls(tup);
+ int i;
- /* ----------------
- * Three cases:
- *
- * 1: No nulls and no variable-width attributes.
- * 2: Has a null or a var-width AFTER att.
- * 3: Has nulls or var-widths BEFORE att.
- * ----------------
- */
-
- data_off = IndexInfoFindDataOffset(tup->t_info);
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
attnum--;
- if (IndexTupleHasNulls(tup))
- {
- /*
- * there's a null somewhere in the tuple
- *
- * check to see if desired att is null
- */
+ data_off = IndexInfoFindDataOffset(tup->t_info);
+ tp = (char *) tup + data_off;
- /* XXX "knows" t_bits are just after fixed tuple header! */
+ /*
+ * To reduce the number of attributes we need to look at, we start at the
+ * highest attribute that we can which has a cached offset. Since the
+ * attcacheoff for an attribute is only valid if there are no NULLs in
+ * prior attribute, we must look for NULLs to determine the start attr.
+ */
+ if (hasnulls)
+ {
bp = (bits8 *) ((char *) tup + sizeof(IndexTupleData));
-
- /*
- * Now check to see if any preceding bits are null...
- */
- {
- int byte = attnum >> 3;
- int finalbit = attnum & 0x07;
-
- /* check for nulls "before" final bit of last byte */
- if ((~bp[byte]) & ((1 << finalbit) - 1))
- slow = true;
- else
- {
- /* check for nulls in any "earlier" bytes */
- int i;
-
- for (i = 0; i < byte; i++)
- {
- if (bp[i] != 0xFF)
- {
- slow = true;
- break;
- }
- }
- }
- }
+ firstNullAttr = first_null_attr(bp, attnum);
}
+ else
+ firstNullAttr = attnum;
- tp = (char *) tup + data_off;
-
- if (!slow)
+ if (tupleDesc->firstNonCachedOffsetAttr > 0)
{
- CompactAttribute *att;
-
- /*
- * If we get here, there are no nulls up to and including the target
- * attribute. If we have a cached offset, we can use it.
- */
- att = TupleDescCompactAttr(tupleDesc, attnum);
- if (att->attcacheoff >= 0)
- return fetchatt(att, tp + att->attcacheoff);
-
- /*
- * Otherwise, check for non-fixed-length attrs up to and including
- * target. If there aren't any, it's safe to cheaply initialize the
- * cached offsets for these attrs.
- */
- if (IndexTupleHasVarwidths(tup))
- {
- int j;
-
- for (j = 0; j <= attnum; j++)
- {
- if (TupleDescCompactAttr(tupleDesc, j)->attlen <= 0)
- {
- slow = true;
- break;
- }
- }
- }
+ startAttr = Min(tupleDesc->firstNonCachedOffsetAttr - 1, firstNullAttr);
+ off = TupleDescCompactAttr(tupleDesc, startAttr)->attcacheoff;
}
-
- if (!slow)
+ else
{
- int natts = tupleDesc->natts;
- int j = 1;
-
- /*
- * If we get here, we have a tuple with no nulls or var-widths up to
- * and including the target attribute, so we can use the cached offset
- * ... only we don't have it yet, or we'd not have got here. Since
- * it's cheap to compute offsets for fixed-width columns, we take the
- * opportunity to initialize the cached offsets for *all* the leading
- * fixed-width columns, in hope of avoiding future visits to this
- * routine.
- */
- TupleDescCompactAttr(tupleDesc, 0)->attcacheoff = 0;
+ startAttr = 0;
+ off = 0;
+ }
- /* we might have set some offsets in the slow path previously */
- while (j < natts && TupleDescCompactAttr(tupleDesc, j)->attcacheoff > 0)
- j++;
+ /*
+ * Calculate 'off' up to the first NULL attr. We use two cheaper loops
+ * when the tuple has no variable-width columns. When variable-width
+ * columns exist we use att_addlength_pointer() to move the offset beyond
+ * the current attribute.
+ */
+ if (IndexTupleHasVarwidths(tup))
+ {
+ /* Calculate the offset up until the first NULL */
+ for (i = startAttr; i < firstNullAttr; i++)
+ {
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- off = TupleDescCompactAttr(tupleDesc, j - 1)->attcacheoff +
- TupleDescCompactAttr(tupleDesc, j - 1)->attlen;
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+ off = att_addlength_pointer(off, cattr->attlen, tp + off);
+ }
- for (; j < natts; j++)
+ /* Calculate the offset for any remaining columns. */
+ for (; i < attnum; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, j);
+ Assert(hasnulls);
- if (att->attlen <= 0)
- break;
+ if (att_isnull(i, bp))
+ continue;
- off = att_nominal_alignby(off, att->attalignby);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- att->attcacheoff = off;
-
- off += att->attlen;
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+ off = att_addlength_pointer(off, cattr->attlen, tp + off);
}
-
- Assert(j > attnum);
-
- off = TupleDescCompactAttr(tupleDesc, attnum)->attcacheoff;
}
else
{
- bool usecache = true;
- int i;
+ /* Handle tuples with only fixed-width attributes */
- /*
- * Now we know that we have to walk the tuple CAREFULLY. But we still
- * might be able to cache some offsets for next time.
- *
- * Note - This loop is a little tricky. For each non-null attribute,
- * we have to first account for alignment padding before the attr,
- * then advance over the attr based on its length. Nulls have no
- * storage and no alignment padding either. We can use/set
- * attcacheoff until we reach either a null or a var-width attribute.
- */
- off = 0;
- for (i = 0;; i++) /* loop exit is at "break" */
+ /* Calculate the offset up until the first NULL */
+ for (i = startAttr; i < firstNullAttr; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, i);
-
- if (IndexTupleHasNulls(tup) && att_isnull(i, bp))
- {
- usecache = false;
- continue; /* this cannot be the target att */
- }
-
- /* If we know the next offset, we can skip the rest */
- if (usecache && att->attcacheoff >= 0)
- off = att->attcacheoff;
- else if (att->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be
- * no pad bytes in any case: then the offset will be valid for
- * either an aligned or unaligned value.
- */
- if (usecache &&
- off == att_nominal_alignby(off, att->attalignby))
- att->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, att->attalignby, -1,
- tp + off);
- usecache = false;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, att->attalignby);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
+
+ Assert(cattr->attlen > 0);
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+ off += cattr->attlen;
+ }
- if (usecache)
- att->attcacheoff = off;
- }
+ /* Calculate the offset for any remaining columns. */
+ for (; i < attnum; i++)
+ {
+ Assert(hasnulls);
- if (i == attnum)
- break;
+ if (att_isnull(i, bp))
+ continue;
- off = att_addlength_pointer(off, att->attlen, tp + off);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- if (usecache && att->attlen <= 0)
- usecache = false;
+ Assert(cattr->attlen > 0);
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+ off += cattr->attlen;
}
}
- return fetchatt(TupleDescCompactAttr(tupleDesc, attnum), tp + off);
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ off = att_pointer_alignby(off, cattr->attalignby,
+ cattr->attlen, tp + off);
+ return fetchatt(cattr, tp + off);
}
/*
@@ -480,63 +389,87 @@ index_deform_tuple_internal(TupleDesc tupleDescriptor,
Datum *values, bool *isnull,
char *tp, bits8 *bp, int hasnulls)
{
+ CompactAttribute *cattr;
int natts = tupleDescriptor->natts; /* number of atts to extract */
- int attnum;
- int off = 0; /* offset in tuple data */
- bool slow = false; /* can we use/set attcacheoff? */
+ int attnum = 0;
+ uint32 off = 0; /* offset in tuple data */
+ int firstNonCacheOffsetAttr;
+ int firstNullAttr;
/* Assert to protect callers who allocate fixed-size arrays */
Assert(natts <= INDEX_MAX_KEYS);
- for (attnum = 0; attnum < natts; attnum++)
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDescriptor->firstNonCachedOffsetAttr >= 0);
+
+ firstNonCacheOffsetAttr = Min(tupleDescriptor->firstNonCachedOffsetAttr, natts);
+
+ if (hasnulls)
+ {
+ firstNullAttr = first_null_attr(bp, natts);
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, firstNullAttr);
+ }
+ else
+ firstNullAttr = natts;
+
+ if (firstNonCacheOffsetAttr > 0)
+ {
+#ifdef USE_ASSERT_CHECKING
+ /* In Assert enabled builds, verify attcacheoff is correct */
+ off = 0;
+#endif
+
+ do
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDescriptor, attnum);
+
+#ifdef USE_ASSERT_CHECKING
+ off = att_nominal_alignby(off, cattr->attalignby);
+ Assert(off == cattr->attcacheoff);
+ off += cattr->attlen;
+#endif
+
+ values[attnum] = fetch_att_noerr(tp + cattr->attcacheoff, cattr->attbyval,
+ cattr->attlen);
+ } while (++attnum < firstNonCacheOffsetAttr);
+
+ off = cattr->attcacheoff + cattr->attlen;
+ }
+
+ for (; attnum < firstNullAttr; attnum++)
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDescriptor, attnum);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
+ }
+
+ for (; attnum < natts; attnum++)
{
- CompactAttribute *thisatt = TupleDescCompactAttr(tupleDescriptor, attnum);
+ Assert(hasnulls);
- if (hasnulls && att_isnull(attnum, bp))
+ if (att_isnull(attnum, bp))
{
values[attnum] = (Datum) 0;
isnull[attnum] = true;
- slow = true; /* can't use attcacheoff anymore */
continue;
}
isnull[attnum] = false;
-
- if (!slow && thisatt->attcacheoff >= 0)
- off = thisatt->attcacheoff;
- else if (thisatt->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be no
- * pad bytes in any case: then the offset will be valid for either
- * an aligned or unaligned value.
- */
- if (!slow &&
- off == att_nominal_alignby(off, thisatt->attalignby))
- thisatt->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, thisatt->attalignby, -1,
- tp + off);
- slow = true;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, thisatt->attalignby);
-
- if (!slow)
- thisatt->attcacheoff = off;
- }
-
- values[attnum] = fetchatt(thisatt, tp + off);
-
- off = att_addlength_pointer(off, thisatt->attlen, tp + off);
-
- if (thisatt->attlen <= 0)
- slow = true; /* can't use attcacheoff anymore */
+ cattr = TupleDescCompactAttr(tupleDescriptor, attnum);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
}
}
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 2137385a833..41085d43c85 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -197,6 +197,10 @@ CreateTemplateTupleDesc(int natts)
desc->tdtypmod = -1;
desc->tdrefcount = -1; /* assume not reference-counted */
+ /* This will be set to the correct value by TupleDescFinalize() */
+ desc->firstNonCachedOffsetAttr = -1;
+ desc->firstNonGuaranteedAttr = -1;
+
return desc;
}
@@ -457,6 +461,9 @@ TupleDescCopy(TupleDesc dst, TupleDesc src)
* descriptor to another.
*
* !!! Constraints and defaults are not copied !!!
+ *
+ * The caller must take care of calling TupleDescFinalize() on once all
+ * TupleDesc changes have been made.
*/
void
TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
@@ -489,6 +496,45 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
populate_compact_attribute(dst, dstAttno - 1);
}
+/*
+ * TupleDescFinalize
+ * Finalize the given TupleDesc. This must be called after the
+ * attributes arrays have been populated or adjusted by any code.
+ *
+ * Must be called after populate_compact_attribute() and before
+ * BlessTupleDesc().
+ */
+void
+TupleDescFinalize(TupleDesc tupdesc)
+{
+ int firstNonCachedOffsetAttr = 0;
+ int firstNonGuaranteedAttr = tupdesc->natts;
+ int off = 0;
+
+ for (int i = 0; i < tupdesc->natts; i++)
+ {
+ CompactAttribute *cattr = TupleDescCompactAttr(tupdesc, i);
+
+ if (firstNonGuaranteedAttr == tupdesc->natts &&
+ (cattr->attnullability != ATTNULLABLE_VALID || !cattr->attbyval ||
+ cattr->atthasmissing || cattr->attisdropped || cattr->attlen <= 0))
+ firstNonGuaranteedAttr = i;
+
+ if (cattr->attlen <= 0)
+ break;
+
+ off = att_nominal_alignby(off, cattr->attalignby);
+
+ cattr->attcacheoff = off;
+
+ off += cattr->attlen;
+ firstNonCachedOffsetAttr = i + 1;
+ }
+
+ tupdesc->firstNonCachedOffsetAttr = firstNonCachedOffsetAttr;
+ tupdesc->firstNonGuaranteedAttr = firstNonGuaranteedAttr;
+}
+
/*
* Free a TupleDesc including all substructure
*/
diff --git a/src/backend/access/spgist/spgutils.c b/src/backend/access/spgist/spgutils.c
index b246e8127db..a4694bd8065 100644
--- a/src/backend/access/spgist/spgutils.c
+++ b/src/backend/access/spgist/spgutils.c
@@ -335,9 +335,6 @@ getSpGistTupleDesc(Relation index, SpGistTypeDesc *keyType)
/* We shouldn't need to bother with making these valid: */
att->attcompression = InvalidCompressionMethod;
att->attcollation = InvalidOid;
- /* In case we changed typlen, we'd better reset following offsets */
- for (int i = spgFirstIncludeColumn; i < outTupDesc->natts; i++)
- TupleDescCompactAttr(outTupDesc, i)->attcacheoff = -1;
populate_compact_attribute(outTupDesc, spgKeyColumn);
TupleDescFinalize(outTupDesc);
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index e6ab51e6404..80faf29b797 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -993,218 +993,242 @@ tts_buffer_heap_store_tuple(TupleTableSlot *slot, HeapTuple tuple,
}
/*
- * slot_deform_heap_tuple_internal
- * An always inline helper function for use in slot_deform_heap_tuple to
- * allow the compiler to emit specialized versions of this function for
- * various combinations of "slow" and "hasnulls". For example, if a
- * given tuple has no nulls, then we needn't check "hasnulls" for every
- * attribute that we're deforming. The caller can just call this
- * function with hasnulls set to constant-false and have the compiler
- * remove the constant-false branches and emit more optimal code.
- *
- * Returns the next attnum to deform, which can be equal to natts when the
- * function manages to deform all requested attributes. *offp is an input and
- * output parameter which is the byte offset within the tuple to start deforming
- * from which, on return, gets set to the offset where the next attribute
- * should be deformed from. *slowp is set to true when subsequent deforming
- * of this tuple must use a version of this function with "slow" passed as
- * true.
- *
- * Callers cannot assume when we return "attnum" (i.e. all requested
- * attributes have been deformed) that slow mode isn't required for any
- * additional deforming as the final attribute may have caused a switch to
- * slow mode.
+ * slot_deform_heap_tuple
+ * Given a TupleTableSlot, extract data from the slot's physical tuple
+ * into its Datum/isnull arrays. Data is extracted up through the
+ * natts'th column (caller must ensure this is a legal column number).
+ *
+ * This is essentially an incremental version of heap_deform_tuple:
+ * on each call we extract attributes up to the one needed, without
+ * re-computing information about previously extracted attributes.
+ * slot->tts_nvalid is the number of attributes already extracted.
+ *
+ * This is marked as always inline, so the different offp for different types
+ * of slots gets optimized away.
*/
-static pg_attribute_always_inline int
-slot_deform_heap_tuple_internal(TupleTableSlot *slot, HeapTuple tuple,
- int attnum, int natts, bool slow,
- bool hasnulls, uint32 *offp, bool *slowp)
+static pg_attribute_always_inline void
+slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
+ int natts)
{
+ CompactAttribute *cattr;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
- Datum *values = slot->tts_values;
- bool *isnull = slot->tts_isnull;
HeapTupleHeader tup = tuple->t_data;
+ int attnum;
+ int firstNonCacheOffsetAttr;
+ int firstNonGuaranteedAttr;
+ int firstNullAttr;
+ Datum *values;
+ bool *isnull;
char *tp; /* ptr to tuple data */
- bits8 *bp = tup->t_bits; /* ptr to null bitmap in tuple */
- bool slownext = false;
+ uint32 off; /* offset in tuple data */
- tp = (char *) tup + tup->t_hoff;
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
- for (; attnum < natts; attnum++)
+ isnull = slot->tts_isnull;
+
+ /*
+ * Some callers may form and deform tuples prior to NOT NULL constraints
+ * being checked. Here we'd like to optimize the case where we only need
+ * to fetch attributes before or up to the point where the attribute is
+ * guaranteed to exist in the tuple. We rely on the slot flag being set
+ * correctly to only enable this optimization when it's valid to do so.
+ * This optimization allows us to save fetching the number of attributes
+ * from the tuple and saves the additional cost of handling non-byval
+ * attrs.
+ */
+ if (TTS_OBEYS_NOT_NULL_CONSTRAINTS(slot))
+ firstNonGuaranteedAttr = Min(natts, tupleDesc->firstNonGuaranteedAttr);
+ else
+ firstNonGuaranteedAttr = 0;
+
+ firstNonCacheOffsetAttr = tupleDesc->firstNonCachedOffsetAttr;
+
+ if (HeapTupleHasNulls(tuple))
{
- CompactAttribute *thisatt = TupleDescCompactAttr(tupleDesc, attnum);
+ int tupnatts = HeapTupleHeaderGetNatts(tup);
- if (hasnulls && att_isnull(attnum, bp))
+ tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits) +
+ BITMAPLEN(tupnatts));
+
+ natts = Min(tupnatts, natts);
+ if (natts > firstNonGuaranteedAttr)
{
- values[attnum] = (Datum) 0;
- isnull[attnum] = true;
- if (!slow)
- {
- *slowp = true;
- return attnum + 1;
- }
- else
- continue;
- }
+ bits8 *bp = tup->t_bits;
- isnull[attnum] = false;
+ /* Find the first NULL attr */
+ firstNullAttr = first_null_attr(bp, natts);
- /* calculate the offset of this attribute */
- if (!slow && thisatt->attcacheoff >= 0)
- *offp = thisatt->attcacheoff;
- else if (thisatt->attlen == -1)
- {
/*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be no
- * pad bytes in any case: then the offset will be valid for either
- * an aligned or unaligned value.
+ * And populate the isnull array for all attributes being fetched
+ * from the tuple.
*/
- if (!slow && *offp == att_nominal_alignby(*offp, thisatt->attalignby))
- thisatt->attcacheoff = *offp;
- else
- {
- *offp = att_pointer_alignby(*offp,
- thisatt->attalignby,
- -1,
- tp + *offp);
+ populate_isnull_array(bp, natts, isnull);
- if (!slow)
- slownext = true;
- }
+ /* We can only use any cached offsets until the first NULL attr */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, firstNullAttr);
}
else
{
- /* not varlena, so safe to use att_nominal_alignby */
- *offp = att_nominal_alignby(*offp, thisatt->attalignby);
-
- if (!slow)
- thisatt->attcacheoff = *offp;
+ /* Otherwise all required columns are guaranteed to exist */
+ firstNullAttr = natts;
}
+ }
+ else
+ {
+ tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits));
+
+ /*
+ * We only need to look at the tuple's natts if we need more than the
+ * guaranteed number of columns
+ */
+ if (natts > firstNonGuaranteedAttr)
+ natts = Min(HeapTupleHeaderGetNatts(tup), natts);
+
+ /* All attrs can be fetched without checking for NULLs */
+ firstNullAttr = natts;
+ }
- values[attnum] = fetchatt(thisatt, tp + *offp);
+ attnum = slot->tts_nvalid;
+ values = slot->tts_values;
+ slot->tts_nvalid = natts;
- *offp = att_addlength_pointer(*offp, thisatt->attlen, tp + *offp);
+ /* Ensure we calculated tp correctly */
+ Assert(tp == (char *) tup + tup->t_hoff);
- /* check if we need to switch to slow mode */
- if (!slow)
+ if (attnum < firstNonGuaranteedAttr)
+ {
+ do
{
+ int attlen;
+
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ attlen = cattr->attlen;
+
+ /* We don't expect any non-byval types */
+ pg_assume(attlen > 0);
+
/*
- * We're unable to deform any further if the above code set
- * 'slownext', or if this isn't a fixed-width attribute.
+ * Technically we could support non-byval fixed-width types, but
+ * not doing so allows us to pass true to fetch_att_noerr() which
+ * eliminates the !attbyval branch.
*/
- if (slownext || thisatt->attlen <= 0)
- {
- *slowp = true;
- return attnum + 1;
- }
- }
- }
+ Assert(cattr->attbyval == true);
- return natts;
-}
-
-/*
- * slot_deform_heap_tuple
- * Given a TupleTableSlot, extract data from the slot's physical tuple
- * into its Datum/isnull arrays. Data is extracted up through the
- * natts'th column (caller must ensure this is a legal column number).
- *
- * This is essentially an incremental version of heap_deform_tuple:
- * on each call we extract attributes up to the one needed, without
- * re-computing information about previously extracted attributes.
- * slot->tts_nvalid is the number of attributes already extracted.
- *
- * This is marked as always inline, so the different offp for different types
- * of slots gets optimized away.
- */
-static pg_attribute_always_inline void
-slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
- int natts)
-{
- bool hasnulls = HeapTupleHasNulls(tuple);
- int attnum;
- uint32 off; /* offset in tuple data */
- bool slow; /* can we use/set attcacheoff? */
+ off = cattr->attcacheoff;
+ values[attnum] = fetch_att_noerr(tp + off, true, attlen);
+ attnum++;
+ } while (attnum < firstNonGuaranteedAttr);
- /* We can only fetch as many attributes as the tuple has. */
- natts = Min(HeapTupleHeaderGetNatts(tuple->t_data), natts);
+ off += cattr->attlen;
- /*
- * Check whether the first call for this tuple, and initialize or restore
- * loop state.
- */
- attnum = slot->tts_nvalid;
- if (attnum == 0)
- {
- /* Start from the first attribute */
- off = 0;
- slow = false;
+ if (attnum == natts)
+ goto done;
}
else
{
/* Restore state from previous execution */
off = *offp;
- slow = TTS_SLOW(slot);
+
+ /* We expect *offp to be set to 0 when attnum == 0 */
+ Assert(off == 0 || attnum > 0);
}
+ /* We can only fetch as many attributes as the tuple has. */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, natts);
+
/*
- * If 'slow' isn't set, try deforming using deforming code that does not
- * contain any of the extra checks required for non-fixed offset
- * deforming. During deforming, if or when we find a NULL or a variable
- * length attribute, we'll switch to a deforming method which includes the
- * extra code required for non-fixed offset deforming, a.k.a slow mode.
- * Because this is performance critical, we inline
- * slot_deform_heap_tuple_internal passing the 'slow' and 'hasnull'
- * parameters as constants to allow the compiler to emit specialized code
- * with the known-const false comparisons and subsequent branches removed.
+ * Handle the portion of the tuple that we have cached the offset for up
+ * to the first NULL attribute. The offset is effectively fixed for these
+ * so we can use the CompactAttribute's attcacheoff.
*/
- if (!slow)
+ if (attnum < firstNonCacheOffsetAttr)
{
- /* Tuple without any NULLs? We can skip doing any NULL checking */
- if (!hasnulls)
- attnum = slot_deform_heap_tuple_internal(slot,
- tuple,
- attnum,
- natts,
- false, /* slow */
- false, /* hasnulls */
- &off,
- &slow);
- else
- attnum = slot_deform_heap_tuple_internal(slot,
- tuple,
- attnum,
- natts,
- false, /* slow */
- true, /* hasnulls */
- &off,
- &slow);
+ do
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+
+ off = cattr->attcacheoff;
+ values[attnum] = fetch_att_noerr(tp + off,
+ cattr->attbyval,
+ cattr->attlen);
+ } while (++attnum < firstNonCacheOffsetAttr);
+
+ /*
+ * Point the offset after the end of the last attribute with a cached
+ * offset. We expect the final cached offset attribute to have a
+ * fixed width, so just add the attlen to the attcacheoff
+ */
+ Assert(cattr->attlen > 0);
+ off += cattr->attlen;
+ }
+
+ /*
+ * Handle any portion of the tuple that doesn't have a fixed offset up
+ * until the first NULL attribute. This loops only differs from the one
+ * after it by the NULL checks.
+ */
+ for (; attnum < firstNullAttr; attnum++)
+ {
+ int attlen;
+
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ attlen = cattr->attlen;
+
+ /*
+ * cstrings don't exist in heap tuples. Use pg_assume to instruct the
+ * compiler not to emit the cstring related code in
+ * align_fetch_then_add().
+ */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ attlen,
+ cattr->attalignby);
}
- /* If there's still work to do then we must be in slow mode */
- if (attnum < natts)
+ /*
+ * Now handle any remaining attributes in the tuple up to the requested
+ * attnum. This time, include NULL checks as we're now at the first NULL
+ * attribute.
+ */
+ for (; attnum < natts; attnum++)
{
- /* XXX is it worth adding a separate call when hasnulls is false? */
- attnum = slot_deform_heap_tuple_internal(slot,
- tuple,
- attnum,
- natts,
- true, /* slow */
- hasnulls,
- &off,
- &slow);
+ int attlen;
+
+ if (isnull[attnum])
+ {
+ values[attnum] = (Datum) 0;
+ continue;
+ }
+
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ attlen = cattr->attlen;
+
+ /* As above, we don't expect cstrings */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ attlen,
+ cattr->attalignby);
}
+done:
+
/*
* Save state for next execution
*/
slot->tts_nvalid = attnum;
*offp = off;
- if (slow)
- slot->tts_flags |= TTS_FLAG_SLOW;
- else
- slot->tts_flags &= ~TTS_FLAG_SLOW;
}
const TupleTableSlotOps TTSOpsVirtual = {
@@ -1508,7 +1532,7 @@ ExecSetSlotDescriptor(TupleTableSlot *slot, /* slot to change */
slot->tts_values = (Datum *)
MemoryContextAlloc(slot->tts_mcxt, tupdesc->natts * sizeof(Datum));
slot->tts_isnull = (bool *)
- MemoryContextAlloc(slot->tts_mcxt, tupdesc->natts * sizeof(bool));
+ MemoryContextAlloc(slot->tts_mcxt, MAXALIGN(tupdesc->natts * sizeof(bool)));
}
/* --------------------------------
@@ -2259,10 +2283,16 @@ ExecTypeSetColNames(TupleDesc typeInfo, List *namesList)
* This happens "for free" if the tupdesc came from a relcache entry, but
* not if we have manufactured a tupdesc for a transient RECORD datatype.
* In that case we have to notify typcache.c of the existence of the type.
+ *
+ * TupleDescFinalize() must be called on the TupleDesc before calling this
+ * function.
*/
TupleDesc
BlessTupleDesc(TupleDesc tupdesc)
{
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupdesc->firstNonCachedOffsetAttr >= 0);
+
if (tupdesc->tdtypeid == RECORDOID &&
tupdesc->tdtypmod < 0)
assign_record_type_typmod(tupdesc);
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index af3c788ce8b..7f74a8ddcb2 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -246,6 +246,8 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags)
RelationGetDescr(scanstate->ss.ss_currentRelation),
table_slot_callbacks(scanstate->ss.ss_currentRelation));
+ scanstate->ss.ss_ScanTupleSlot->tts_flags |= TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
+
/*
* Initialize result type and projection.
*/
diff --git a/src/backend/jit/llvm/llvmjit_deform.c b/src/backend/jit/llvm/llvmjit_deform.c
index 3eb087eb56b..12521e3e46a 100644
--- a/src/backend/jit/llvm/llvmjit_deform.c
+++ b/src/backend/jit/llvm/llvmjit_deform.c
@@ -62,7 +62,6 @@ slot_compile_deform(LLVMJitContext *context, TupleDesc desc,
LLVMValueRef v_tts_values;
LLVMValueRef v_tts_nulls;
LLVMValueRef v_slotoffp;
- LLVMValueRef v_flagsp;
LLVMValueRef v_nvalidp;
LLVMValueRef v_nvalid;
LLVMValueRef v_maxatt;
@@ -178,7 +177,6 @@ slot_compile_deform(LLVMJitContext *context, TupleDesc desc,
v_tts_nulls =
l_load_struct_gep(b, StructTupleTableSlot, v_slot, FIELDNO_TUPLETABLESLOT_ISNULL,
"tts_ISNULL");
- v_flagsp = l_struct_gep(b, StructTupleTableSlot, v_slot, FIELDNO_TUPLETABLESLOT_FLAGS, "");
v_nvalidp = l_struct_gep(b, StructTupleTableSlot, v_slot, FIELDNO_TUPLETABLESLOT_NVALID, "");
if (ops == &TTSOpsHeapTuple || ops == &TTSOpsBufferHeapTuple)
@@ -747,14 +745,10 @@ slot_compile_deform(LLVMJitContext *context, TupleDesc desc,
{
LLVMValueRef v_off = l_load(b, TypeSizeT, v_offp, "");
- LLVMValueRef v_flags;
LLVMBuildStore(b, l_int16_const(lc, natts), v_nvalidp);
v_off = LLVMBuildTrunc(b, v_off, LLVMInt32TypeInContext(lc), "");
LLVMBuildStore(b, v_off, v_slotoffp);
- v_flags = l_load(b, LLVMInt16TypeInContext(lc), v_flagsp, "tts_flags");
- v_flags = LLVMBuildOr(b, v_flags, l_int16_const(lc, TTS_FLAG_SLOW), "");
- LLVMBuildStore(b, v_flags, v_flagsp);
LLVMBuildRetVoid(b);
}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 770edb34e08..998be24ac41 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -666,14 +666,6 @@ RelationBuildTupleDesc(Relation relation)
elog(ERROR, "pg_attribute catalog is missing %d attribute(s) for relation OID %u",
need, RelationGetRelid(relation));
- /*
- * We can easily set the attcacheoff value for the first attribute: it
- * must be zero. This eliminates the need for special cases for attnum=1
- * that used to exist in fastgetattr() and index_getattr().
- */
- if (RelationGetNumberOfAttributes(relation) > 0)
- TupleDescCompactAttr(relation->rd_att, 0)->attcacheoff = 0;
-
/*
* Set up constraint/default info
*/
@@ -1985,8 +1977,6 @@ formrdesc(const char *relationName, Oid relationReltype,
populate_compact_attribute(relation->rd_att, i);
}
- /* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
- TupleDescCompactAttr(relation->rd_att, 0)->attcacheoff = 0;
TupleDescFinalize(relation->rd_att);
/* mark not-null status */
@@ -4446,8 +4436,6 @@ BuildHardcodedDescriptor(int natts, const FormData_pg_attribute *attrs)
populate_compact_attribute(result, i);
}
- /* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
- TupleDescCompactAttr(result, 0)->attcacheoff = 0;
TupleDescFinalize(result);
/* Note: we don't bother to set up a TupleConstr entry */
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 595413dbbc5..80e1dd0e3c7 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -131,6 +131,19 @@ typedef struct CompactAttribute
* Any code making changes manually to and fields in the FormData_pg_attribute
* array must subsequently call populate_compact_attribute() to flush the
* changes out to the corresponding 'compact_attrs' element.
+ *
+ * firstNonCachedOffsetAttr stores the index into the compact_attrs array for
+ * the first attribute that we don't have a known attcacheoff for.
+ *
+ * firstNonGuaranteedAttr stores the index to into the compact_attrs array for
+ * the first attribute that is either NULLable, missing, or !attbyval. This
+ * can be used in locations as a guarantee that attributes before this will
+ * always exist in tuples. The !attbyval part isn't required for this, but
+ * including this allows various tuple deforming routines to forego any checks
+ * for !attbyval.
+ *
+ * Once a TupleDesc has been populated, before it is used for any purpose
+ * TupleDescFinalize() must be called on it.
*/
typedef struct TupleDescData
{
@@ -138,6 +151,10 @@ typedef struct TupleDescData
Oid tdtypeid; /* composite type ID for tuple type */
int32 tdtypmod; /* typmod for tuple type */
int tdrefcount; /* reference count, or -1 if not counting */
+ int firstNonCachedOffsetAttr; /* index of the first att without
+ * an attcacheoff */
+ int firstNonGuaranteedAttr; /* index of the first nullable,
+ * missing or !attbyval attribute. */
TupleConstr *constr; /* constraints, or NULL if none */
/* compact_attrs[N] is the compact metadata of Attribute Number N+1 */
CompactAttribute compact_attrs[FLEXIBLE_ARRAY_MEMBER];
@@ -195,7 +212,6 @@ extern TupleDesc CreateTupleDescTruncatedCopy(TupleDesc tupdesc, int natts);
extern TupleDesc CreateTupleDescCopyConstr(TupleDesc tupdesc);
-#define TupleDescFinalize(d) ((void) 0)
#define TupleDescSize(src) \
(offsetof(struct TupleDescData, compact_attrs) + \
(src)->natts * sizeof(CompactAttribute) + \
@@ -206,6 +222,7 @@ extern void TupleDescCopy(TupleDesc dst, TupleDesc src);
extern void TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
TupleDesc src, AttrNumber srcAttno);
+extern void TupleDescFinalize(TupleDesc tupdesc);
extern void FreeTupleDesc(TupleDesc tupdesc);
extern void IncrTupleDescRefCount(TupleDesc tupdesc);
diff --git a/src/include/access/tupmacs.h b/src/include/access/tupmacs.h
index d64c18b950b..4d97c27d872 100644
--- a/src/include/access/tupmacs.h
+++ b/src/include/access/tupmacs.h
@@ -15,7 +15,9 @@
#define TUPMACS_H
#include "catalog/pg_type_d.h" /* for TYPALIGN macros */
-
+#include "port/pg_bitutils.h"
+#include "port/pg_bswap.h"
+#include "varatt.h"
/*
* Check a tuple's null bitmap to determine whether the attribute is null.
@@ -28,6 +30,57 @@ att_isnull(int ATT, const bits8 *BITS)
return !(BITS[ATT >> 3] & (1 << (ATT & 0x07)));
}
+/*
+ * populate_isnull_array
+ * Transform a tuple's null bitmap into a boolean array.
+ *
+ * Caller must ensure that the isnull array is sized so it contains
+ * at least as many elements as there are bits in the 'bits' array.
+ * This is required because we always round 'natts' up to the next multiple
+ * of 8.
+ */
+static inline void
+populate_isnull_array(const bits8 *bits, int natts, bool *isnull)
+{
+ int nbytes = (natts + 7) >> 3;
+
+ /*
+ * Multiplying an inverted NULL bitmap byte by this value results in the
+ * lowest bit in each byte being set the same as each bit of the inverted
+ * byte. We perform this as 2 32-bit operations rather than a single
+ * 64-bit operation as multiplying by the required value to do this in
+ * 64-bits would result in overflowing a uint64 in some cases.
+ */
+#define SPREAD_BITS_MULTIPLIER_32 0x204081U
+
+ for (int i = 0; i < nbytes; i++, isnull += 8)
+ {
+ uint64 isnull_8;
+ bits8 nullbyte = ~bits[i];
+
+ /* convert the lower 4 bits of null bitmap word into 32 bit int */
+ isnull_8 = (nullbyte & 0xf) * SPREAD_BITS_MULTIPLIER_32;
+
+ /*
+ * convert the upper 4 bits of null bitmap word into 32 bit int, shift
+ * into the upper 32 bit
+ */
+ isnull_8 |= ((uint64) ((nullbyte >> 4) * SPREAD_BITS_MULTIPLIER_32)) << 32;
+
+ /* mask out all other bits apart from the lowest bit of each byte */
+ isnull_8 &= UINT64CONST(0x0101010101010101);
+
+#ifdef WORDS_BIGENDIAN
+
+ /*
+ * Fix byte order on big-endian machines before copying to the array.
+ */
+ isnull_8 = pg_bswap64(isnull_8);
+#endif
+ memcpy(isnull, &isnull_8, sizeof(uint64));
+ }
+}
+
#ifndef FRONTEND
/*
* Given an attbyval and an attlen from either a Form_pg_attribute or
@@ -69,6 +122,157 @@ fetch_att(const void *T, bool attbyval, int attlen)
else
return PointerGetDatum(T);
}
+
+/*
+ * Same, but no error checking for invalid attlens for byval types. This
+ * is safe to use when attlen comes from CompactAttribute as we validate the
+ * length when populating that struct.
+ */
+static inline Datum
+fetch_att_noerr(const void *T, bool attbyval, int attlen)
+{
+ if (attbyval)
+ {
+ switch (attlen)
+ {
+ case sizeof(int32):
+ return Int32GetDatum(*((const int32 *) T));
+ case sizeof(int16):
+ return Int16GetDatum(*((const int16 *) T));
+ case sizeof(char):
+ return CharGetDatum(*((const char *) T));
+ default:
+ Assert(attlen == sizeof(int64));
+ return Int64GetDatum(*((const int64 *) T));
+ }
+ }
+ else
+ return PointerGetDatum(T);
+}
+
+
+/*
+ * align_fetch_then_add
+ * Applies all the functionality of att_pointer_alignby(),
+ * fetch_att_noerr() and att_addlength_pointer() resulting in *off
+ * pointer to the perhaps unaligned number of bytes into 'tupptr', ready
+ * to deform the next attribute.
+ *
+ * tupptr: pointer to the beginning of the tuple, after the header and any
+ * NULL bitmask.
+ * off: offset in bytes for reading tuple data, possibly unaligned.
+ * attbyval, attlen, attalignby are values from CompactAttribute.
+ */
+static inline Datum
+align_fetch_then_add(const char *tupptr, uint32 *off, bool attbyval, int attlen,
+ uint8 attalignby)
+{
+ Datum res;
+
+ if (attlen > 0)
+ {
+ const char *offset_ptr;
+
+ *off = TYPEALIGN(attalignby, *off);
+ offset_ptr = tupptr + *off;
+ *off += attlen;
+ if (attbyval)
+ {
+ switch (attlen)
+ {
+ case sizeof(char):
+ return CharGetDatum(*((const char *) offset_ptr));
+ case sizeof(int16):
+ return Int16GetDatum(*((const int16 *) offset_ptr));
+ case sizeof(int32):
+ return Int32GetDatum(*((const int32 *) offset_ptr));
+ default:
+
+ /*
+ * populate_compact_attribute_internal() should have
+ * checked
+ */
+ Assert(attlen == sizeof(int64));
+ return Int64GetDatum(*((const int64 *) offset_ptr));
+ }
+ }
+ return PointerGetDatum(offset_ptr);
+ }
+ else if (attlen == -1)
+ {
+ if (!VARATT_IS_SHORT(tupptr + *off))
+ *off = TYPEALIGN(attalignby, *off);
+
+ res = PointerGetDatum(tupptr + *off);
+ *off += VARSIZE_ANY(DatumGetPointer(res));
+ return res;
+ }
+ else
+ {
+ Assert(attlen == -2);
+ *off = TYPEALIGN(attalignby, *off);
+ res = PointerGetDatum(tupptr + *off);
+ *off += strlen(tupptr + *off) + 1;
+ return res;
+ }
+}
+
+/*
+ * first_null_attr
+ * Inspect a NULL bitmask from a tuple and return the 0-based attnum of the
+ * first NULL attribute. Returns natts if no NULLs were found.
+ */
+static inline int
+first_null_attr(const bits8 *bits, int natts)
+{
+ int nattByte = natts >> 3;
+ int bytenum;
+ int res;
+
+#ifdef USE_ASSERT_CHECKING
+ int firstnull_check = natts;
+
+ /* Do it the slow way and check we get the same answer. */
+ for (int i = 0; i < natts; i++)
+ {
+ if (att_isnull(i, bits))
+ {
+ firstnull_check = i;
+ break;
+ }
+ }
+#endif
+
+ /* Process all bytes up to just before the byte for the natts attribute */
+ for (bytenum = 0; bytenum < nattByte; bytenum++)
+ {
+ /* break if there's any NULL attrs (a 0 bit) */
+ if (bits[bytenum] != 0xFF)
+ break;
+ }
+
+ /*
+ * Look for the highest 0-bit in the 'bytenum' element. To do this, we
+ * promote the uint8 to uint32 before performing the bitwise NOT and
+ * looking for the first 1-bit. This works even when the byte is 0xFF, as
+ * the bitwise NOT of 0xFF in 32 bits is 0xFFFFFF00, in which case
+ * pg_rightmost_one_pos32() will return 8. We may end up with a value
+ * higher than natts here, but we'll fix that with the Min() below.
+ */
+ res = bytenum << 3;
+ res += pg_rightmost_one_pos32(~((uint32) bits[bytenum]));
+
+ /*
+ * Since we did no masking to mask out bits beyond the natt'th bit, we may
+ * have found a bit higher than natts, so we must cap res to natts
+ */
+ res = Min(res, natts);
+
+ /* Ensure we got the same answer as the att_isnull() loop got */
+ Assert(res == firstnull_check);
+
+ return res;
+}
#endif /* FRONTEND */
/*
diff --git a/src/include/executor/tuptable.h b/src/include/executor/tuptable.h
index a2dfd707e78..8346be77302 100644
--- a/src/include/executor/tuptable.h
+++ b/src/include/executor/tuptable.h
@@ -84,9 +84,6 @@
* tts_values/tts_isnull are allocated either when the slot is created (when
* the descriptor is provided), or when a descriptor is assigned to the slot;
* they are of length equal to the descriptor's natts.
- *
- * The TTS_FLAG_SLOW flag is saved state for
- * slot_deform_heap_tuple, and should not be touched by any other code.
*----------
*/
@@ -98,12 +95,13 @@
#define TTS_FLAG_SHOULDFREE (1 << 2)
#define TTS_SHOULDFREE(slot) (((slot)->tts_flags & TTS_FLAG_SHOULDFREE) != 0)
-/* saved state for slot_deform_heap_tuple */
-#define TTS_FLAG_SLOW (1 << 3)
-#define TTS_SLOW(slot) (((slot)->tts_flags & TTS_FLAG_SLOW) != 0)
+/* true = formed tuple guaranteed to not have NULLs in NOT NULLable columns */
+#define TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS (1 << 3)
+#define TTS_OBEYS_NOT_NULL_CONSTRAINTS(slot) \
+ (((slot)->tts_flags & TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS) != 0)
/* fixed tuple descriptor */
-#define TTS_FLAG_FIXED (1 << 4)
+#define TTS_FLAG_FIXED (1 << 4) /* XXX change to #3? */
#define TTS_FIXED(slot) (((slot)->tts_flags & TTS_FLAG_FIXED) != 0)
struct TupleTableSlotOps;
@@ -123,7 +121,9 @@ typedef struct TupleTableSlot
#define FIELDNO_TUPLETABLESLOT_VALUES 5
Datum *tts_values; /* current per-attribute values */
#define FIELDNO_TUPLETABLESLOT_ISNULL 6
- bool *tts_isnull; /* current per-attribute isnull flags */
+ bool *tts_isnull; /* current per-attribute isnull flags. Array
+ * size must always be rounded up to the next
+ * multiple of 8 elements. */
MemoryContext tts_mcxt; /* slot itself is in this context */
ItemPointerData tts_tid; /* stored tuple's tid */
Oid tts_tableOid; /* table oid of tuple */
diff --git a/src/test/modules/deform_bench/deform_bench.c b/src/test/modules/deform_bench/deform_bench.c
index 7838f639bef..de39fecf8fd 100644
--- a/src/test/modules/deform_bench/deform_bench.c
+++ b/src/test/modules/deform_bench/deform_bench.c
@@ -49,6 +49,7 @@ deform_bench(PG_FUNCTION_ARGS)
tupdesc = RelationGetDescr(rel);
slot = MakeTupleTableSlot(tupdesc, &TTSOpsBufferHeapTuple);
+ slot->tts_flags |= TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
scan = table_beginscan_strat(rel, GetActiveSnapshot(), 0, NULL, true, false);
/*
--
2.51.0
From 8acf98c57a7ced3e081c911b47e1d6d644bc17db Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Mon, 16 Feb 2026 14:20:19 +1300
Subject: [PATCH v10 4/5] Allow sibling call optimization in
slot_getsomeattrs_int()
This changes the TupleTableSlotOps contract to make it so the
getsomeattrs() function is in charge of calling
slot_getmissingattrs().
Since this removes all code from slot_getsomeattrs_int() aside from the
getsomeattrs() call itself, we may as well adjust slot_getsomeattrs() so
that it calls getsomeattrs() directly. We leave slot_getsomeattrs_int()
intact as this is still called from the JIT code.
---
src/backend/executor/execTuples.c | 79 ++++++++++++++++---------------
src/include/executor/tuptable.h | 7 +--
2 files changed, 44 insertions(+), 42 deletions(-)
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 80faf29b797..2070c665d2f 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -73,7 +73,7 @@
static TupleDesc ExecTypeFromTLInternal(List *targetList,
bool skipjunk);
static pg_attribute_always_inline void slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
- int natts);
+ int reqnatts);
static inline void tts_buffer_heap_store_tuple(TupleTableSlot *slot,
HeapTuple tuple,
Buffer buffer,
@@ -996,7 +996,10 @@ tts_buffer_heap_store_tuple(TupleTableSlot *slot, HeapTuple tuple,
* slot_deform_heap_tuple
* Given a TupleTableSlot, extract data from the slot's physical tuple
* into its Datum/isnull arrays. Data is extracted up through the
- * natts'th column (caller must ensure this is a legal column number).
+ * reqnatts'th column. If there are insufficient attributes in the given
+ * tuple, then slot_getmissingattrs() is called to populate the
+ * remainder. If reqnatts is above the number of attributes in the
+ * slot's TupleDesc, an error is raised.
*
* This is essentially an incremental version of heap_deform_tuple:
* on each call we extract attributes up to the one needed, without
@@ -1008,7 +1011,7 @@ tts_buffer_heap_store_tuple(TupleTableSlot *slot, HeapTuple tuple,
*/
static pg_attribute_always_inline void
slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
- int natts)
+ int reqnatts)
{
CompactAttribute *cattr;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
@@ -1017,6 +1020,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int firstNonCacheOffsetAttr;
int firstNonGuaranteedAttr;
int firstNullAttr;
+ int natts;
Datum *values;
bool *isnull;
char *tp; /* ptr to tuple data */
@@ -1038,7 +1042,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
* attrs.
*/
if (TTS_OBEYS_NOT_NULL_CONSTRAINTS(slot))
- firstNonGuaranteedAttr = Min(natts, tupleDesc->firstNonGuaranteedAttr);
+ firstNonGuaranteedAttr = Min(reqnatts, tupleDesc->firstNonGuaranteedAttr);
else
firstNonGuaranteedAttr = 0;
@@ -1046,12 +1050,11 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
if (HeapTupleHasNulls(tuple))
{
- int tupnatts = HeapTupleHeaderGetNatts(tup);
-
+ natts = HeapTupleHeaderGetNatts(tup);
tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits) +
- BITMAPLEN(tupnatts));
+ BITMAPLEN(natts));
- natts = Min(tupnatts, natts);
+ natts = Min(natts, reqnatts);
if (natts > firstNonGuaranteedAttr)
{
bits8 *bp = tup->t_bits;
@@ -1082,8 +1085,13 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
* We only need to look at the tuple's natts if we need more than the
* guaranteed number of columns
*/
- if (natts > firstNonGuaranteedAttr)
- natts = Min(HeapTupleHeaderGetNatts(tup), natts);
+ if (reqnatts > firstNonGuaranteedAttr)
+ natts = Min(HeapTupleHeaderGetNatts(tup), reqnatts);
+ else
+ {
+ /* No need to access the number of attributes in the tuple */
+ natts = reqnatts;
+ }
/* All attrs can be fetched without checking for NULLs */
firstNullAttr = natts;
@@ -1091,7 +1099,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
attnum = slot->tts_nvalid;
values = slot->tts_values;
- slot->tts_nvalid = natts;
+ slot->tts_nvalid = reqnatts;
/* Ensure we calculated tp correctly */
Assert(tp == (char *) tup + tup->t_hoff);
@@ -1123,7 +1131,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
off += cattr->attlen;
- if (attnum == natts)
+ if (attnum == reqnatts)
goto done;
}
else
@@ -1222,12 +1230,12 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
cattr->attalignby);
}
+ /* Fetch any missing attrs and raise an error if reqnatts is invalid */
+ if (unlikely(attnum < reqnatts))
+ slot_getmissingattrs(slot, attnum, reqnatts);
done:
- /*
- * Save state for next execution
- */
- slot->tts_nvalid = attnum;
+ /* Save current offset for next execution */
*offp = off;
}
@@ -2088,28 +2096,29 @@ slot_getmissingattrs(TupleTableSlot *slot, int startAttNum, int lastAttNum)
if (!attrmiss)
{
/* no missing values array at all, so just fill everything in as NULL */
- memset(slot->tts_values + startAttNum, 0,
- (lastAttNum - startAttNum) * sizeof(Datum));
- memset(slot->tts_isnull + startAttNum, 1,
- (lastAttNum - startAttNum) * sizeof(bool));
+ for (int attnum = startAttNum; attnum < lastAttNum; attnum++)
+ {
+ slot->tts_values[attnum] = (Datum) 0;
+ slot->tts_isnull[attnum] = true;
+ }
}
else
{
- int missattnum;
-
- /* if there is a missing values array we must process them one by one */
- for (missattnum = startAttNum;
- missattnum < lastAttNum;
- missattnum++)
+ /* use attrmiss to set the missing values */
+ for (int attnum = startAttNum; attnum < lastAttNum; attnum++)
{
- slot->tts_values[missattnum] = attrmiss[missattnum].am_value;
- slot->tts_isnull[missattnum] = !attrmiss[missattnum].am_present;
+ slot->tts_values[attnum] = attrmiss[attnum].am_value;
+ slot->tts_isnull[attnum] = !attrmiss[attnum].am_present;
}
}
+
+ if (unlikely(lastAttNum > slot->tts_tupleDescriptor->natts))
+ elog(ERROR, "invalid attribute number %d", lastAttNum);
}
/*
- * slot_getsomeattrs_int - workhorse for slot_getsomeattrs()
+ * slot_getsomeattrs_int
+ * external function to call getsomeattrs() for use in JIT
*/
void
slot_getsomeattrs_int(TupleTableSlot *slot, int attnum)
@@ -2118,21 +2127,13 @@ slot_getsomeattrs_int(TupleTableSlot *slot, int attnum)
Assert(slot->tts_nvalid < attnum); /* checked in slot_getsomeattrs */
Assert(attnum > 0);
- if (unlikely(attnum > slot->tts_tupleDescriptor->natts))
- elog(ERROR, "invalid attribute number %d", attnum);
-
/* Fetch as many attributes as possible from the underlying tuple. */
slot->tts_ops->getsomeattrs(slot, attnum);
/*
- * If the underlying tuple doesn't have enough attributes, tuple
- * descriptor must have the missing attributes.
+ * Avoid putting new code here as that would prevent the compiler from
+ * using the sibling call optimization for the above function.
*/
- if (unlikely(slot->tts_nvalid < attnum))
- {
- slot_getmissingattrs(slot, slot->tts_nvalid, attnum);
- slot->tts_nvalid = attnum;
- }
}
/* ----------------------------------------------------------------
diff --git a/src/include/executor/tuptable.h b/src/include/executor/tuptable.h
index 8346be77302..1922c912089 100644
--- a/src/include/executor/tuptable.h
+++ b/src/include/executor/tuptable.h
@@ -153,8 +153,8 @@ struct TupleTableSlotOps
* Fill up first natts entries of tts_values and tts_isnull arrays with
* values from the tuple contained in the slot. The function may be called
* with natts more than the number of attributes available in the tuple,
- * in which case it should set tts_nvalid to the number of returned
- * columns.
+ * in which case the function must call slot_getmissingattrs() to populate
+ * the remaining attributes.
*/
void (*getsomeattrs) (TupleTableSlot *slot, int natts);
@@ -357,8 +357,9 @@ extern void slot_getsomeattrs_int(TupleTableSlot *slot, int attnum);
static inline void
slot_getsomeattrs(TupleTableSlot *slot, int attnum)
{
+ /* Populate slot with attributes up to 'attnum', if it's not already */
if (slot->tts_nvalid < attnum)
- slot_getsomeattrs_int(slot, attnum);
+ slot->tts_ops->getsomeattrs(slot, attnum);
}
/*
--
2.51.0
From 3749a0f9d298f5bd7e1ac06195a6a25fd10b63a5 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Mon, 23 Feb 2026 09:39:37 +1300
Subject: [PATCH v10 5/5] Reduce size of CompactAttribute struct to 8 bytes
Previously, this was 16 bytes. With the use of some bitflags and by
reducing the attcacheoff field size to a 16-bit type, we can halve the
size of the struct.
It's unlikely that caching the offsets for offsets larger than what will
fit in a 16-bit int will help much as the tuple is very likely to have
some non-fixed-width types anyway, the offsets of which we cannot cache.
---
src/backend/access/common/tupdesc.c | 10 ++++++++++
src/backend/executor/execTuples.c | 16 ++++++++++++----
src/include/access/tupdesc.h | 16 ++++++++--------
3 files changed, 30 insertions(+), 12 deletions(-)
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 41085d43c85..ca84913f3ad 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -525,6 +525,16 @@ TupleDescFinalize(TupleDesc tupdesc)
off = att_nominal_alignby(off, cattr->attalignby);
+ /*
+ * attcacheoff is an int16, so don't try and cache any offsets larger
+ * than will fit in that type. Any attributes which are offset more
+ * than 2^15 are likely due to variable length attributes. Since we
+ * don't cache offsets for or beyond variable length attributes, using
+ * an int16 rather than an int32 here is unlikely to cost us anything.
+ */
+ if (off > PG_INT16_MAX)
+ break;
+
cattr->attcacheoff = off;
off += cattr->attlen;
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 2070c665d2f..b3699b77649 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -1013,6 +1013,7 @@ static pg_attribute_always_inline void
slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int reqnatts)
{
+ CompactAttribute *cattrs;
CompactAttribute *cattr;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
HeapTupleHeader tup = tuple->t_data;
@@ -1101,6 +1102,13 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
values = slot->tts_values;
slot->tts_nvalid = reqnatts;
+ /*
+ * We store the tupleDesc's CompactAttribute array in 'cattrs' as gcc
+ * seems to be unwilling to optimize accessing the CompactAttribute
+ * element efficiently when accessing it via TupleDescCompactAttr().
+ */
+ cattrs = tupleDesc->compact_attrs;
+
/* Ensure we calculated tp correctly */
Assert(tp == (char *) tup + tup->t_hoff);
@@ -1111,7 +1119,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int attlen;
isnull[attnum] = false;
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
attlen = cattr->attlen;
/* We don't expect any non-byval types */
@@ -1156,7 +1164,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
do
{
isnull[attnum] = false;
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
off = cattr->attcacheoff;
values[attnum] = fetch_att_noerr(tp + off,
@@ -1183,7 +1191,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int attlen;
isnull[attnum] = false;
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
attlen = cattr->attlen;
/*
@@ -1216,7 +1224,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
continue;
}
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
attlen = cattr->attlen;
/* As above, we don't expect cstrings */
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 80e1dd0e3c7..9c5eac49172 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -55,7 +55,7 @@ typedef struct TupleConstr
* directly after the FormData_pg_attribute struct is populated or
* altered in any way.
*
- * Currently, this struct is 16 bytes. Any code changes which enlarge this
+ * Currently, this struct is 8 bytes. Any code changes which enlarge this
* struct should be considered very carefully.
*
* Code which must access a TupleDesc's attribute data should always make use
@@ -67,17 +67,17 @@ typedef struct TupleConstr
*/
typedef struct CompactAttribute
{
- int32 attcacheoff; /* fixed offset into tuple, if known, or -1 */
+ int16 attcacheoff; /* fixed offset into tuple, if known, or -1 */
int16 attlen; /* attr len in bytes or -1 = varlen, -2 =
* cstring */
bool attbyval; /* as FormData_pg_attribute.attbyval */
- bool attispackable; /* FormData_pg_attribute.attstorage !=
- * TYPSTORAGE_PLAIN */
- bool atthasmissing; /* as FormData_pg_attribute.atthasmissing */
- bool attisdropped; /* as FormData_pg_attribute.attisdropped */
- bool attgenerated; /* FormData_pg_attribute.attgenerated != '\0' */
- char attnullability; /* status of not-null constraint, see below */
uint8 attalignby; /* alignment requirement in bytes */
+ bool attispackable:1; /* FormData_pg_attribute.attstorage !=
+ * TYPSTORAGE_PLAIN */
+ bool atthasmissing:1; /* as FormData_pg_attribute.atthasmissing */
+ bool attisdropped:1; /* as FormData_pg_attribute.attisdropped */
+ bool attgenerated:1; /* FormData_pg_attribute.attgenerated != '\0' */
+ char attnullability; /* status of not-null constraint, see below */
} CompactAttribute;
/* Valid values for CompactAttribute->attnullability */
--
2.51.0
Attachments:
[text/plain] v10-0001-Introduce-deform_bench-test-module.patch (7.3K, 2-v10-0001-Introduce-deform_bench-test-module.patch)
download | inline diff:
From b9e6d0cbe19c2ea5506edef13d0ad960f824eae0 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Tue, 27 Jan 2026 15:08:09 +1300
Subject: [PATCH v10 1/5] Introduce deform_bench test module
For benchmarking tuple deformation.
---
src/test/modules/deform_bench/.gitignore | 4 +
src/test/modules/deform_bench/Makefile | 21 ++++
.../deform_bench/deform_bench--1.0.sql | 8 ++
src/test/modules/deform_bench/deform_bench.c | 107 ++++++++++++++++++
.../modules/deform_bench/deform_bench.control | 4 +
src/test/modules/deform_bench/meson.build | 22 ++++
src/test/modules/meson.build | 1 +
7 files changed, 167 insertions(+)
create mode 100644 src/test/modules/deform_bench/.gitignore
create mode 100644 src/test/modules/deform_bench/Makefile
create mode 100644 src/test/modules/deform_bench/deform_bench--1.0.sql
create mode 100644 src/test/modules/deform_bench/deform_bench.c
create mode 100644 src/test/modules/deform_bench/deform_bench.control
create mode 100644 src/test/modules/deform_bench/meson.build
diff --git a/src/test/modules/deform_bench/.gitignore b/src/test/modules/deform_bench/.gitignore
new file mode 100644
index 00000000000..5dcb3ff9723
--- /dev/null
+++ b/src/test/modules/deform_bench/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/src/test/modules/deform_bench/Makefile b/src/test/modules/deform_bench/Makefile
new file mode 100644
index 00000000000..b5fc0f7a583
--- /dev/null
+++ b/src/test/modules/deform_bench/Makefile
@@ -0,0 +1,21 @@
+# src/test/modules/deform_bench/Makefile
+
+MODULE_big = deform_bench
+OBJS = deform_bench.o
+
+EXTENSION = deform_bench
+DATA = deform_bench--1.0.sql
+PGFILEDESC = "deform_bench - tuple deform benchmarking"
+
+REGRESS = deform_bench
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = src/test/modules/deform_bench
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/src/test/modules/deform_bench/deform_bench--1.0.sql b/src/test/modules/deform_bench/deform_bench--1.0.sql
new file mode 100644
index 00000000000..492b71dba3b
--- /dev/null
+++ b/src/test/modules/deform_bench/deform_bench--1.0.sql
@@ -0,0 +1,8 @@
+/* deform_bench--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION deform_bench" to load this file. \quit
+
+CREATE FUNCTION deform_bench(tableoid Oid, attnum int[]) RETURNS FLOAT
+AS 'MODULE_PATHNAME', 'deform_bench'
+LANGUAGE C VOLATILE STRICT;
diff --git a/src/test/modules/deform_bench/deform_bench.c b/src/test/modules/deform_bench/deform_bench.c
new file mode 100644
index 00000000000..7838f639bef
--- /dev/null
+++ b/src/test/modules/deform_bench/deform_bench.c
@@ -0,0 +1,107 @@
+/*-------------------------------------------------------------------------
+ *
+ * deform_bench.c
+ *
+ * for benchmarking tuple deformation routines
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include <time.h>
+#include <sys/time.h>
+
+#include "access/heapam.h"
+#include "access/relscan.h"
+#include "catalog/pg_am_d.h"
+#include "catalog/pg_type_d.h"
+#include "funcapi.h"
+#include "miscadmin.h"
+#include "utils/array.h"
+#include "utils/arrayaccess.h"
+#include "utils/builtins.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(deform_bench);
+
+Datum
+deform_bench(PG_FUNCTION_ARGS)
+{
+ Oid tableoid = PG_GETARG_OID(0);
+ ArrayType *array = PG_GETARG_ARRAYTYPE_P(1);
+ TableScanDesc scan;
+ Relation rel;
+ TupleDesc tupdesc;
+ TupleTableSlot *slot;
+ Datum *elem_datums = NULL;
+ bool *elem_nulls = NULL;
+ int elem_count;
+ int *attnums;
+ clock_t start,
+ end;
+
+ rel = relation_open(tableoid, AccessShareLock);
+
+ if (rel->rd_rel->relam != HEAP_TABLE_AM_OID)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("only heap AM is supported")));
+
+ tupdesc = RelationGetDescr(rel);
+ slot = MakeTupleTableSlot(tupdesc, &TTSOpsBufferHeapTuple);
+ scan = table_beginscan_strat(rel, GetActiveSnapshot(), 0, NULL, true, false);
+
+ /*
+ * The array is used to allow callers to define how many atts to deform.
+ * e.g: '{1,10}'::int[] would deform attnum=1, then in a 2nd pass deform
+ * the remainder up to attnum=10. Passing an element as NULL means all
+ * attnums. This allows simulation of incremental deformation. Generally
+ * if you're passing an array with more than 1 element, then the array
+ * should be in ascending order. Doing something like '{10,1}' would mean
+ * we've already deformed 10 attributes and on the 2nd pass there's
+ * nothing to do since attnum=1 was already deformed in the first pass.
+ *
+ * You'll get an ERROR if you pass a number higher than the number of
+ * attributes in the table.
+ */
+ deconstruct_array(array,
+ INT4OID,
+ sizeof(int32),
+ true,
+ 'i',
+ &elem_datums,
+ &elem_nulls,
+ &elem_count);
+
+ attnums = palloc_array(int, elem_count);
+
+ for (int i = 0; i < elem_count; i++)
+ {
+ /* Make a NULL element mean all attributes */
+ if (elem_nulls[i])
+ attnums[i] = tupdesc->natts;
+ else
+ attnums[i] = DatumGetInt32(elem_datums[i]);
+ }
+
+ start = clock();
+
+ while (heap_getnextslot(scan, ForwardScanDirection, slot))
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ /* Deform in stages according to the attnums array */
+ for (int i = 0; i < elem_count; i++)
+ slot_getsomeattrs(slot, attnums[i]);
+ }
+
+ end = clock();
+
+ ExecDropSingleTupleTableSlot(slot);
+ table_endscan(scan);
+ relation_close(rel, AccessShareLock);
+
+
+ /* Returns the number of milliseconds to run the test */
+ PG_RETURN_FLOAT8((double) (end - start) / (CLOCKS_PER_SEC / 1000));
+}
diff --git a/src/test/modules/deform_bench/deform_bench.control b/src/test/modules/deform_bench/deform_bench.control
new file mode 100644
index 00000000000..a2023f9d738
--- /dev/null
+++ b/src/test/modules/deform_bench/deform_bench.control
@@ -0,0 +1,4 @@
+# deform_bench extension
+comment = 'functions for benchmarking tuple deformation'
+default_version = '1.0'
+module_pathname = '$libdir/deform_bench'
diff --git a/src/test/modules/deform_bench/meson.build b/src/test/modules/deform_bench/meson.build
new file mode 100644
index 00000000000..82049585244
--- /dev/null
+++ b/src/test/modules/deform_bench/meson.build
@@ -0,0 +1,22 @@
+# Copyright (c) 2026, PostgreSQL Global Development Group
+
+deform_bench_sources = files(
+ 'deform_bench.c',
+)
+
+if host_system == 'windows'
+ deform_bench_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+ '--NAME', 'deform_bench',
+ '--FILEDESC', 'deform_bench - benchmarking tuple deformation',])
+endif
+
+deform_bench = shared_module('deform_bench',
+ deform_bench_sources,
+ kwargs: pg_test_mod_args,
+)
+test_install_libs += deform_bench
+
+test_install_data += files(
+ 'deform_bench--1.0.sql',
+ 'deform_bench.control',
+)
diff --git a/src/test/modules/meson.build b/src/test/modules/meson.build
index 2634a519935..ef2b0af4581 100644
--- a/src/test/modules/meson.build
+++ b/src/test/modules/meson.build
@@ -2,6 +2,7 @@
subdir('brin')
subdir('commit_ts')
+subdir('deform_bench')
subdir('delay_execution')
subdir('dummy_index_am')
subdir('dummy_seclabel')
--
2.51.0
[text/plain] v10-0002-Add-empty-TupleDescFinalize-function.patch (29.0K, 3-v10-0002-Add-empty-TupleDescFinalize-function.patch)
download | inline diff:
From 04de30bdc8833148513fe07a0b9c7eb1e06c6b81 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Wed, 21 Jan 2026 15:41:37 +1300
Subject: [PATCH v10 2/5] Add empty TupleDescFinalize() function
Currently does nothing, but will in a future commit.
---
contrib/dblink/dblink.c | 4 ++++
contrib/pg_buffercache/pg_buffercache_pages.c | 2 ++
contrib/pg_visibility/pg_visibility.c | 2 ++
src/backend/access/brin/brin_tuple.c | 1 +
src/backend/access/common/tupdesc.c | 13 +++++++++++++
src/backend/access/gin/ginutil.c | 1 +
src/backend/access/gist/gistscan.c | 1 +
src/backend/access/spgist/spgutils.c | 1 +
src/backend/access/transam/twophase.c | 1 +
src/backend/access/transam/xlogfuncs.c | 1 +
src/backend/backup/basebackup_copy.c | 3 +++
src/backend/catalog/index.c | 2 ++
src/backend/catalog/pg_publication.c | 1 +
src/backend/catalog/toasting.c | 6 ++++++
src/backend/commands/explain.c | 1 +
src/backend/commands/functioncmds.c | 1 +
src/backend/commands/sequence.c | 1 +
src/backend/commands/tablecmds.c | 4 ++++
src/backend/commands/wait.c | 1 +
src/backend/executor/execSRF.c | 2 ++
src/backend/executor/execTuples.c | 4 ++++
src/backend/executor/nodeFunctionscan.c | 2 ++
src/backend/parser/parse_relation.c | 4 +++-
src/backend/parser/parse_target.c | 2 ++
.../replication/libpqwalreceiver/libpqwalreceiver.c | 1 +
src/backend/replication/walsender.c | 5 +++++
src/backend/utils/adt/acl.c | 1 +
src/backend/utils/adt/genfile.c | 1 +
src/backend/utils/adt/lockfuncs.c | 1 +
src/backend/utils/adt/orderedsetaggs.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 5 +++++
src/backend/utils/adt/tsvector_op.c | 1 +
src/backend/utils/cache/relcache.c | 8 ++++++++
src/backend/utils/fmgr/funcapi.c | 6 ++++++
src/backend/utils/misc/guc_funcs.c | 5 +++++
src/include/access/tupdesc.h | 1 +
src/pl/plpgsql/src/pl_comp.c | 2 ++
.../test_custom_stats/test_custom_fixed_stats.c | 1 +
src/test/modules/test_predtest/test_predtest.c | 1 +
39 files changed, 100 insertions(+), 1 deletion(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 2498d80c8e7..4038950a6ef 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -881,6 +881,7 @@ materializeResult(FunctionCallInfo fcinfo, PGconn *conn, PGresult *res)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "status",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
ntuples = 1;
nfields = 1;
}
@@ -1044,6 +1045,7 @@ materializeQueryResult(FunctionCallInfo fcinfo,
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "status",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
attinmeta = TupleDescGetAttInMetadata(tupdesc);
oldcontext = MemoryContextSwitchTo(rsinfo->econtext->ecxt_per_query_memory);
@@ -1529,6 +1531,8 @@ dblink_get_pkey(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 2, "colname",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+
/*
* Generate attribute metadata needed later to produce tuples from raw
* C strings
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 89b86855243..a6b4fb5252b 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -174,6 +174,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
INT4OID, -1, 0);
+ TupleDescFinalize(tupledesc);
fctx->tupdesc = BlessTupleDesc(tupledesc);
/* Allocate NBuffers worth of BufferCachePagesRec records. */
@@ -442,6 +443,7 @@ pg_buffercache_os_pages_internal(FunctionCallInfo fcinfo, bool include_numa)
TupleDescInitEntry(tupledesc, (AttrNumber) 3, "numa_node",
INT4OID, -1, 0);
+ TupleDescFinalize(tupledesc);
fctx->tupdesc = BlessTupleDesc(tupledesc);
fctx->include_numa = include_numa;
diff --git a/contrib/pg_visibility/pg_visibility.c b/contrib/pg_visibility/pg_visibility.c
index 9bc3a784bf7..dfab0b64cf5 100644
--- a/contrib/pg_visibility/pg_visibility.c
+++ b/contrib/pg_visibility/pg_visibility.c
@@ -469,6 +469,8 @@ pg_visibility_tupdesc(bool include_blkno, bool include_pd)
TupleDescInitEntry(tupdesc, ++a, "pd_all_visible", BOOLOID, -1, 0);
Assert(a == maxattr);
+ TupleDescFinalize(tupdesc);
+
return BlessTupleDesc(tupdesc);
}
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 69c233c62eb..742ac089a28 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -84,6 +84,7 @@ brtuple_disk_tupdesc(BrinDesc *brdesc)
MemoryContextSwitchTo(oldcxt);
+ TupleDescFinalize(tupdesc);
brdesc->bd_disktdesc = tupdesc;
}
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index b69d10f0a45..2137385a833 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -221,6 +221,9 @@ CreateTupleDesc(int natts, Form_pg_attribute *attrs)
memcpy(TupleDescAttr(desc, i), attrs[i], ATTRIBUTE_FIXED_PART_SIZE);
populate_compact_attribute(desc, i);
}
+
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -265,6 +268,8 @@ CreateTupleDescCopy(TupleDesc tupdesc)
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -311,6 +316,8 @@ CreateTupleDescTruncatedCopy(TupleDesc tupdesc, int natts)
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -396,6 +403,8 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -438,6 +447,8 @@ TupleDescCopy(TupleDesc dst, TupleDesc src)
* source's refcount would be wrong in any case.)
*/
dst->tdrefcount = -1;
+
+ TupleDescFinalize(dst);
}
/*
@@ -1065,6 +1076,8 @@ BuildDescFromLists(const List *names, const List *types, const List *typmods, co
TupleDescInitEntryCollation(desc, attnum, attcollation);
}
+ TupleDescFinalize(desc);
+
return desc;
}
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index ff927279cc3..fe7b984ff32 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -129,6 +129,7 @@ initGinState(GinState *state, Relation index)
attr->attndims);
TupleDescInitEntryCollation(state->tupdesc[i], (AttrNumber) 2,
attr->attcollation);
+ TupleDescFinalize(state->tupdesc[i]);
}
/*
diff --git a/src/backend/access/gist/gistscan.c b/src/backend/access/gist/gistscan.c
index f23bc4a6757..c65f93abdae 100644
--- a/src/backend/access/gist/gistscan.c
+++ b/src/backend/access/gist/gistscan.c
@@ -201,6 +201,7 @@ gistrescan(IndexScanDesc scan, ScanKey key, int nkeys,
attno - 1)->atttypid,
-1, 0);
}
+ TupleDescFinalize(so->giststate->fetchTupdesc);
scan->xs_hitupdesc = so->giststate->fetchTupdesc;
/* Also create a memory context that will hold the returned tuples */
diff --git a/src/backend/access/spgist/spgutils.c b/src/backend/access/spgist/spgutils.c
index 9f5379b87ac..b246e8127db 100644
--- a/src/backend/access/spgist/spgutils.c
+++ b/src/backend/access/spgist/spgutils.c
@@ -340,6 +340,7 @@ getSpGistTupleDesc(Relation index, SpGistTypeDesc *keyType)
TupleDescCompactAttr(outTupDesc, i)->attcacheoff = -1;
populate_compact_attribute(outTupDesc, spgKeyColumn);
+ TupleDescFinalize(outTupDesc);
}
return outTupDesc;
}
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index e4340b59640..7f4ed02a6b9 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -744,6 +744,7 @@ pg_prepared_xact(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 5, "dbid",
OIDOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
/*
diff --git a/src/backend/access/transam/xlogfuncs.c b/src/backend/access/transam/xlogfuncs.c
index 2efe4105efb..b6bc616c74c 100644
--- a/src/backend/access/transam/xlogfuncs.c
+++ b/src/backend/access/transam/xlogfuncs.c
@@ -400,6 +400,7 @@ pg_walfile_name_offset(PG_FUNCTION_ARGS)
TupleDescInitEntry(resultTupleDesc, (AttrNumber) 2, "file_offset",
INT4OID, -1, 0);
+ TupleDescFinalize(resultTupleDesc);
resultTupleDesc = BlessTupleDesc(resultTupleDesc);
/*
diff --git a/src/backend/backup/basebackup_copy.c b/src/backend/backup/basebackup_copy.c
index 07f58b39d8c..6c3453efd80 100644
--- a/src/backend/backup/basebackup_copy.c
+++ b/src/backend/backup/basebackup_copy.c
@@ -357,6 +357,8 @@ SendXlogRecPtrResult(XLogRecPtr ptr, TimeLineID tli)
*/
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "tli", INT8OID, -1, 0);
+ TupleDescFinalize(tupdesc);
+
/* send RowDescription */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -388,6 +390,7 @@ SendTablespaceList(List *tablespaces)
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 1, "spcoid", OIDOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "spclocation", TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 3, "size", INT8OID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* send RowDescription */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 43de42ce39e..75e97fb394a 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -481,6 +481,8 @@ ConstructTupleDescriptor(Relation heapRelation,
populate_compact_attribute(indexTupDesc, i);
}
+ TupleDescFinalize(indexTupDesc);
+
return indexTupDesc;
}
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index 9a4791c573e..fa353a0dd37 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -1230,6 +1230,7 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 4, "qual",
PG_NODE_TREEOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
funcctx->user_fctx = table_infos;
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index c78dcea98c1..078a1cf5127 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -229,6 +229,12 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attcompression = InvalidCompressionMethod;
TupleDescAttr(tupdesc, 2)->attcompression = InvalidCompressionMethod;
+ populate_compact_attribute(tupdesc, 0);
+ populate_compact_attribute(tupdesc, 1);
+ populate_compact_attribute(tupdesc, 2);
+
+ TupleDescFinalize(tupdesc);
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 93918a223b8..5f922c3f5c2 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -281,6 +281,7 @@ ExplainResultDesc(ExplainStmt *stmt)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "QUERY PLAN",
result_type, -1, 0);
+ TupleDescFinalize(tupdesc);
return tupdesc;
}
diff --git a/src/backend/commands/functioncmds.c b/src/backend/commands/functioncmds.c
index 242372b1e68..3afd762e9dc 100644
--- a/src/backend/commands/functioncmds.c
+++ b/src/backend/commands/functioncmds.c
@@ -2424,6 +2424,7 @@ CallStmtResultDesc(CallStmt *stmt)
-1,
0);
}
+ TupleDescFinalize(tupdesc);
}
return tupdesc;
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index e1b808bbb60..551667650ba 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -1808,6 +1808,7 @@ pg_get_sequence_data(PG_FUNCTION_ARGS)
BOOLOID, -1, 0);
TupleDescInitEntry(resultTupleDesc, (AttrNumber) 3, "page_lsn",
LSNOID, -1, 0);
+ TupleDescFinalize(resultTupleDesc);
resultTupleDesc = BlessTupleDesc(resultTupleDesc);
seqrel = try_relation_open(relid, AccessShareLock);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index b04b0dbd2a0..8678cecd53f 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -1030,6 +1030,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
}
}
+ TupleDescFinalize(descriptor);
+
/*
* For relations with table AM and partitioned tables, select access
* method to use: an explicitly indicated one, or (in the case of a
@@ -1458,6 +1460,8 @@ BuildDescForRelation(const List *columns)
populate_compact_attribute(desc, attnum - 1);
}
+ TupleDescFinalize(desc);
+
return desc;
}
diff --git a/src/backend/commands/wait.c b/src/backend/commands/wait.c
index 1290df10c6f..8e920a72372 100644
--- a/src/backend/commands/wait.c
+++ b/src/backend/commands/wait.c
@@ -338,5 +338,6 @@ WaitStmtResultDesc(WaitStmt *stmt)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "status",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
return tupdesc;
}
diff --git a/src/backend/executor/execSRF.c b/src/backend/executor/execSRF.c
index a0b111dc0e4..b481e50acfb 100644
--- a/src/backend/executor/execSRF.c
+++ b/src/backend/executor/execSRF.c
@@ -272,6 +272,7 @@ ExecMakeTableFunctionResult(SetExprState *setexpr,
funcrettype,
-1,
0);
+ TupleDescFinalize(tupdesc);
rsinfo.setDesc = tupdesc;
}
MemoryContextSwitchTo(oldcontext);
@@ -776,6 +777,7 @@ init_sexpr(Oid foid, Oid input_collation, Expr *node,
funcrettype,
-1,
0);
+ TupleDescFinalize(tupdesc);
sexpr->funcResultDesc = tupdesc;
sexpr->funcReturnsTuple = false;
}
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index b768eae9e53..e6ab51e6404 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -2173,6 +2173,8 @@ ExecTypeFromTLInternal(List *targetList, bool skipjunk)
cur_resno++;
}
+ TupleDescFinalize(typeInfo);
+
return typeInfo;
}
@@ -2207,6 +2209,8 @@ ExecTypeFromExprList(List *exprList)
cur_resno++;
}
+ TupleDescFinalize(typeInfo);
+
return typeInfo;
}
diff --git a/src/backend/executor/nodeFunctionscan.c b/src/backend/executor/nodeFunctionscan.c
index 63e605e1f81..feb82d64967 100644
--- a/src/backend/executor/nodeFunctionscan.c
+++ b/src/backend/executor/nodeFunctionscan.c
@@ -414,6 +414,7 @@ ExecInitFunctionScan(FunctionScan *node, EState *estate, int eflags)
TupleDescInitEntryCollation(tupdesc,
(AttrNumber) 1,
exprCollation(funcexpr));
+ TupleDescFinalize(tupdesc);
}
else
{
@@ -485,6 +486,7 @@ ExecInitFunctionScan(FunctionScan *node, EState *estate, int eflags)
0);
}
+ TupleDescFinalize(scan_tupdesc);
Assert(attno == natts);
}
diff --git a/src/backend/parser/parse_relation.c b/src/backend/parser/parse_relation.c
index e003db520de..9c415e166ee 100644
--- a/src/backend/parser/parse_relation.c
+++ b/src/backend/parser/parse_relation.c
@@ -1883,6 +1883,7 @@ addRangeTableEntryForFunction(ParseState *pstate,
TupleDescInitEntryCollation(tupdesc,
(AttrNumber) 1,
exprCollation(funcexpr));
+ TupleDescFinalize(tupdesc);
}
else if (functypclass == TYPEFUNC_RECORD)
{
@@ -1940,6 +1941,7 @@ addRangeTableEntryForFunction(ParseState *pstate,
i++;
}
+ TupleDescFinalize(tupdesc);
/*
* Ensure that the coldeflist defines a legal set of names (no
@@ -2008,7 +2010,7 @@ addRangeTableEntryForFunction(ParseState *pstate,
0);
/* no need to set collation */
}
-
+ TupleDescFinalize(tupdesc);
Assert(natts == totalatts);
}
else
diff --git a/src/backend/parser/parse_target.c b/src/backend/parser/parse_target.c
index dbf5b2b5c01..a03d82c0540 100644
--- a/src/backend/parser/parse_target.c
+++ b/src/backend/parser/parse_target.c
@@ -1572,6 +1572,8 @@ expandRecordVariable(ParseState *pstate, Var *var, int levelsup)
}
Assert(lname == NULL && lvar == NULL); /* lists same length? */
+ TupleDescFinalize(tupleDesc);
+
return tupleDesc;
}
diff --git a/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c b/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
index 7c8639b32e9..9f04c9ed25d 100644
--- a/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
+++ b/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
@@ -1073,6 +1073,7 @@ libpqrcv_processTuples(PGresult *pgres, WalRcvExecResult *walres,
for (coln = 0; coln < nRetTypes; coln++)
TupleDescInitEntry(walres->tupledesc, (AttrNumber) coln + 1,
PQfname(pgres, coln), retTypes[coln], -1, 0);
+ TupleDescFinalize(walres->tupledesc);
attinmeta = TupleDescGetAttInMetadata(walres->tupledesc);
/* No point in doing more here if there were no tuples returned. */
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c
index 2cde8ebc729..33a9e8d7f21 100644
--- a/src/backend/replication/walsender.c
+++ b/src/backend/replication/walsender.c
@@ -451,6 +451,7 @@ IdentifySystem(void)
TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 4, "dbname",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -496,6 +497,7 @@ ReadReplicationSlot(ReadReplicationSlotCmd *cmd)
/* TimeLineID is unsigned, so int4 is not wide enough. */
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 3, "restart_tli",
INT8OID, -1, 0);
+ TupleDescFinalize(tupdesc);
memset(nulls, true, READ_REPLICATION_SLOT_COLS * sizeof(bool));
@@ -598,6 +600,7 @@ SendTimeLineHistory(TimeLineHistoryCmd *cmd)
tupdesc = CreateTemplateTupleDesc(2);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 1, "filename", TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "content", TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
TLHistoryFileName(histfname, cmd->timeline);
TLHistoryFilePath(path, cmd->timeline);
@@ -1015,6 +1018,7 @@ StartReplication(StartReplicationCmd *cmd)
INT8OID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "next_tli_startpos",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuple */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -1369,6 +1373,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd)
TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 4, "output_plugin",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
diff --git a/src/backend/utils/adt/acl.c b/src/backend/utils/adt/acl.c
index 641673f0b0e..ce07f2bc046 100644
--- a/src/backend/utils/adt/acl.c
+++ b/src/backend/utils/adt/acl.c
@@ -1819,6 +1819,7 @@ aclexplode(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 4, "is_grantable",
BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
/* allocate memory for user context */
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index c083608b1d5..bfb949401d0 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -454,6 +454,7 @@ pg_stat_file(PG_FUNCTION_ARGS)
"creation", TIMESTAMPTZOID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 6,
"isdir", BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
memset(isnull, false, sizeof(isnull));
diff --git a/src/backend/utils/adt/lockfuncs.c b/src/backend/utils/adt/lockfuncs.c
index 9dadd6da672..4481c354fd6 100644
--- a/src/backend/utils/adt/lockfuncs.c
+++ b/src/backend/utils/adt/lockfuncs.c
@@ -146,6 +146,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 16, "waitstart",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
/*
diff --git a/src/backend/utils/adt/orderedsetaggs.c b/src/backend/utils/adt/orderedsetaggs.c
index 3b6da8e36ac..fd8b8676470 100644
--- a/src/backend/utils/adt/orderedsetaggs.c
+++ b/src/backend/utils/adt/orderedsetaggs.c
@@ -233,6 +233,7 @@ ordered_set_startup(FunctionCallInfo fcinfo, bool use_tuples)
-1,
0);
+ TupleDescFinalize(newdesc);
FreeTupleDesc(qstate->tupdesc);
qstate->tupdesc = newdesc;
}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index b1df96e7b0b..0b10da3b180 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -769,6 +769,7 @@ pg_stat_get_backend_subxact(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 2, "subxact_overflow",
BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
if ((local_beentry = pgstat_get_local_beentry_by_proc_number(procNumber)) != NULL)
@@ -1670,6 +1671,7 @@ pg_stat_wal_build_tuple(PgStat_WalCounters wal_counters,
TupleDescInitEntry(tupdesc, (AttrNumber) 6, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
/* Fill values and NULLs */
@@ -2097,6 +2099,7 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 7, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
/* Get statistics about the archiver process */
@@ -2178,6 +2181,7 @@ pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
TIMESTAMPTZOID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 13, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
namestrcpy(&slotname, text_to_cstring(slotname_text));
@@ -2265,6 +2269,7 @@ pg_stat_get_subscription_stats(PG_FUNCTION_ARGS)
INT8OID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 13, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
if (!subentry)
diff --git a/src/backend/utils/adt/tsvector_op.c b/src/backend/utils/adt/tsvector_op.c
index 71c7c7d3b3c..d8dece42b9b 100644
--- a/src/backend/utils/adt/tsvector_op.c
+++ b/src/backend/utils/adt/tsvector_op.c
@@ -651,6 +651,7 @@ tsvector_unnest(PG_FUNCTION_ARGS)
TEXTARRAYOID, -1, 0);
if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
elog(ERROR, "return type must be a row type");
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = tupdesc;
funcctx->user_fctx = PG_GETARG_TSVECTOR_COPY(0);
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 6b634c9fff1..770edb34e08 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -729,6 +729,8 @@ RelationBuildTupleDesc(Relation relation)
pfree(constr);
relation->rd_att->constr = NULL;
}
+
+ TupleDescFinalize(relation->rd_att);
}
/*
@@ -1985,6 +1987,7 @@ formrdesc(const char *relationName, Oid relationReltype,
/* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
TupleDescCompactAttr(relation->rd_att, 0)->attcacheoff = 0;
+ TupleDescFinalize(relation->rd_att);
/* mark not-null status */
if (has_not_null)
@@ -3688,6 +3691,8 @@ RelationBuildLocalRelation(const char *relname,
for (i = 0; i < natts; i++)
TupleDescAttr(rel->rd_att, i)->attrelid = relid;
+ TupleDescFinalize(rel->rd_att);
+
rel->rd_rel->reltablespace = reltablespace;
if (mapped_relation)
@@ -4443,6 +4448,7 @@ BuildHardcodedDescriptor(int natts, const FormData_pg_attribute *attrs)
/* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
TupleDescCompactAttr(result, 0)->attcacheoff = 0;
+ TupleDescFinalize(result);
/* Note: we don't bother to set up a TupleConstr entry */
@@ -6268,6 +6274,8 @@ load_relcache_init_file(bool shared)
populate_compact_attribute(rel->rd_att, i);
}
+ TupleDescFinalize(rel->rd_att);
+
/* next read the access method specific field */
if (fread(&len, 1, sizeof(len), fp) != sizeof(len))
goto read_failed;
diff --git a/src/backend/utils/fmgr/funcapi.c b/src/backend/utils/fmgr/funcapi.c
index 8a934ea8dca..516d02cfb82 100644
--- a/src/backend/utils/fmgr/funcapi.c
+++ b/src/backend/utils/fmgr/funcapi.c
@@ -340,6 +340,8 @@ get_expr_result_type(Node *expr,
exprCollation(col));
i++;
}
+ TupleDescFinalize(tupdesc);
+
if (resultTypeId)
*resultTypeId = rexpr->row_typeid;
if (resultTupleDesc)
@@ -1044,6 +1046,7 @@ resolve_polymorphic_tupdesc(TupleDesc tupdesc, oidvector *declared_args,
}
}
+ TupleDescFinalize(tupdesc);
return true;
}
@@ -1853,6 +1856,8 @@ build_function_result_tupdesc_d(char prokind,
0);
}
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -1970,6 +1975,7 @@ TypeGetTupleDesc(Oid typeoid, List *colaliases)
typeoid,
-1,
0);
+ TupleDescFinalize(tupdesc);
}
else if (functypclass == TYPEFUNC_RECORD)
{
diff --git a/src/backend/utils/misc/guc_funcs.c b/src/backend/utils/misc/guc_funcs.c
index 8524dd3a981..472cb5393ce 100644
--- a/src/backend/utils/misc/guc_funcs.c
+++ b/src/backend/utils/misc/guc_funcs.c
@@ -444,6 +444,7 @@ GetPGVariableResultDesc(const char *name)
TupleDescInitEntry(tupdesc, (AttrNumber) 1, varname,
TEXTOID, -1, 0);
}
+ TupleDescFinalize(tupdesc);
return tupdesc;
}
@@ -465,6 +466,7 @@ ShowGUCConfigOption(const char *name, DestReceiver *dest)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 1, varname,
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -499,6 +501,7 @@ ShowAllGUCConfig(DestReceiver *dest)
TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 3, "description",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -934,6 +937,8 @@ show_all_settings(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 17, "pending_restart",
BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+
/*
* Generate attribute metadata needed later to produce tuples from raw
* C strings
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index d46cdbf7a3c..595413dbbc5 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -195,6 +195,7 @@ extern TupleDesc CreateTupleDescTruncatedCopy(TupleDesc tupdesc, int natts);
extern TupleDesc CreateTupleDescCopyConstr(TupleDesc tupdesc);
+#define TupleDescFinalize(d) ((void) 0)
#define TupleDescSize(src) \
(offsetof(struct TupleDescData, compact_attrs) + \
(src)->natts * sizeof(CompactAttribute) + \
diff --git a/src/pl/plpgsql/src/pl_comp.c b/src/pl/plpgsql/src/pl_comp.c
index 5ecc7766757..b72c963b3be 100644
--- a/src/pl/plpgsql/src/pl_comp.c
+++ b/src/pl/plpgsql/src/pl_comp.c
@@ -1912,6 +1912,8 @@ build_row_from_vars(PLpgSQL_variable **vars, int numvars)
TupleDescInitEntryCollation(row->rowtupdesc, i + 1, typcoll);
}
+ TupleDescFinalize(row->rowtupdesc);
+
return row;
}
diff --git a/src/test/modules/test_custom_stats/test_custom_fixed_stats.c b/src/test/modules/test_custom_stats/test_custom_fixed_stats.c
index 485e08e5c19..f9e7c717280 100644
--- a/src/test/modules/test_custom_stats/test_custom_fixed_stats.c
+++ b/src/test/modules/test_custom_stats/test_custom_fixed_stats.c
@@ -206,6 +206,7 @@ test_custom_stats_fixed_report(PG_FUNCTION_ARGS)
INT8OID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 2, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
values[0] = Int64GetDatum(stats->numcalls);
diff --git a/src/test/modules/test_predtest/test_predtest.c b/src/test/modules/test_predtest/test_predtest.c
index 679a5de456d..48ca2a4ea70 100644
--- a/src/test/modules/test_predtest/test_predtest.c
+++ b/src/test/modules/test_predtest/test_predtest.c
@@ -230,6 +230,7 @@ test_predtest(PG_FUNCTION_ARGS)
"s_r_holds", BOOLOID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 8,
"w_r_holds", BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
tupdesc = BlessTupleDesc(tupdesc);
values[0] = BoolGetDatum(strong_implied_by);
--
2.51.0
[text/plain] v10-0003-Optimize-tuple-deformation.patch (60.4K, 4-v10-0003-Optimize-tuple-deformation.patch)
download | inline diff:
From 46c7e9dd888247866b0b2931b87d8a7cfec9bb1b Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Tue, 31 Dec 2024 09:19:24 +1300
Subject: [PATCH v10 3/5] Optimize tuple deformation
This commit includes various optimizations to improve the performance of
tuple deformation.
We now precalculate CompactAttribute's attcacheoff, which allows us to
remove the code from the deform routines which was setting the
attcacheoff. Setting the attcacheoff is handled by TupleDescFinalize(),
which must be called before the TupleDesc is used for anything. Having
this TupleDescFinalize() function means we can store the first
attribute in the TupleDesc which does not have an offset cached. That
allows us to add a dedicated deforming loop to deform all attributes up
to the final one with an attcacheoff set, or up to the first NULL
attribute, whichever comes first.
We also record the maximum attribute number which is guaranteed to exist
in the tuple, that is, has a NOT NULL constraint and isn't an
atthasmissing attribute. When deforming only attributes prior to the
guaranteed attnum, we've no need to access the tuple's natt count. As an
additional optimization, we only count fixed-width columns when
calculating the maximum guaranteed column as this eliminates the need to
emit code to fetch byref types in the deformation loop for guaranteed
attributes.
Some locations in the code deform tuples that have yet to go through NOT
NULL constraint validation. We're unable to perform the guaranteed
attribute optimization when that's the case. The optimization is opt-in
via the TupleTableSlot using the TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS
flag.
This commit also adds a more efficient way of populating the isnull
array by using a bit-wise trick which performs multiplication on the
inverse of the tuple's bitmap byte and masking out all but the lower bit
of each of the boolean's byte. This results in much more optimal code
when compared to determining the NULLness via att_isnull(). 8 isnull
elements are processed at once using this method, which means we need to
round the tts_isnull array size up to the next 8 bytes. The palloc code
does this anyway, but the round-up needed to be formalized so as not to
overwrite the sentinel byte in debug builds.
---
src/backend/access/common/heaptuple.c | 362 ++++++++----------
src/backend/access/common/indextuple.c | 371 ++++++++-----------
src/backend/access/common/tupdesc.c | 46 +++
src/backend/access/spgist/spgutils.c | 3 -
src/backend/executor/execTuples.c | 360 +++++++++---------
src/backend/executor/nodeSeqscan.c | 2 +
src/backend/jit/llvm/llvmjit_deform.c | 6 -
src/backend/utils/cache/relcache.c | 12 -
src/include/access/tupdesc.h | 19 +-
src/include/access/tupmacs.h | 206 +++++++++-
src/include/executor/tuptable.h | 16 +-
src/test/modules/deform_bench/deform_bench.c | 1 +
12 files changed, 781 insertions(+), 623 deletions(-)
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 11bec20e82e..606c1f67568 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -498,19 +498,7 @@ heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc)
* nocachegetattr
*
* This only gets called from fastgetattr(), in cases where we
- * can't use a cacheoffset and the value is not null.
- *
- * This caches attribute offsets in the attribute descriptor.
- *
- * An alternative way to speed things up would be to cache offsets
- * with the tuple, but that seems more difficult unless you take
- * the storage hit of actually putting those offsets into the
- * tuple you send to disk. Yuck.
- *
- * This scheme will be slightly slower than that, but should
- * perform well for queries which hit large #'s of tuples. After
- * you cache the offsets once, examining all the other tuples using
- * the same attribute descriptor will go much quicker. -cim 5/4/91
+ * can't use the attcacheoff and the value is not null.
*
* NOTE: if you need to change this code, see also heap_deform_tuple.
* Also see nocache_index_getattr, which is the same code for index
@@ -522,194 +510,123 @@ nocachegetattr(HeapTuple tup,
int attnum,
TupleDesc tupleDesc)
{
+ CompactAttribute *cattr;
HeapTupleHeader td = tup->t_data;
char *tp; /* ptr to data part of tuple */
bits8 *bp = td->t_bits; /* ptr to null bitmap in tuple */
- bool slow = false; /* do we have to walk attrs? */
int off; /* current offset within data */
+ int startAttr;
+ int firstNullAttr;
+ int i;
+ bool hasnulls = HeapTupleHasNulls(tup);
- /* ----------------
- * Three cases:
- *
- * 1: No nulls and no variable-width attributes.
- * 2: Has a null or a var-width AFTER att.
- * 3: Has nulls or var-widths BEFORE att.
- * ----------------
- */
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
attnum--;
- if (!HeapTupleNoNulls(tup))
- {
- /*
- * there's a null somewhere in the tuple
- *
- * check to see if any preceding bits are null...
- */
- int byte = attnum >> 3;
- int finalbit = attnum & 0x07;
-
- /* check for nulls "before" final bit of last byte */
- if ((~bp[byte]) & ((1 << finalbit) - 1))
- slow = true;
- else
- {
- /* check for nulls in any "earlier" bytes */
- int i;
+ /*
+ * To reduce the number of attributes we need to look at, we start at the
+ * highest attribute that we can which has a cached offset. Since the
+ * attcacheoff for an attribute is only valid if there are no NULLs in
+ * prior attribute, we must look for NULLs to determine the start attr.
+ */
+ if (hasnulls)
+ firstNullAttr = first_null_attr(bp, attnum);
+ else
+ firstNullAttr = attnum;
- for (i = 0; i < byte; i++)
- {
- if (bp[i] != 0xFF)
- {
- slow = true;
- break;
- }
- }
- }
+ if (tupleDesc->firstNonCachedOffsetAttr > 0)
+ {
+ startAttr = Min(tupleDesc->firstNonCachedOffsetAttr - 1, firstNullAttr);
+ off = TupleDescCompactAttr(tupleDesc, startAttr)->attcacheoff;
+ }
+ else
+ {
+ startAttr = 0;
+ off = 0;
}
tp = (char *) td + td->t_hoff;
- if (!slow)
+ /*
+ * Calculate 'off' up to the first NULL attr. We use two cheaper loops
+ * when the tuple has no variable-width columns. When variable-width
+ * columns exist we use att_addlength_pointer() to move the offset beyond
+ * the current attribute.
+ */
+ if (!HeapTupleHasVarWidth(tup))
{
- CompactAttribute *att;
-
- /*
- * If we get here, there are no nulls up to and including the target
- * attribute. If we have a cached offset, we can use it.
- */
- att = TupleDescCompactAttr(tupleDesc, attnum);
- if (att->attcacheoff >= 0)
- return fetchatt(att, tp + att->attcacheoff);
-
- /*
- * Otherwise, check for non-fixed-length attrs up to and including
- * target. If there aren't any, it's safe to cheaply initialize the
- * cached offsets for these attrs.
- */
- if (HeapTupleHasVarWidth(tup))
+ for (i = startAttr; i < firstNullAttr; i++)
{
- int j;
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- for (j = 0; j <= attnum; j++)
- {
- if (TupleDescCompactAttr(tupleDesc, j)->attlen <= 0)
- {
- slow = true;
- break;
- }
- }
+ off = att_nominal_alignby(off, cattr->attalignby);
+ off += cattr->attlen;
}
- }
-
- if (!slow)
- {
- int natts = tupleDesc->natts;
- int j = 1;
-
- /*
- * If we get here, we have a tuple with no nulls or var-widths up to
- * and including the target attribute, so we can use the cached offset
- * ... only we don't have it yet, or we'd not have got here. Since
- * it's cheap to compute offsets for fixed-width columns, we take the
- * opportunity to initialize the cached offsets for *all* the leading
- * fixed-width columns, in hope of avoiding future visits to this
- * routine.
- */
- TupleDescCompactAttr(tupleDesc, 0)->attcacheoff = 0;
-
- /* we might have set some offsets in the slow path previously */
- while (j < natts && TupleDescCompactAttr(tupleDesc, j)->attcacheoff > 0)
- j++;
- off = TupleDescCompactAttr(tupleDesc, j - 1)->attcacheoff +
- TupleDescCompactAttr(tupleDesc, j - 1)->attlen;
-
- for (; j < natts; j++)
+ for (; i < attnum; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, j);
-
- if (att->attlen <= 0)
- break;
-
- off = att_nominal_alignby(off, att->attalignby);
+ if (att_isnull(i, bp))
+ continue;
- att->attcacheoff = off;
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- off += att->attlen;
+ off = att_pointer_alignby(off, cattr->attalignby, cattr->attlen,
+ tp + off);
+ off += cattr->attlen;
}
-
- Assert(j > attnum);
-
- off = TupleDescCompactAttr(tupleDesc, attnum)->attcacheoff;
}
else
{
- bool usecache = true;
- int i;
-
- /*
- * Now we know that we have to walk the tuple CAREFULLY. But we still
- * might be able to cache some offsets for next time.
- *
- * Note - This loop is a little tricky. For each non-null attribute,
- * we have to first account for alignment padding before the attr,
- * then advance over the attr based on its length. Nulls have no
- * storage and no alignment padding either. We can use/set
- * attcacheoff until we reach either a null or a var-width attribute.
- */
- off = 0;
- for (i = 0;; i++) /* loop exit is at "break" */
+ for (i = startAttr; i < firstNullAttr; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, i);
+ int attlen;
- if (HeapTupleHasNulls(tup) && att_isnull(i, bp))
- {
- usecache = false;
- continue; /* this cannot be the target att */
- }
+ cattr = TupleDescCompactAttr(tupleDesc, i);
+ attlen = cattr->attlen;
- /* If we know the next offset, we can skip the rest */
- if (usecache && att->attcacheoff >= 0)
- off = att->attcacheoff;
- else if (att->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be
- * no pad bytes in any case: then the offset will be valid for
- * either an aligned or unaligned value.
- */
- if (usecache &&
- off == att_nominal_alignby(off, att->attalignby))
- att->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, att->attalignby, -1,
- tp + off);
- usecache = false;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, att->attalignby);
+ /*
+ * cstrings don't exist in heap tuples. Use pg_assume to instruct
+ * the compiler not to emit the cstring related code in
+ * att_addlength_pointer().
+ */
+ pg_assume(attlen > 0 || attlen == -1);
- if (usecache)
- att->attcacheoff = off;
- }
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ attlen,
+ tp + off);
+ off = att_addlength_pointer(off, attlen, tp + off);
+ }
- if (i == attnum)
- break;
+ for (; i < attnum; i++)
+ {
+ int attlen;
- off = att_addlength_pointer(off, att->attlen, tp + off);
+ if (att_isnull(i, bp))
+ continue;
- if (usecache && att->attlen <= 0)
- usecache = false;
+ cattr = TupleDescCompactAttr(tupleDesc, i);
+ attlen = cattr->attlen;
+
+ /* As above, heaptuples have no cstrings */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ off = att_pointer_alignby(off, cattr->attalignby, attlen,
+ tp + off);
+ off = att_addlength_pointer(off, attlen, tp + off);
}
+
}
- return fetchatt(TupleDescCompactAttr(tupleDesc, attnum), tp + off);
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+
+ return fetchatt(cattr, tp + off);
}
/* ----------------
@@ -1347,6 +1264,7 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,
Datum *values, bool *isnull)
{
HeapTupleHeader tup = tuple->t_data;
+ CompactAttribute *cattr;
bool hasnulls = HeapTupleHasNulls(tuple);
int tdesc_natts = tupleDesc->natts;
int natts; /* number of atts to extract */
@@ -1354,70 +1272,98 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,
char *tp; /* ptr to tuple data */
uint32 off; /* offset in tuple data */
bits8 *bp = tup->t_bits; /* ptr to null bitmap in tuple */
- bool slow = false; /* can we use/set attcacheoff? */
+ int firstNonCacheOffsetAttr;
+ int firstNullAttr;
natts = HeapTupleHeaderGetNatts(tup);
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
+
/*
* In inheritance situations, it is possible that the given tuple actually
* has more fields than the caller is expecting. Don't run off the end of
* the caller's arrays.
*/
natts = Min(natts, tdesc_natts);
+ firstNonCacheOffsetAttr = Min(tupleDesc->firstNonCachedOffsetAttr, natts);
+
+ if (hasnulls)
+ {
+ firstNullAttr = first_null_attr(bp, natts);
+
+ /*
+ * XXX: it'd be nice to use populate_isnull_array() here, but that
+ * requires that the isnull array's size is rounded up to the next
+ * multiple of 8. Doing that would require adjusting many location
+ * that allocate the array.
+ */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, firstNullAttr);
+ }
+ else
+ firstNullAttr = natts;
tp = (char *) tup + tup->t_hoff;
+ attnum = 0;
- off = 0;
+ if (firstNonCacheOffsetAttr > 0)
+ {
+#ifdef USE_ASSERT_CHECKING
+ /* In Assert enabled builds, verify attcacheoff is correct */
+ int offcheck = 0;
+#endif
+ do
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ off = cattr->attcacheoff;
- for (attnum = 0; attnum < natts; attnum++)
+#ifdef USE_ASSERT_CHECKING
+ offcheck = att_nominal_alignby(offcheck, cattr->attalignby);
+ Assert(offcheck == cattr->attcacheoff);
+ offcheck += cattr->attlen;
+#endif
+
+ values[attnum] = fetch_att_noerr(tp + off,
+ cattr->attbyval,
+ cattr->attlen);
+ } while (++attnum < firstNonCacheOffsetAttr);
+ off += cattr->attlen;
+ }
+ else
+ off = 0;
+
+ for (; attnum < firstNullAttr; attnum++)
{
- CompactAttribute *thisatt = TupleDescCompactAttr(tupleDesc, attnum);
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
+ }
+
+ for (; attnum < natts; attnum++)
+ {
+ Assert(hasnulls);
- if (hasnulls && att_isnull(attnum, bp))
+ if (att_isnull(attnum, bp))
{
values[attnum] = (Datum) 0;
isnull[attnum] = true;
- slow = true; /* can't use attcacheoff anymore */
continue;
}
isnull[attnum] = false;
-
- if (!slow && thisatt->attcacheoff >= 0)
- off = thisatt->attcacheoff;
- else if (thisatt->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be no
- * pad bytes in any case: then the offset will be valid for either
- * an aligned or unaligned value.
- */
- if (!slow &&
- off == att_nominal_alignby(off, thisatt->attalignby))
- thisatt->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, thisatt->attalignby, -1,
- tp + off);
- slow = true;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, thisatt->attalignby);
-
- if (!slow)
- thisatt->attcacheoff = off;
- }
-
- values[attnum] = fetchatt(thisatt, tp + off);
-
- off = att_addlength_pointer(off, thisatt->attlen, tp + off);
-
- if (thisatt->attlen <= 0)
- slow = true; /* can't use attcacheoff anymore */
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
}
/*
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index d6350201e01..92282039671 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -223,18 +223,6 @@ index_form_tuple_context(TupleDesc tupleDescriptor,
*
* This gets called from index_getattr() macro, and only in cases
* where we can't use cacheoffset and the value is not null.
- *
- * This caches attribute offsets in the attribute descriptor.
- *
- * An alternative way to speed things up would be to cache offsets
- * with the tuple, but that seems more difficult unless you take
- * the storage hit of actually putting those offsets into the
- * tuple you send to disk. Yuck.
- *
- * This scheme will be slightly slower than that, but should
- * perform well for queries which hit large #'s of tuples. After
- * you cache the offsets once, examining all the other tuples using
- * the same attribute descriptor will go much quicker. -cim 5/4/91
* ----------------
*/
Datum
@@ -242,205 +230,126 @@ nocache_index_getattr(IndexTuple tup,
int attnum,
TupleDesc tupleDesc)
{
+ CompactAttribute *cattr;
char *tp; /* ptr to data part of tuple */
bits8 *bp = NULL; /* ptr to null bitmap in tuple */
- bool slow = false; /* do we have to walk attrs? */
int data_off; /* tuple data offset */
int off; /* current offset within data */
+ int startAttr;
+ int firstNullAttr;
+ bool hasnulls = IndexTupleHasNulls(tup);
+ int i;
- /* ----------------
- * Three cases:
- *
- * 1: No nulls and no variable-width attributes.
- * 2: Has a null or a var-width AFTER att.
- * 3: Has nulls or var-widths BEFORE att.
- * ----------------
- */
-
- data_off = IndexInfoFindDataOffset(tup->t_info);
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
attnum--;
- if (IndexTupleHasNulls(tup))
- {
- /*
- * there's a null somewhere in the tuple
- *
- * check to see if desired att is null
- */
+ data_off = IndexInfoFindDataOffset(tup->t_info);
+ tp = (char *) tup + data_off;
- /* XXX "knows" t_bits are just after fixed tuple header! */
+ /*
+ * To reduce the number of attributes we need to look at, we start at the
+ * highest attribute that we can which has a cached offset. Since the
+ * attcacheoff for an attribute is only valid if there are no NULLs in
+ * prior attribute, we must look for NULLs to determine the start attr.
+ */
+ if (hasnulls)
+ {
bp = (bits8 *) ((char *) tup + sizeof(IndexTupleData));
-
- /*
- * Now check to see if any preceding bits are null...
- */
- {
- int byte = attnum >> 3;
- int finalbit = attnum & 0x07;
-
- /* check for nulls "before" final bit of last byte */
- if ((~bp[byte]) & ((1 << finalbit) - 1))
- slow = true;
- else
- {
- /* check for nulls in any "earlier" bytes */
- int i;
-
- for (i = 0; i < byte; i++)
- {
- if (bp[i] != 0xFF)
- {
- slow = true;
- break;
- }
- }
- }
- }
+ firstNullAttr = first_null_attr(bp, attnum);
}
+ else
+ firstNullAttr = attnum;
- tp = (char *) tup + data_off;
-
- if (!slow)
+ if (tupleDesc->firstNonCachedOffsetAttr > 0)
{
- CompactAttribute *att;
-
- /*
- * If we get here, there are no nulls up to and including the target
- * attribute. If we have a cached offset, we can use it.
- */
- att = TupleDescCompactAttr(tupleDesc, attnum);
- if (att->attcacheoff >= 0)
- return fetchatt(att, tp + att->attcacheoff);
-
- /*
- * Otherwise, check for non-fixed-length attrs up to and including
- * target. If there aren't any, it's safe to cheaply initialize the
- * cached offsets for these attrs.
- */
- if (IndexTupleHasVarwidths(tup))
- {
- int j;
-
- for (j = 0; j <= attnum; j++)
- {
- if (TupleDescCompactAttr(tupleDesc, j)->attlen <= 0)
- {
- slow = true;
- break;
- }
- }
- }
+ startAttr = Min(tupleDesc->firstNonCachedOffsetAttr - 1, firstNullAttr);
+ off = TupleDescCompactAttr(tupleDesc, startAttr)->attcacheoff;
}
-
- if (!slow)
+ else
{
- int natts = tupleDesc->natts;
- int j = 1;
-
- /*
- * If we get here, we have a tuple with no nulls or var-widths up to
- * and including the target attribute, so we can use the cached offset
- * ... only we don't have it yet, or we'd not have got here. Since
- * it's cheap to compute offsets for fixed-width columns, we take the
- * opportunity to initialize the cached offsets for *all* the leading
- * fixed-width columns, in hope of avoiding future visits to this
- * routine.
- */
- TupleDescCompactAttr(tupleDesc, 0)->attcacheoff = 0;
+ startAttr = 0;
+ off = 0;
+ }
- /* we might have set some offsets in the slow path previously */
- while (j < natts && TupleDescCompactAttr(tupleDesc, j)->attcacheoff > 0)
- j++;
+ /*
+ * Calculate 'off' up to the first NULL attr. We use two cheaper loops
+ * when the tuple has no variable-width columns. When variable-width
+ * columns exist we use att_addlength_pointer() to move the offset beyond
+ * the current attribute.
+ */
+ if (IndexTupleHasVarwidths(tup))
+ {
+ /* Calculate the offset up until the first NULL */
+ for (i = startAttr; i < firstNullAttr; i++)
+ {
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- off = TupleDescCompactAttr(tupleDesc, j - 1)->attcacheoff +
- TupleDescCompactAttr(tupleDesc, j - 1)->attlen;
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+ off = att_addlength_pointer(off, cattr->attlen, tp + off);
+ }
- for (; j < natts; j++)
+ /* Calculate the offset for any remaining columns. */
+ for (; i < attnum; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, j);
+ Assert(hasnulls);
- if (att->attlen <= 0)
- break;
+ if (att_isnull(i, bp))
+ continue;
- off = att_nominal_alignby(off, att->attalignby);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- att->attcacheoff = off;
-
- off += att->attlen;
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+ off = att_addlength_pointer(off, cattr->attlen, tp + off);
}
-
- Assert(j > attnum);
-
- off = TupleDescCompactAttr(tupleDesc, attnum)->attcacheoff;
}
else
{
- bool usecache = true;
- int i;
+ /* Handle tuples with only fixed-width attributes */
- /*
- * Now we know that we have to walk the tuple CAREFULLY. But we still
- * might be able to cache some offsets for next time.
- *
- * Note - This loop is a little tricky. For each non-null attribute,
- * we have to first account for alignment padding before the attr,
- * then advance over the attr based on its length. Nulls have no
- * storage and no alignment padding either. We can use/set
- * attcacheoff until we reach either a null or a var-width attribute.
- */
- off = 0;
- for (i = 0;; i++) /* loop exit is at "break" */
+ /* Calculate the offset up until the first NULL */
+ for (i = startAttr; i < firstNullAttr; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, i);
-
- if (IndexTupleHasNulls(tup) && att_isnull(i, bp))
- {
- usecache = false;
- continue; /* this cannot be the target att */
- }
-
- /* If we know the next offset, we can skip the rest */
- if (usecache && att->attcacheoff >= 0)
- off = att->attcacheoff;
- else if (att->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be
- * no pad bytes in any case: then the offset will be valid for
- * either an aligned or unaligned value.
- */
- if (usecache &&
- off == att_nominal_alignby(off, att->attalignby))
- att->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, att->attalignby, -1,
- tp + off);
- usecache = false;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, att->attalignby);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
+
+ Assert(cattr->attlen > 0);
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+ off += cattr->attlen;
+ }
- if (usecache)
- att->attcacheoff = off;
- }
+ /* Calculate the offset for any remaining columns. */
+ for (; i < attnum; i++)
+ {
+ Assert(hasnulls);
- if (i == attnum)
- break;
+ if (att_isnull(i, bp))
+ continue;
- off = att_addlength_pointer(off, att->attlen, tp + off);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- if (usecache && att->attlen <= 0)
- usecache = false;
+ Assert(cattr->attlen > 0);
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+ off += cattr->attlen;
}
}
- return fetchatt(TupleDescCompactAttr(tupleDesc, attnum), tp + off);
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ off = att_pointer_alignby(off, cattr->attalignby,
+ cattr->attlen, tp + off);
+ return fetchatt(cattr, tp + off);
}
/*
@@ -480,63 +389,87 @@ index_deform_tuple_internal(TupleDesc tupleDescriptor,
Datum *values, bool *isnull,
char *tp, bits8 *bp, int hasnulls)
{
+ CompactAttribute *cattr;
int natts = tupleDescriptor->natts; /* number of atts to extract */
- int attnum;
- int off = 0; /* offset in tuple data */
- bool slow = false; /* can we use/set attcacheoff? */
+ int attnum = 0;
+ uint32 off = 0; /* offset in tuple data */
+ int firstNonCacheOffsetAttr;
+ int firstNullAttr;
/* Assert to protect callers who allocate fixed-size arrays */
Assert(natts <= INDEX_MAX_KEYS);
- for (attnum = 0; attnum < natts; attnum++)
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDescriptor->firstNonCachedOffsetAttr >= 0);
+
+ firstNonCacheOffsetAttr = Min(tupleDescriptor->firstNonCachedOffsetAttr, natts);
+
+ if (hasnulls)
+ {
+ firstNullAttr = first_null_attr(bp, natts);
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, firstNullAttr);
+ }
+ else
+ firstNullAttr = natts;
+
+ if (firstNonCacheOffsetAttr > 0)
+ {
+#ifdef USE_ASSERT_CHECKING
+ /* In Assert enabled builds, verify attcacheoff is correct */
+ off = 0;
+#endif
+
+ do
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDescriptor, attnum);
+
+#ifdef USE_ASSERT_CHECKING
+ off = att_nominal_alignby(off, cattr->attalignby);
+ Assert(off == cattr->attcacheoff);
+ off += cattr->attlen;
+#endif
+
+ values[attnum] = fetch_att_noerr(tp + cattr->attcacheoff, cattr->attbyval,
+ cattr->attlen);
+ } while (++attnum < firstNonCacheOffsetAttr);
+
+ off = cattr->attcacheoff + cattr->attlen;
+ }
+
+ for (; attnum < firstNullAttr; attnum++)
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDescriptor, attnum);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
+ }
+
+ for (; attnum < natts; attnum++)
{
- CompactAttribute *thisatt = TupleDescCompactAttr(tupleDescriptor, attnum);
+ Assert(hasnulls);
- if (hasnulls && att_isnull(attnum, bp))
+ if (att_isnull(attnum, bp))
{
values[attnum] = (Datum) 0;
isnull[attnum] = true;
- slow = true; /* can't use attcacheoff anymore */
continue;
}
isnull[attnum] = false;
-
- if (!slow && thisatt->attcacheoff >= 0)
- off = thisatt->attcacheoff;
- else if (thisatt->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be no
- * pad bytes in any case: then the offset will be valid for either
- * an aligned or unaligned value.
- */
- if (!slow &&
- off == att_nominal_alignby(off, thisatt->attalignby))
- thisatt->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, thisatt->attalignby, -1,
- tp + off);
- slow = true;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, thisatt->attalignby);
-
- if (!slow)
- thisatt->attcacheoff = off;
- }
-
- values[attnum] = fetchatt(thisatt, tp + off);
-
- off = att_addlength_pointer(off, thisatt->attlen, tp + off);
-
- if (thisatt->attlen <= 0)
- slow = true; /* can't use attcacheoff anymore */
+ cattr = TupleDescCompactAttr(tupleDescriptor, attnum);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
}
}
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 2137385a833..41085d43c85 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -197,6 +197,10 @@ CreateTemplateTupleDesc(int natts)
desc->tdtypmod = -1;
desc->tdrefcount = -1; /* assume not reference-counted */
+ /* This will be set to the correct value by TupleDescFinalize() */
+ desc->firstNonCachedOffsetAttr = -1;
+ desc->firstNonGuaranteedAttr = -1;
+
return desc;
}
@@ -457,6 +461,9 @@ TupleDescCopy(TupleDesc dst, TupleDesc src)
* descriptor to another.
*
* !!! Constraints and defaults are not copied !!!
+ *
+ * The caller must take care of calling TupleDescFinalize() on once all
+ * TupleDesc changes have been made.
*/
void
TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
@@ -489,6 +496,45 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
populate_compact_attribute(dst, dstAttno - 1);
}
+/*
+ * TupleDescFinalize
+ * Finalize the given TupleDesc. This must be called after the
+ * attributes arrays have been populated or adjusted by any code.
+ *
+ * Must be called after populate_compact_attribute() and before
+ * BlessTupleDesc().
+ */
+void
+TupleDescFinalize(TupleDesc tupdesc)
+{
+ int firstNonCachedOffsetAttr = 0;
+ int firstNonGuaranteedAttr = tupdesc->natts;
+ int off = 0;
+
+ for (int i = 0; i < tupdesc->natts; i++)
+ {
+ CompactAttribute *cattr = TupleDescCompactAttr(tupdesc, i);
+
+ if (firstNonGuaranteedAttr == tupdesc->natts &&
+ (cattr->attnullability != ATTNULLABLE_VALID || !cattr->attbyval ||
+ cattr->atthasmissing || cattr->attisdropped || cattr->attlen <= 0))
+ firstNonGuaranteedAttr = i;
+
+ if (cattr->attlen <= 0)
+ break;
+
+ off = att_nominal_alignby(off, cattr->attalignby);
+
+ cattr->attcacheoff = off;
+
+ off += cattr->attlen;
+ firstNonCachedOffsetAttr = i + 1;
+ }
+
+ tupdesc->firstNonCachedOffsetAttr = firstNonCachedOffsetAttr;
+ tupdesc->firstNonGuaranteedAttr = firstNonGuaranteedAttr;
+}
+
/*
* Free a TupleDesc including all substructure
*/
diff --git a/src/backend/access/spgist/spgutils.c b/src/backend/access/spgist/spgutils.c
index b246e8127db..a4694bd8065 100644
--- a/src/backend/access/spgist/spgutils.c
+++ b/src/backend/access/spgist/spgutils.c
@@ -335,9 +335,6 @@ getSpGistTupleDesc(Relation index, SpGistTypeDesc *keyType)
/* We shouldn't need to bother with making these valid: */
att->attcompression = InvalidCompressionMethod;
att->attcollation = InvalidOid;
- /* In case we changed typlen, we'd better reset following offsets */
- for (int i = spgFirstIncludeColumn; i < outTupDesc->natts; i++)
- TupleDescCompactAttr(outTupDesc, i)->attcacheoff = -1;
populate_compact_attribute(outTupDesc, spgKeyColumn);
TupleDescFinalize(outTupDesc);
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index e6ab51e6404..80faf29b797 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -993,218 +993,242 @@ tts_buffer_heap_store_tuple(TupleTableSlot *slot, HeapTuple tuple,
}
/*
- * slot_deform_heap_tuple_internal
- * An always inline helper function for use in slot_deform_heap_tuple to
- * allow the compiler to emit specialized versions of this function for
- * various combinations of "slow" and "hasnulls". For example, if a
- * given tuple has no nulls, then we needn't check "hasnulls" for every
- * attribute that we're deforming. The caller can just call this
- * function with hasnulls set to constant-false and have the compiler
- * remove the constant-false branches and emit more optimal code.
- *
- * Returns the next attnum to deform, which can be equal to natts when the
- * function manages to deform all requested attributes. *offp is an input and
- * output parameter which is the byte offset within the tuple to start deforming
- * from which, on return, gets set to the offset where the next attribute
- * should be deformed from. *slowp is set to true when subsequent deforming
- * of this tuple must use a version of this function with "slow" passed as
- * true.
- *
- * Callers cannot assume when we return "attnum" (i.e. all requested
- * attributes have been deformed) that slow mode isn't required for any
- * additional deforming as the final attribute may have caused a switch to
- * slow mode.
+ * slot_deform_heap_tuple
+ * Given a TupleTableSlot, extract data from the slot's physical tuple
+ * into its Datum/isnull arrays. Data is extracted up through the
+ * natts'th column (caller must ensure this is a legal column number).
+ *
+ * This is essentially an incremental version of heap_deform_tuple:
+ * on each call we extract attributes up to the one needed, without
+ * re-computing information about previously extracted attributes.
+ * slot->tts_nvalid is the number of attributes already extracted.
+ *
+ * This is marked as always inline, so the different offp for different types
+ * of slots gets optimized away.
*/
-static pg_attribute_always_inline int
-slot_deform_heap_tuple_internal(TupleTableSlot *slot, HeapTuple tuple,
- int attnum, int natts, bool slow,
- bool hasnulls, uint32 *offp, bool *slowp)
+static pg_attribute_always_inline void
+slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
+ int natts)
{
+ CompactAttribute *cattr;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
- Datum *values = slot->tts_values;
- bool *isnull = slot->tts_isnull;
HeapTupleHeader tup = tuple->t_data;
+ int attnum;
+ int firstNonCacheOffsetAttr;
+ int firstNonGuaranteedAttr;
+ int firstNullAttr;
+ Datum *values;
+ bool *isnull;
char *tp; /* ptr to tuple data */
- bits8 *bp = tup->t_bits; /* ptr to null bitmap in tuple */
- bool slownext = false;
+ uint32 off; /* offset in tuple data */
- tp = (char *) tup + tup->t_hoff;
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
- for (; attnum < natts; attnum++)
+ isnull = slot->tts_isnull;
+
+ /*
+ * Some callers may form and deform tuples prior to NOT NULL constraints
+ * being checked. Here we'd like to optimize the case where we only need
+ * to fetch attributes before or up to the point where the attribute is
+ * guaranteed to exist in the tuple. We rely on the slot flag being set
+ * correctly to only enable this optimization when it's valid to do so.
+ * This optimization allows us to save fetching the number of attributes
+ * from the tuple and saves the additional cost of handling non-byval
+ * attrs.
+ */
+ if (TTS_OBEYS_NOT_NULL_CONSTRAINTS(slot))
+ firstNonGuaranteedAttr = Min(natts, tupleDesc->firstNonGuaranteedAttr);
+ else
+ firstNonGuaranteedAttr = 0;
+
+ firstNonCacheOffsetAttr = tupleDesc->firstNonCachedOffsetAttr;
+
+ if (HeapTupleHasNulls(tuple))
{
- CompactAttribute *thisatt = TupleDescCompactAttr(tupleDesc, attnum);
+ int tupnatts = HeapTupleHeaderGetNatts(tup);
- if (hasnulls && att_isnull(attnum, bp))
+ tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits) +
+ BITMAPLEN(tupnatts));
+
+ natts = Min(tupnatts, natts);
+ if (natts > firstNonGuaranteedAttr)
{
- values[attnum] = (Datum) 0;
- isnull[attnum] = true;
- if (!slow)
- {
- *slowp = true;
- return attnum + 1;
- }
- else
- continue;
- }
+ bits8 *bp = tup->t_bits;
- isnull[attnum] = false;
+ /* Find the first NULL attr */
+ firstNullAttr = first_null_attr(bp, natts);
- /* calculate the offset of this attribute */
- if (!slow && thisatt->attcacheoff >= 0)
- *offp = thisatt->attcacheoff;
- else if (thisatt->attlen == -1)
- {
/*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be no
- * pad bytes in any case: then the offset will be valid for either
- * an aligned or unaligned value.
+ * And populate the isnull array for all attributes being fetched
+ * from the tuple.
*/
- if (!slow && *offp == att_nominal_alignby(*offp, thisatt->attalignby))
- thisatt->attcacheoff = *offp;
- else
- {
- *offp = att_pointer_alignby(*offp,
- thisatt->attalignby,
- -1,
- tp + *offp);
+ populate_isnull_array(bp, natts, isnull);
- if (!slow)
- slownext = true;
- }
+ /* We can only use any cached offsets until the first NULL attr */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, firstNullAttr);
}
else
{
- /* not varlena, so safe to use att_nominal_alignby */
- *offp = att_nominal_alignby(*offp, thisatt->attalignby);
-
- if (!slow)
- thisatt->attcacheoff = *offp;
+ /* Otherwise all required columns are guaranteed to exist */
+ firstNullAttr = natts;
}
+ }
+ else
+ {
+ tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits));
+
+ /*
+ * We only need to look at the tuple's natts if we need more than the
+ * guaranteed number of columns
+ */
+ if (natts > firstNonGuaranteedAttr)
+ natts = Min(HeapTupleHeaderGetNatts(tup), natts);
+
+ /* All attrs can be fetched without checking for NULLs */
+ firstNullAttr = natts;
+ }
- values[attnum] = fetchatt(thisatt, tp + *offp);
+ attnum = slot->tts_nvalid;
+ values = slot->tts_values;
+ slot->tts_nvalid = natts;
- *offp = att_addlength_pointer(*offp, thisatt->attlen, tp + *offp);
+ /* Ensure we calculated tp correctly */
+ Assert(tp == (char *) tup + tup->t_hoff);
- /* check if we need to switch to slow mode */
- if (!slow)
+ if (attnum < firstNonGuaranteedAttr)
+ {
+ do
{
+ int attlen;
+
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ attlen = cattr->attlen;
+
+ /* We don't expect any non-byval types */
+ pg_assume(attlen > 0);
+
/*
- * We're unable to deform any further if the above code set
- * 'slownext', or if this isn't a fixed-width attribute.
+ * Technically we could support non-byval fixed-width types, but
+ * not doing so allows us to pass true to fetch_att_noerr() which
+ * eliminates the !attbyval branch.
*/
- if (slownext || thisatt->attlen <= 0)
- {
- *slowp = true;
- return attnum + 1;
- }
- }
- }
+ Assert(cattr->attbyval == true);
- return natts;
-}
-
-/*
- * slot_deform_heap_tuple
- * Given a TupleTableSlot, extract data from the slot's physical tuple
- * into its Datum/isnull arrays. Data is extracted up through the
- * natts'th column (caller must ensure this is a legal column number).
- *
- * This is essentially an incremental version of heap_deform_tuple:
- * on each call we extract attributes up to the one needed, without
- * re-computing information about previously extracted attributes.
- * slot->tts_nvalid is the number of attributes already extracted.
- *
- * This is marked as always inline, so the different offp for different types
- * of slots gets optimized away.
- */
-static pg_attribute_always_inline void
-slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
- int natts)
-{
- bool hasnulls = HeapTupleHasNulls(tuple);
- int attnum;
- uint32 off; /* offset in tuple data */
- bool slow; /* can we use/set attcacheoff? */
+ off = cattr->attcacheoff;
+ values[attnum] = fetch_att_noerr(tp + off, true, attlen);
+ attnum++;
+ } while (attnum < firstNonGuaranteedAttr);
- /* We can only fetch as many attributes as the tuple has. */
- natts = Min(HeapTupleHeaderGetNatts(tuple->t_data), natts);
+ off += cattr->attlen;
- /*
- * Check whether the first call for this tuple, and initialize or restore
- * loop state.
- */
- attnum = slot->tts_nvalid;
- if (attnum == 0)
- {
- /* Start from the first attribute */
- off = 0;
- slow = false;
+ if (attnum == natts)
+ goto done;
}
else
{
/* Restore state from previous execution */
off = *offp;
- slow = TTS_SLOW(slot);
+
+ /* We expect *offp to be set to 0 when attnum == 0 */
+ Assert(off == 0 || attnum > 0);
}
+ /* We can only fetch as many attributes as the tuple has. */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, natts);
+
/*
- * If 'slow' isn't set, try deforming using deforming code that does not
- * contain any of the extra checks required for non-fixed offset
- * deforming. During deforming, if or when we find a NULL or a variable
- * length attribute, we'll switch to a deforming method which includes the
- * extra code required for non-fixed offset deforming, a.k.a slow mode.
- * Because this is performance critical, we inline
- * slot_deform_heap_tuple_internal passing the 'slow' and 'hasnull'
- * parameters as constants to allow the compiler to emit specialized code
- * with the known-const false comparisons and subsequent branches removed.
+ * Handle the portion of the tuple that we have cached the offset for up
+ * to the first NULL attribute. The offset is effectively fixed for these
+ * so we can use the CompactAttribute's attcacheoff.
*/
- if (!slow)
+ if (attnum < firstNonCacheOffsetAttr)
{
- /* Tuple without any NULLs? We can skip doing any NULL checking */
- if (!hasnulls)
- attnum = slot_deform_heap_tuple_internal(slot,
- tuple,
- attnum,
- natts,
- false, /* slow */
- false, /* hasnulls */
- &off,
- &slow);
- else
- attnum = slot_deform_heap_tuple_internal(slot,
- tuple,
- attnum,
- natts,
- false, /* slow */
- true, /* hasnulls */
- &off,
- &slow);
+ do
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+
+ off = cattr->attcacheoff;
+ values[attnum] = fetch_att_noerr(tp + off,
+ cattr->attbyval,
+ cattr->attlen);
+ } while (++attnum < firstNonCacheOffsetAttr);
+
+ /*
+ * Point the offset after the end of the last attribute with a cached
+ * offset. We expect the final cached offset attribute to have a
+ * fixed width, so just add the attlen to the attcacheoff
+ */
+ Assert(cattr->attlen > 0);
+ off += cattr->attlen;
+ }
+
+ /*
+ * Handle any portion of the tuple that doesn't have a fixed offset up
+ * until the first NULL attribute. This loops only differs from the one
+ * after it by the NULL checks.
+ */
+ for (; attnum < firstNullAttr; attnum++)
+ {
+ int attlen;
+
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ attlen = cattr->attlen;
+
+ /*
+ * cstrings don't exist in heap tuples. Use pg_assume to instruct the
+ * compiler not to emit the cstring related code in
+ * align_fetch_then_add().
+ */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ attlen,
+ cattr->attalignby);
}
- /* If there's still work to do then we must be in slow mode */
- if (attnum < natts)
+ /*
+ * Now handle any remaining attributes in the tuple up to the requested
+ * attnum. This time, include NULL checks as we're now at the first NULL
+ * attribute.
+ */
+ for (; attnum < natts; attnum++)
{
- /* XXX is it worth adding a separate call when hasnulls is false? */
- attnum = slot_deform_heap_tuple_internal(slot,
- tuple,
- attnum,
- natts,
- true, /* slow */
- hasnulls,
- &off,
- &slow);
+ int attlen;
+
+ if (isnull[attnum])
+ {
+ values[attnum] = (Datum) 0;
+ continue;
+ }
+
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ attlen = cattr->attlen;
+
+ /* As above, we don't expect cstrings */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ attlen,
+ cattr->attalignby);
}
+done:
+
/*
* Save state for next execution
*/
slot->tts_nvalid = attnum;
*offp = off;
- if (slow)
- slot->tts_flags |= TTS_FLAG_SLOW;
- else
- slot->tts_flags &= ~TTS_FLAG_SLOW;
}
const TupleTableSlotOps TTSOpsVirtual = {
@@ -1508,7 +1532,7 @@ ExecSetSlotDescriptor(TupleTableSlot *slot, /* slot to change */
slot->tts_values = (Datum *)
MemoryContextAlloc(slot->tts_mcxt, tupdesc->natts * sizeof(Datum));
slot->tts_isnull = (bool *)
- MemoryContextAlloc(slot->tts_mcxt, tupdesc->natts * sizeof(bool));
+ MemoryContextAlloc(slot->tts_mcxt, MAXALIGN(tupdesc->natts * sizeof(bool)));
}
/* --------------------------------
@@ -2259,10 +2283,16 @@ ExecTypeSetColNames(TupleDesc typeInfo, List *namesList)
* This happens "for free" if the tupdesc came from a relcache entry, but
* not if we have manufactured a tupdesc for a transient RECORD datatype.
* In that case we have to notify typcache.c of the existence of the type.
+ *
+ * TupleDescFinalize() must be called on the TupleDesc before calling this
+ * function.
*/
TupleDesc
BlessTupleDesc(TupleDesc tupdesc)
{
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupdesc->firstNonCachedOffsetAttr >= 0);
+
if (tupdesc->tdtypeid == RECORDOID &&
tupdesc->tdtypmod < 0)
assign_record_type_typmod(tupdesc);
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index af3c788ce8b..7f74a8ddcb2 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -246,6 +246,8 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags)
RelationGetDescr(scanstate->ss.ss_currentRelation),
table_slot_callbacks(scanstate->ss.ss_currentRelation));
+ scanstate->ss.ss_ScanTupleSlot->tts_flags |= TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
+
/*
* Initialize result type and projection.
*/
diff --git a/src/backend/jit/llvm/llvmjit_deform.c b/src/backend/jit/llvm/llvmjit_deform.c
index 3eb087eb56b..12521e3e46a 100644
--- a/src/backend/jit/llvm/llvmjit_deform.c
+++ b/src/backend/jit/llvm/llvmjit_deform.c
@@ -62,7 +62,6 @@ slot_compile_deform(LLVMJitContext *context, TupleDesc desc,
LLVMValueRef v_tts_values;
LLVMValueRef v_tts_nulls;
LLVMValueRef v_slotoffp;
- LLVMValueRef v_flagsp;
LLVMValueRef v_nvalidp;
LLVMValueRef v_nvalid;
LLVMValueRef v_maxatt;
@@ -178,7 +177,6 @@ slot_compile_deform(LLVMJitContext *context, TupleDesc desc,
v_tts_nulls =
l_load_struct_gep(b, StructTupleTableSlot, v_slot, FIELDNO_TUPLETABLESLOT_ISNULL,
"tts_ISNULL");
- v_flagsp = l_struct_gep(b, StructTupleTableSlot, v_slot, FIELDNO_TUPLETABLESLOT_FLAGS, "");
v_nvalidp = l_struct_gep(b, StructTupleTableSlot, v_slot, FIELDNO_TUPLETABLESLOT_NVALID, "");
if (ops == &TTSOpsHeapTuple || ops == &TTSOpsBufferHeapTuple)
@@ -747,14 +745,10 @@ slot_compile_deform(LLVMJitContext *context, TupleDesc desc,
{
LLVMValueRef v_off = l_load(b, TypeSizeT, v_offp, "");
- LLVMValueRef v_flags;
LLVMBuildStore(b, l_int16_const(lc, natts), v_nvalidp);
v_off = LLVMBuildTrunc(b, v_off, LLVMInt32TypeInContext(lc), "");
LLVMBuildStore(b, v_off, v_slotoffp);
- v_flags = l_load(b, LLVMInt16TypeInContext(lc), v_flagsp, "tts_flags");
- v_flags = LLVMBuildOr(b, v_flags, l_int16_const(lc, TTS_FLAG_SLOW), "");
- LLVMBuildStore(b, v_flags, v_flagsp);
LLVMBuildRetVoid(b);
}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 770edb34e08..998be24ac41 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -666,14 +666,6 @@ RelationBuildTupleDesc(Relation relation)
elog(ERROR, "pg_attribute catalog is missing %d attribute(s) for relation OID %u",
need, RelationGetRelid(relation));
- /*
- * We can easily set the attcacheoff value for the first attribute: it
- * must be zero. This eliminates the need for special cases for attnum=1
- * that used to exist in fastgetattr() and index_getattr().
- */
- if (RelationGetNumberOfAttributes(relation) > 0)
- TupleDescCompactAttr(relation->rd_att, 0)->attcacheoff = 0;
-
/*
* Set up constraint/default info
*/
@@ -1985,8 +1977,6 @@ formrdesc(const char *relationName, Oid relationReltype,
populate_compact_attribute(relation->rd_att, i);
}
- /* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
- TupleDescCompactAttr(relation->rd_att, 0)->attcacheoff = 0;
TupleDescFinalize(relation->rd_att);
/* mark not-null status */
@@ -4446,8 +4436,6 @@ BuildHardcodedDescriptor(int natts, const FormData_pg_attribute *attrs)
populate_compact_attribute(result, i);
}
- /* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
- TupleDescCompactAttr(result, 0)->attcacheoff = 0;
TupleDescFinalize(result);
/* Note: we don't bother to set up a TupleConstr entry */
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 595413dbbc5..80e1dd0e3c7 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -131,6 +131,19 @@ typedef struct CompactAttribute
* Any code making changes manually to and fields in the FormData_pg_attribute
* array must subsequently call populate_compact_attribute() to flush the
* changes out to the corresponding 'compact_attrs' element.
+ *
+ * firstNonCachedOffsetAttr stores the index into the compact_attrs array for
+ * the first attribute that we don't have a known attcacheoff for.
+ *
+ * firstNonGuaranteedAttr stores the index to into the compact_attrs array for
+ * the first attribute that is either NULLable, missing, or !attbyval. This
+ * can be used in locations as a guarantee that attributes before this will
+ * always exist in tuples. The !attbyval part isn't required for this, but
+ * including this allows various tuple deforming routines to forego any checks
+ * for !attbyval.
+ *
+ * Once a TupleDesc has been populated, before it is used for any purpose
+ * TupleDescFinalize() must be called on it.
*/
typedef struct TupleDescData
{
@@ -138,6 +151,10 @@ typedef struct TupleDescData
Oid tdtypeid; /* composite type ID for tuple type */
int32 tdtypmod; /* typmod for tuple type */
int tdrefcount; /* reference count, or -1 if not counting */
+ int firstNonCachedOffsetAttr; /* index of the first att without
+ * an attcacheoff */
+ int firstNonGuaranteedAttr; /* index of the first nullable,
+ * missing or !attbyval attribute. */
TupleConstr *constr; /* constraints, or NULL if none */
/* compact_attrs[N] is the compact metadata of Attribute Number N+1 */
CompactAttribute compact_attrs[FLEXIBLE_ARRAY_MEMBER];
@@ -195,7 +212,6 @@ extern TupleDesc CreateTupleDescTruncatedCopy(TupleDesc tupdesc, int natts);
extern TupleDesc CreateTupleDescCopyConstr(TupleDesc tupdesc);
-#define TupleDescFinalize(d) ((void) 0)
#define TupleDescSize(src) \
(offsetof(struct TupleDescData, compact_attrs) + \
(src)->natts * sizeof(CompactAttribute) + \
@@ -206,6 +222,7 @@ extern void TupleDescCopy(TupleDesc dst, TupleDesc src);
extern void TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
TupleDesc src, AttrNumber srcAttno);
+extern void TupleDescFinalize(TupleDesc tupdesc);
extern void FreeTupleDesc(TupleDesc tupdesc);
extern void IncrTupleDescRefCount(TupleDesc tupdesc);
diff --git a/src/include/access/tupmacs.h b/src/include/access/tupmacs.h
index d64c18b950b..4d97c27d872 100644
--- a/src/include/access/tupmacs.h
+++ b/src/include/access/tupmacs.h
@@ -15,7 +15,9 @@
#define TUPMACS_H
#include "catalog/pg_type_d.h" /* for TYPALIGN macros */
-
+#include "port/pg_bitutils.h"
+#include "port/pg_bswap.h"
+#include "varatt.h"
/*
* Check a tuple's null bitmap to determine whether the attribute is null.
@@ -28,6 +30,57 @@ att_isnull(int ATT, const bits8 *BITS)
return !(BITS[ATT >> 3] & (1 << (ATT & 0x07)));
}
+/*
+ * populate_isnull_array
+ * Transform a tuple's null bitmap into a boolean array.
+ *
+ * Caller must ensure that the isnull array is sized so it contains
+ * at least as many elements as there are bits in the 'bits' array.
+ * This is required because we always round 'natts' up to the next multiple
+ * of 8.
+ */
+static inline void
+populate_isnull_array(const bits8 *bits, int natts, bool *isnull)
+{
+ int nbytes = (natts + 7) >> 3;
+
+ /*
+ * Multiplying an inverted NULL bitmap byte by this value results in the
+ * lowest bit in each byte being set the same as each bit of the inverted
+ * byte. We perform this as 2 32-bit operations rather than a single
+ * 64-bit operation as multiplying by the required value to do this in
+ * 64-bits would result in overflowing a uint64 in some cases.
+ */
+#define SPREAD_BITS_MULTIPLIER_32 0x204081U
+
+ for (int i = 0; i < nbytes; i++, isnull += 8)
+ {
+ uint64 isnull_8;
+ bits8 nullbyte = ~bits[i];
+
+ /* convert the lower 4 bits of null bitmap word into 32 bit int */
+ isnull_8 = (nullbyte & 0xf) * SPREAD_BITS_MULTIPLIER_32;
+
+ /*
+ * convert the upper 4 bits of null bitmap word into 32 bit int, shift
+ * into the upper 32 bit
+ */
+ isnull_8 |= ((uint64) ((nullbyte >> 4) * SPREAD_BITS_MULTIPLIER_32)) << 32;
+
+ /* mask out all other bits apart from the lowest bit of each byte */
+ isnull_8 &= UINT64CONST(0x0101010101010101);
+
+#ifdef WORDS_BIGENDIAN
+
+ /*
+ * Fix byte order on big-endian machines before copying to the array.
+ */
+ isnull_8 = pg_bswap64(isnull_8);
+#endif
+ memcpy(isnull, &isnull_8, sizeof(uint64));
+ }
+}
+
#ifndef FRONTEND
/*
* Given an attbyval and an attlen from either a Form_pg_attribute or
@@ -69,6 +122,157 @@ fetch_att(const void *T, bool attbyval, int attlen)
else
return PointerGetDatum(T);
}
+
+/*
+ * Same, but no error checking for invalid attlens for byval types. This
+ * is safe to use when attlen comes from CompactAttribute as we validate the
+ * length when populating that struct.
+ */
+static inline Datum
+fetch_att_noerr(const void *T, bool attbyval, int attlen)
+{
+ if (attbyval)
+ {
+ switch (attlen)
+ {
+ case sizeof(int32):
+ return Int32GetDatum(*((const int32 *) T));
+ case sizeof(int16):
+ return Int16GetDatum(*((const int16 *) T));
+ case sizeof(char):
+ return CharGetDatum(*((const char *) T));
+ default:
+ Assert(attlen == sizeof(int64));
+ return Int64GetDatum(*((const int64 *) T));
+ }
+ }
+ else
+ return PointerGetDatum(T);
+}
+
+
+/*
+ * align_fetch_then_add
+ * Applies all the functionality of att_pointer_alignby(),
+ * fetch_att_noerr() and att_addlength_pointer() resulting in *off
+ * pointer to the perhaps unaligned number of bytes into 'tupptr', ready
+ * to deform the next attribute.
+ *
+ * tupptr: pointer to the beginning of the tuple, after the header and any
+ * NULL bitmask.
+ * off: offset in bytes for reading tuple data, possibly unaligned.
+ * attbyval, attlen, attalignby are values from CompactAttribute.
+ */
+static inline Datum
+align_fetch_then_add(const char *tupptr, uint32 *off, bool attbyval, int attlen,
+ uint8 attalignby)
+{
+ Datum res;
+
+ if (attlen > 0)
+ {
+ const char *offset_ptr;
+
+ *off = TYPEALIGN(attalignby, *off);
+ offset_ptr = tupptr + *off;
+ *off += attlen;
+ if (attbyval)
+ {
+ switch (attlen)
+ {
+ case sizeof(char):
+ return CharGetDatum(*((const char *) offset_ptr));
+ case sizeof(int16):
+ return Int16GetDatum(*((const int16 *) offset_ptr));
+ case sizeof(int32):
+ return Int32GetDatum(*((const int32 *) offset_ptr));
+ default:
+
+ /*
+ * populate_compact_attribute_internal() should have
+ * checked
+ */
+ Assert(attlen == sizeof(int64));
+ return Int64GetDatum(*((const int64 *) offset_ptr));
+ }
+ }
+ return PointerGetDatum(offset_ptr);
+ }
+ else if (attlen == -1)
+ {
+ if (!VARATT_IS_SHORT(tupptr + *off))
+ *off = TYPEALIGN(attalignby, *off);
+
+ res = PointerGetDatum(tupptr + *off);
+ *off += VARSIZE_ANY(DatumGetPointer(res));
+ return res;
+ }
+ else
+ {
+ Assert(attlen == -2);
+ *off = TYPEALIGN(attalignby, *off);
+ res = PointerGetDatum(tupptr + *off);
+ *off += strlen(tupptr + *off) + 1;
+ return res;
+ }
+}
+
+/*
+ * first_null_attr
+ * Inspect a NULL bitmask from a tuple and return the 0-based attnum of the
+ * first NULL attribute. Returns natts if no NULLs were found.
+ */
+static inline int
+first_null_attr(const bits8 *bits, int natts)
+{
+ int nattByte = natts >> 3;
+ int bytenum;
+ int res;
+
+#ifdef USE_ASSERT_CHECKING
+ int firstnull_check = natts;
+
+ /* Do it the slow way and check we get the same answer. */
+ for (int i = 0; i < natts; i++)
+ {
+ if (att_isnull(i, bits))
+ {
+ firstnull_check = i;
+ break;
+ }
+ }
+#endif
+
+ /* Process all bytes up to just before the byte for the natts attribute */
+ for (bytenum = 0; bytenum < nattByte; bytenum++)
+ {
+ /* break if there's any NULL attrs (a 0 bit) */
+ if (bits[bytenum] != 0xFF)
+ break;
+ }
+
+ /*
+ * Look for the highest 0-bit in the 'bytenum' element. To do this, we
+ * promote the uint8 to uint32 before performing the bitwise NOT and
+ * looking for the first 1-bit. This works even when the byte is 0xFF, as
+ * the bitwise NOT of 0xFF in 32 bits is 0xFFFFFF00, in which case
+ * pg_rightmost_one_pos32() will return 8. We may end up with a value
+ * higher than natts here, but we'll fix that with the Min() below.
+ */
+ res = bytenum << 3;
+ res += pg_rightmost_one_pos32(~((uint32) bits[bytenum]));
+
+ /*
+ * Since we did no masking to mask out bits beyond the natt'th bit, we may
+ * have found a bit higher than natts, so we must cap res to natts
+ */
+ res = Min(res, natts);
+
+ /* Ensure we got the same answer as the att_isnull() loop got */
+ Assert(res == firstnull_check);
+
+ return res;
+}
#endif /* FRONTEND */
/*
diff --git a/src/include/executor/tuptable.h b/src/include/executor/tuptable.h
index a2dfd707e78..8346be77302 100644
--- a/src/include/executor/tuptable.h
+++ b/src/include/executor/tuptable.h
@@ -84,9 +84,6 @@
* tts_values/tts_isnull are allocated either when the slot is created (when
* the descriptor is provided), or when a descriptor is assigned to the slot;
* they are of length equal to the descriptor's natts.
- *
- * The TTS_FLAG_SLOW flag is saved state for
- * slot_deform_heap_tuple, and should not be touched by any other code.
*----------
*/
@@ -98,12 +95,13 @@
#define TTS_FLAG_SHOULDFREE (1 << 2)
#define TTS_SHOULDFREE(slot) (((slot)->tts_flags & TTS_FLAG_SHOULDFREE) != 0)
-/* saved state for slot_deform_heap_tuple */
-#define TTS_FLAG_SLOW (1 << 3)
-#define TTS_SLOW(slot) (((slot)->tts_flags & TTS_FLAG_SLOW) != 0)
+/* true = formed tuple guaranteed to not have NULLs in NOT NULLable columns */
+#define TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS (1 << 3)
+#define TTS_OBEYS_NOT_NULL_CONSTRAINTS(slot) \
+ (((slot)->tts_flags & TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS) != 0)
/* fixed tuple descriptor */
-#define TTS_FLAG_FIXED (1 << 4)
+#define TTS_FLAG_FIXED (1 << 4) /* XXX change to #3? */
#define TTS_FIXED(slot) (((slot)->tts_flags & TTS_FLAG_FIXED) != 0)
struct TupleTableSlotOps;
@@ -123,7 +121,9 @@ typedef struct TupleTableSlot
#define FIELDNO_TUPLETABLESLOT_VALUES 5
Datum *tts_values; /* current per-attribute values */
#define FIELDNO_TUPLETABLESLOT_ISNULL 6
- bool *tts_isnull; /* current per-attribute isnull flags */
+ bool *tts_isnull; /* current per-attribute isnull flags. Array
+ * size must always be rounded up to the next
+ * multiple of 8 elements. */
MemoryContext tts_mcxt; /* slot itself is in this context */
ItemPointerData tts_tid; /* stored tuple's tid */
Oid tts_tableOid; /* table oid of tuple */
diff --git a/src/test/modules/deform_bench/deform_bench.c b/src/test/modules/deform_bench/deform_bench.c
index 7838f639bef..de39fecf8fd 100644
--- a/src/test/modules/deform_bench/deform_bench.c
+++ b/src/test/modules/deform_bench/deform_bench.c
@@ -49,6 +49,7 @@ deform_bench(PG_FUNCTION_ARGS)
tupdesc = RelationGetDescr(rel);
slot = MakeTupleTableSlot(tupdesc, &TTSOpsBufferHeapTuple);
+ slot->tts_flags |= TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
scan = table_beginscan_strat(rel, GetActiveSnapshot(), 0, NULL, true, false);
/*
--
2.51.0
[text/plain] v10-0004-Allow-sibling-call-optimization-in-slot_getsomea.patch (8.5K, 5-v10-0004-Allow-sibling-call-optimization-in-slot_getsomea.patch)
download | inline diff:
From 8acf98c57a7ced3e081c911b47e1d6d644bc17db Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Mon, 16 Feb 2026 14:20:19 +1300
Subject: [PATCH v10 4/5] Allow sibling call optimization in
slot_getsomeattrs_int()
This changes the TupleTableSlotOps contract to make it so the
getsomeattrs() function is in charge of calling
slot_getmissingattrs().
Since this removes all code from slot_getsomeattrs_int() aside from the
getsomeattrs() call itself, we may as well adjust slot_getsomeattrs() so
that it calls getsomeattrs() directly. We leave slot_getsomeattrs_int()
intact as this is still called from the JIT code.
---
src/backend/executor/execTuples.c | 79 ++++++++++++++++---------------
src/include/executor/tuptable.h | 7 +--
2 files changed, 44 insertions(+), 42 deletions(-)
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 80faf29b797..2070c665d2f 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -73,7 +73,7 @@
static TupleDesc ExecTypeFromTLInternal(List *targetList,
bool skipjunk);
static pg_attribute_always_inline void slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
- int natts);
+ int reqnatts);
static inline void tts_buffer_heap_store_tuple(TupleTableSlot *slot,
HeapTuple tuple,
Buffer buffer,
@@ -996,7 +996,10 @@ tts_buffer_heap_store_tuple(TupleTableSlot *slot, HeapTuple tuple,
* slot_deform_heap_tuple
* Given a TupleTableSlot, extract data from the slot's physical tuple
* into its Datum/isnull arrays. Data is extracted up through the
- * natts'th column (caller must ensure this is a legal column number).
+ * reqnatts'th column. If there are insufficient attributes in the given
+ * tuple, then slot_getmissingattrs() is called to populate the
+ * remainder. If reqnatts is above the number of attributes in the
+ * slot's TupleDesc, an error is raised.
*
* This is essentially an incremental version of heap_deform_tuple:
* on each call we extract attributes up to the one needed, without
@@ -1008,7 +1011,7 @@ tts_buffer_heap_store_tuple(TupleTableSlot *slot, HeapTuple tuple,
*/
static pg_attribute_always_inline void
slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
- int natts)
+ int reqnatts)
{
CompactAttribute *cattr;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
@@ -1017,6 +1020,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int firstNonCacheOffsetAttr;
int firstNonGuaranteedAttr;
int firstNullAttr;
+ int natts;
Datum *values;
bool *isnull;
char *tp; /* ptr to tuple data */
@@ -1038,7 +1042,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
* attrs.
*/
if (TTS_OBEYS_NOT_NULL_CONSTRAINTS(slot))
- firstNonGuaranteedAttr = Min(natts, tupleDesc->firstNonGuaranteedAttr);
+ firstNonGuaranteedAttr = Min(reqnatts, tupleDesc->firstNonGuaranteedAttr);
else
firstNonGuaranteedAttr = 0;
@@ -1046,12 +1050,11 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
if (HeapTupleHasNulls(tuple))
{
- int tupnatts = HeapTupleHeaderGetNatts(tup);
-
+ natts = HeapTupleHeaderGetNatts(tup);
tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits) +
- BITMAPLEN(tupnatts));
+ BITMAPLEN(natts));
- natts = Min(tupnatts, natts);
+ natts = Min(natts, reqnatts);
if (natts > firstNonGuaranteedAttr)
{
bits8 *bp = tup->t_bits;
@@ -1082,8 +1085,13 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
* We only need to look at the tuple's natts if we need more than the
* guaranteed number of columns
*/
- if (natts > firstNonGuaranteedAttr)
- natts = Min(HeapTupleHeaderGetNatts(tup), natts);
+ if (reqnatts > firstNonGuaranteedAttr)
+ natts = Min(HeapTupleHeaderGetNatts(tup), reqnatts);
+ else
+ {
+ /* No need to access the number of attributes in the tuple */
+ natts = reqnatts;
+ }
/* All attrs can be fetched without checking for NULLs */
firstNullAttr = natts;
@@ -1091,7 +1099,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
attnum = slot->tts_nvalid;
values = slot->tts_values;
- slot->tts_nvalid = natts;
+ slot->tts_nvalid = reqnatts;
/* Ensure we calculated tp correctly */
Assert(tp == (char *) tup + tup->t_hoff);
@@ -1123,7 +1131,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
off += cattr->attlen;
- if (attnum == natts)
+ if (attnum == reqnatts)
goto done;
}
else
@@ -1222,12 +1230,12 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
cattr->attalignby);
}
+ /* Fetch any missing attrs and raise an error if reqnatts is invalid */
+ if (unlikely(attnum < reqnatts))
+ slot_getmissingattrs(slot, attnum, reqnatts);
done:
- /*
- * Save state for next execution
- */
- slot->tts_nvalid = attnum;
+ /* Save current offset for next execution */
*offp = off;
}
@@ -2088,28 +2096,29 @@ slot_getmissingattrs(TupleTableSlot *slot, int startAttNum, int lastAttNum)
if (!attrmiss)
{
/* no missing values array at all, so just fill everything in as NULL */
- memset(slot->tts_values + startAttNum, 0,
- (lastAttNum - startAttNum) * sizeof(Datum));
- memset(slot->tts_isnull + startAttNum, 1,
- (lastAttNum - startAttNum) * sizeof(bool));
+ for (int attnum = startAttNum; attnum < lastAttNum; attnum++)
+ {
+ slot->tts_values[attnum] = (Datum) 0;
+ slot->tts_isnull[attnum] = true;
+ }
}
else
{
- int missattnum;
-
- /* if there is a missing values array we must process them one by one */
- for (missattnum = startAttNum;
- missattnum < lastAttNum;
- missattnum++)
+ /* use attrmiss to set the missing values */
+ for (int attnum = startAttNum; attnum < lastAttNum; attnum++)
{
- slot->tts_values[missattnum] = attrmiss[missattnum].am_value;
- slot->tts_isnull[missattnum] = !attrmiss[missattnum].am_present;
+ slot->tts_values[attnum] = attrmiss[attnum].am_value;
+ slot->tts_isnull[attnum] = !attrmiss[attnum].am_present;
}
}
+
+ if (unlikely(lastAttNum > slot->tts_tupleDescriptor->natts))
+ elog(ERROR, "invalid attribute number %d", lastAttNum);
}
/*
- * slot_getsomeattrs_int - workhorse for slot_getsomeattrs()
+ * slot_getsomeattrs_int
+ * external function to call getsomeattrs() for use in JIT
*/
void
slot_getsomeattrs_int(TupleTableSlot *slot, int attnum)
@@ -2118,21 +2127,13 @@ slot_getsomeattrs_int(TupleTableSlot *slot, int attnum)
Assert(slot->tts_nvalid < attnum); /* checked in slot_getsomeattrs */
Assert(attnum > 0);
- if (unlikely(attnum > slot->tts_tupleDescriptor->natts))
- elog(ERROR, "invalid attribute number %d", attnum);
-
/* Fetch as many attributes as possible from the underlying tuple. */
slot->tts_ops->getsomeattrs(slot, attnum);
/*
- * If the underlying tuple doesn't have enough attributes, tuple
- * descriptor must have the missing attributes.
+ * Avoid putting new code here as that would prevent the compiler from
+ * using the sibling call optimization for the above function.
*/
- if (unlikely(slot->tts_nvalid < attnum))
- {
- slot_getmissingattrs(slot, slot->tts_nvalid, attnum);
- slot->tts_nvalid = attnum;
- }
}
/* ----------------------------------------------------------------
diff --git a/src/include/executor/tuptable.h b/src/include/executor/tuptable.h
index 8346be77302..1922c912089 100644
--- a/src/include/executor/tuptable.h
+++ b/src/include/executor/tuptable.h
@@ -153,8 +153,8 @@ struct TupleTableSlotOps
* Fill up first natts entries of tts_values and tts_isnull arrays with
* values from the tuple contained in the slot. The function may be called
* with natts more than the number of attributes available in the tuple,
- * in which case it should set tts_nvalid to the number of returned
- * columns.
+ * in which case the function must call slot_getmissingattrs() to populate
+ * the remaining attributes.
*/
void (*getsomeattrs) (TupleTableSlot *slot, int natts);
@@ -357,8 +357,9 @@ extern void slot_getsomeattrs_int(TupleTableSlot *slot, int attnum);
static inline void
slot_getsomeattrs(TupleTableSlot *slot, int attnum)
{
+ /* Populate slot with attributes up to 'attnum', if it's not already */
if (slot->tts_nvalid < attnum)
- slot_getsomeattrs_int(slot, attnum);
+ slot->tts_ops->getsomeattrs(slot, attnum);
}
/*
--
2.51.0
[text/plain] v10-0005-Reduce-size-of-CompactAttribute-struct-to-8-byte.patch (5.5K, 6-v10-0005-Reduce-size-of-CompactAttribute-struct-to-8-byte.patch)
download | inline diff:
From 3749a0f9d298f5bd7e1ac06195a6a25fd10b63a5 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Mon, 23 Feb 2026 09:39:37 +1300
Subject: [PATCH v10 5/5] Reduce size of CompactAttribute struct to 8 bytes
Previously, this was 16 bytes. With the use of some bitflags and by
reducing the attcacheoff field size to a 16-bit type, we can halve the
size of the struct.
It's unlikely that caching the offsets for offsets larger than what will
fit in a 16-bit int will help much as the tuple is very likely to have
some non-fixed-width types anyway, the offsets of which we cannot cache.
---
src/backend/access/common/tupdesc.c | 10 ++++++++++
src/backend/executor/execTuples.c | 16 ++++++++++++----
src/include/access/tupdesc.h | 16 ++++++++--------
3 files changed, 30 insertions(+), 12 deletions(-)
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 41085d43c85..ca84913f3ad 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -525,6 +525,16 @@ TupleDescFinalize(TupleDesc tupdesc)
off = att_nominal_alignby(off, cattr->attalignby);
+ /*
+ * attcacheoff is an int16, so don't try and cache any offsets larger
+ * than will fit in that type. Any attributes which are offset more
+ * than 2^15 are likely due to variable length attributes. Since we
+ * don't cache offsets for or beyond variable length attributes, using
+ * an int16 rather than an int32 here is unlikely to cost us anything.
+ */
+ if (off > PG_INT16_MAX)
+ break;
+
cattr->attcacheoff = off;
off += cattr->attlen;
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 2070c665d2f..b3699b77649 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -1013,6 +1013,7 @@ static pg_attribute_always_inline void
slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int reqnatts)
{
+ CompactAttribute *cattrs;
CompactAttribute *cattr;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
HeapTupleHeader tup = tuple->t_data;
@@ -1101,6 +1102,13 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
values = slot->tts_values;
slot->tts_nvalid = reqnatts;
+ /*
+ * We store the tupleDesc's CompactAttribute array in 'cattrs' as gcc
+ * seems to be unwilling to optimize accessing the CompactAttribute
+ * element efficiently when accessing it via TupleDescCompactAttr().
+ */
+ cattrs = tupleDesc->compact_attrs;
+
/* Ensure we calculated tp correctly */
Assert(tp == (char *) tup + tup->t_hoff);
@@ -1111,7 +1119,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int attlen;
isnull[attnum] = false;
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
attlen = cattr->attlen;
/* We don't expect any non-byval types */
@@ -1156,7 +1164,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
do
{
isnull[attnum] = false;
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
off = cattr->attcacheoff;
values[attnum] = fetch_att_noerr(tp + off,
@@ -1183,7 +1191,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int attlen;
isnull[attnum] = false;
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
attlen = cattr->attlen;
/*
@@ -1216,7 +1224,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
continue;
}
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
attlen = cattr->attlen;
/* As above, we don't expect cstrings */
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 80e1dd0e3c7..9c5eac49172 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -55,7 +55,7 @@ typedef struct TupleConstr
* directly after the FormData_pg_attribute struct is populated or
* altered in any way.
*
- * Currently, this struct is 16 bytes. Any code changes which enlarge this
+ * Currently, this struct is 8 bytes. Any code changes which enlarge this
* struct should be considered very carefully.
*
* Code which must access a TupleDesc's attribute data should always make use
@@ -67,17 +67,17 @@ typedef struct TupleConstr
*/
typedef struct CompactAttribute
{
- int32 attcacheoff; /* fixed offset into tuple, if known, or -1 */
+ int16 attcacheoff; /* fixed offset into tuple, if known, or -1 */
int16 attlen; /* attr len in bytes or -1 = varlen, -2 =
* cstring */
bool attbyval; /* as FormData_pg_attribute.attbyval */
- bool attispackable; /* FormData_pg_attribute.attstorage !=
- * TYPSTORAGE_PLAIN */
- bool atthasmissing; /* as FormData_pg_attribute.atthasmissing */
- bool attisdropped; /* as FormData_pg_attribute.attisdropped */
- bool attgenerated; /* FormData_pg_attribute.attgenerated != '\0' */
- char attnullability; /* status of not-null constraint, see below */
uint8 attalignby; /* alignment requirement in bytes */
+ bool attispackable:1; /* FormData_pg_attribute.attstorage !=
+ * TYPSTORAGE_PLAIN */
+ bool atthasmissing:1; /* as FormData_pg_attribute.atthasmissing */
+ bool attisdropped:1; /* as FormData_pg_attribute.attisdropped */
+ bool attgenerated:1; /* FormData_pg_attribute.attgenerated != '\0' */
+ char attnullability; /* status of not-null constraint, see below */
} CompactAttribute;
/* Valid values for CompactAttribute->attnullability */
--
2.51.0
^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: More speedups for tuple deformation
@ 2026-02-25 00:59 David Rowley <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 1 reply; 30+ messages in thread
From: David Rowley @ 2026-02-25 00:59 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: John Naylor <[email protected]>; Chao Li <[email protected]>; PostgreSQL Developers <[email protected]>
Thanks for looking.
On Wed, 25 Feb 2026 at 03:39, Andres Freund <[email protected]> wrote:
> ISTM we should just merge 0004. In my testing it's a very clear win, without,
> afaict, any downsides.
I'd like to get them in in sequence as I believe 0004 buys back some
extra overheads such as the Min()s in slot_deform_heap_tuple(). If I
were to do 0004 first, then wait a while, it might look more like I'm
introducing a small regression.
> > I'm not getting great results from benchmarking the 0005 patch. I
> > verified that gcc does access the array without calculating the
> > element address from scratch each time and calculates it once, then
> > increments the pointer by sizeof(CompactAttribute). See the attached
> > .csv for the results on the 3 machines I tested on.
>
> FWIW, where I had seen that be rather beneficial is the TupleDescCompactAttr()
> at the start of the various loops, where the compiler has little choice to
> compute the address of the tupdesc->compact_attrs[firstNeededCol]. That
> matters only when only deforming a small number of columns, of course.
oh ok. I wasn't aware that LEA's scaling factor can only be 1,2 4 or
8. With the 8-byte struct, the compiler should be able to do the shift
and add as one operation, whereas with the 16-byte struct would
require a separate shift and add.
Looking at the generated code, with 0004, I see:
1c79: 48 c1 e2 04 shl rdx,0x4
1c7d: 48 8d 4c 15 20 lea rcx,[rbp+rdx*1+0x20]
whereas with 0005 I see:
1c6b: 4a 8d 1c dd 00 00 00 lea rbx,[r11*8+0x0]
Is that what you meant?
> > I've also resequenced the patches to make the deform_bench test module
> > part of the 0001 patch. This makes it easier to test the performance
> > of master.
>
> What are your thoughts about merging the deform_bench tooling? I wonder if we
> should have src/test/modules/benchmark_tools or such, so we can add a few more
> micro-benchmarky tools over time?
I'd like to see us give these tools a proper home. It helps lower the
bar for anyone else who'd like to experiment at some future date, and
also allows people to more easily test for performance regressions if
they're forced to change related code. I've also got a tool that
benchmarks the MemoryContext code which I keep in some local repo that
I dig out from time to time. Given that, it's probably unlikely
deform_bench would be the only extension in there if we did make a
directory for these.
On the otherhand, it does add some maintenance overhead, but IMO,
helping to ensure various key routines are optimal is a worthy enough
cause to make the maintenance overhead worthwhile.
David
^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: More speedups for tuple deformation
@ 2026-02-25 07:03 John Naylor <[email protected]>
parent: David Rowley <[email protected]>
0 siblings, 0 replies; 30+ messages in thread
From: John Naylor @ 2026-02-25 07:03 UTC (permalink / raw)
To: David Rowley <[email protected]>; +Cc: Zsolt Parragi <[email protected]>; Andres Freund <[email protected]>; Chao Li <[email protected]>; PostgreSQL Developers <[email protected]>
On Wed, Feb 25, 2026 at 7:40 AM David Rowley <[email protected]> wrote:
> On Wed, 25 Feb 2026 at 07:33, Zsolt Parragi <[email protected]> wrote:
> > Won't this mix up column numbers on big-endian systems?
>
> Yes, I believe you're right. I've added the following before the memcpy().
>
> #ifdef WORDS_BIGENDIAN
>
> /*
> * Fix byte order on big-endian machines before copying to the array.
> */
> isnull_8 = pg_bswap64(isnull_8);
> #endif
I confirmed regression tests bail out early on a big-endian machine
with v9 and pass with v10.
--
John Naylor
Amazon Web Services
^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: More speedups for tuple deformation
@ 2026-02-25 18:05 Andres Freund <[email protected]>
parent: David Rowley <[email protected]>
0 siblings, 1 reply; 30+ messages in thread
From: Andres Freund @ 2026-02-25 18:05 UTC (permalink / raw)
To: David Rowley <[email protected]>; +Cc: John Naylor <[email protected]>; Chao Li <[email protected]>; PostgreSQL Developers <[email protected]>
Hi,
On 2026-02-25 13:59:53 +1300, David Rowley wrote:
> On Wed, 25 Feb 2026 at 03:39, Andres Freund <[email protected]> wrote:
> > ISTM we should just merge 0004. In my testing it's a very clear win, without,
> > afaict, any downsides.
>
> I'd like to get them in in sequence as I believe 0004 buys back some
> extra overheads such as the Min()s in slot_deform_heap_tuple(). If I
> were to do 0004 first, then wait a while, it might look more like I'm
> introducing a small regression.
I'd rather get more stuff merged out of the way. We can handle a small
regression by comparing to 18 instead. I think we tend to be too worried
about intra-master regressions, and not worried enough about intra branch
regressions.
But FWIW, I don't actually see any regressions due to the Min() on my hardware
as the patches stand.
I do actually see a small regression in some cases due to 0004, but that's
easy to fix:
/* Fetch any missing attrs and raise an error if reqnatts is invalid */
if (unlikely(attnum < reqnatts))
slot_getmissingattrs(slot, attnum, reqnatts);
done:
/* Save current offset for next execution */
*offp = off;
This forces pushing a bunch of stack state before the call to
slot_getmissingattrs, doing the call, returning, restoring state from the
stack, just to then jump to the end of the function to set *offp which then
again does a bunch of stack restoring and returns.
If you instead make it something like:
/* Fetch any missing attrs and raise an error if reqnatts is invalid */
if (unlikely(attnum < reqnatts))
{
*offp = off;
slot_getmissingattrs(slot, attnum, reqnatts);
return;
}
(or just duplicate the *offp = off in the goto done; case)
the overhead seems to be removed.
At least gcc is doing some truly weird shit in the
firstNonGuaranteed/firstNonCachedOffsetAttr loop "header" (i.e. just before
the first entrance to the loop) , which leads to the register pressure being
high, which leads to spilling on the stack, making the few-tuples case slower:
r9 is tts_nvalid, r11 is firstNonGuaranteedAttr, r15 is
slot->tts_tupleDescriptor, rbx is tts_values, -0x10(%rsp) is tts_isnull, rax
is tup.
if (attnum < firstNonGuaranteedAttr)
cmp %r9d,%r11d
jle <tts_buffer_heap_getsomeattrs+0x3c0>
lea 0x0(,%r9,8),%r12
multiply r9/tts_nvalid by 8, store in r12
sub %r9d,%r11d
r11=firstNonGuaranteedAttr-tts_nvalid
xor %ecx,%ecx
Set ecx to 0
lea 0x20(%r15,%r12,1),%rdx
compute (char *) tupleDesc->compact_attrs + r12, i.e. the address of the first
needed compact attribute
add %rbx,%r12
compute (char*) tts_values + tts_nvalid * sizeof(Datum)
mov -0x10(%rsp),%rbx
restore tts_isnull from stack
lea (%rbx,%r9,1),%rsi
compute tts_isnull[tts_nvalid]
jmp <tts_buffer_heap_getsomeattrs+0xee>
Then the loop body:
...
movb $0x0,(%rsi,%rcx,1)
set (tts_isnull + tts_nvalid)[i] = 0
mov %rdx,%r13
movswl (%rdx),%edi
load cattr->attcacheoff into rdi/edi
movzwl 0x2(%rdx),%r10d
load attlen into r10
mov %rdi,%rbx
add %rax,%rdi
rdi = tup + attcacheoff
[bunch of attlen related branches omitted]
movslq (%rdi),%rdi
store *(tup + off) in rdi
mov %rdi,(%r12,%rcx,8)
store rdi in tts_values[i]
lea 0x1(%rcx),%rdi
increment i by one, in a weird way, store in rdi
add $0x8,%rdx
increment cattr by 8
cmp %rdi,%r11
check if i < (firstNonGuaranteedAttr-tts_nvalid)
I.e. the compiler creates an offset version of tts_values[tts_nvalid],
tts_isnull[tts_nvalid], which then creates register allocation pressure,
because later the original tts_values/tts_isnulll etc are accessed again and
thus the underlying registers are preserved. And this is all for zero gain,
from what I can tell, because the acceses are still done with indexed
addressing (like mov %rdi,(%r12,%rcx,8)), which would work just as
well if rcx were indexed based on attnum, not zero indexed within the loop.
I see about a 10% improvement if I dissuade the compiler from doing that by
adding
__asm__ volatile ("" : "+r"(attnum) : :);
In the loop body.
I'm getting to the point where I'd like to just hand write the assembler for
this stupid function. Gah.
> > > I'm not getting great results from benchmarking the 0005 patch. I
> > > verified that gcc does access the array without calculating the
> > > element address from scratch each time and calculates it once, then
> > > increments the pointer by sizeof(CompactAttribute). See the attached
> > > .csv for the results on the 3 machines I tested on.
> >
> > FWIW, where I had seen that be rather beneficial is the TupleDescCompactAttr()
> > at the start of the various loops, where the compiler has little choice to
> > compute the address of the tupdesc->compact_attrs[firstNeededCol]. That
> > matters only when only deforming a small number of columns, of course.
>
> oh ok. I wasn't aware that LEA's scaling factor can only be 1,2 4 or
> 8. With the 8-byte struct, the compiler should be able to do the shift
> and add as one operation, whereas with the 16-byte struct would
> require a separate shift and add.
>
> Looking at the generated code, with 0004, I see:
>
> 1c79: 48 c1 e2 04 shl rdx,0x4
> 1c7d: 48 8d 4c 15 20 lea rcx,[rbp+rdx*1+0x20]
>
> whereas with 0005 I see:
>
> 1c6b: 4a 8d 1c dd 00 00 00 lea rbx,[r11*8+0x0]
>
> Is that what you meant?
Yep.
> > > I've also resequenced the patches to make the deform_bench test module
> > > part of the 0001 patch. This makes it easier to test the performance
> > > of master.
> >
> > What are your thoughts about merging the deform_bench tooling? I wonder if we
> > should have src/test/modules/benchmark_tools or such, so we can add a few more
> > micro-benchmarky tools over time?
>
> I'd like to see us give these tools a proper home. It helps lower the
> bar for anyone else who'd like to experiment at some future date, and
> also allows people to more easily test for performance regressions if
> they're forced to change related code. I've also got a tool that
> benchmarks the MemoryContext code which I keep in some local repo that
> I dig out from time to time. Given that, it's probably unlikely
> deform_bench would be the only extension in there if we did make a
> directory for these.
I'd probably lean towards one extensions with different functions for
different purposes, but that's boring details.
> On the otherhand, it does add some maintenance overhead, but IMO,
> helping to ensure various key routines are optimal is a worthy enough
> cause to make the maintenance overhead worthwhile.
Agreed.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: More speedups for tuple deformation
@ 2026-02-25 20:29 Andres Freund <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 1 reply; 30+ messages in thread
From: Andres Freund @ 2026-02-25 20:29 UTC (permalink / raw)
To: David Rowley <[email protected]>; +Cc: John Naylor <[email protected]>; Chao Li <[email protected]>; PostgreSQL Developers <[email protected]>
Hi,
On 2026-02-25 13:05:14 -0500, Andres Freund wrote:
> At least gcc is doing some truly weird shit in the
> firstNonGuaranteed/firstNonCachedOffsetAttr loop "header" (i.e. just before
> the first entrance to the loop) , which leads to the register pressure being
> high, which leads to spilling on the stack, making the few-tuples case slower:
>
> [ lots of stuff trimmed ]
>
> I.e. the compiler creates an offset version of tts_values[tts_nvalid],
> tts_isnull[tts_nvalid], which then creates register allocation pressure,
> because later the original tts_values/tts_isnulll etc are accessed again and
> thus the underlying registers are preserved. And this is all for zero gain,
> from what I can tell, because the acceses are still done with indexed
> addressing (like mov %rdi,(%r12,%rcx,8)), which would work just as
> well if rcx were indexed based on attnum, not zero indexed within the loop.
>
> I see about a 10% improvement if I dissuade the compiler from doing that by
> adding
> __asm__ volatile ("" : "+r"(attnum) : :);
>
> In the loop body.
>
>
> I'm getting to the point where I'd like to just hand write the assembler for
> this stupid function. Gah.
Huh. It, at least partially, seems to be related to using an integer for
attnum et al. Due to us using -fwrapv, the compiler can't actually assume that
an attnum++ won't overflow. An overflow would make the loop trip counts a lot
more complicated. Even with that I don't understand how it ends up
generating such crappy code, but since using size_t fixes it...
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: More speedups for tuple deformation
@ 2026-03-01 13:10 David Rowley <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 1 reply; 30+ messages in thread
From: David Rowley @ 2026-03-01 13:10 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: John Naylor <[email protected]>; Chao Li <[email protected]>; PostgreSQL Developers <[email protected]>
On Thu, 26 Feb 2026 at 09:29, Andres Freund <[email protected]> wrote:
> Huh. It, at least partially, seems to be related to using an integer for
> attnum et al. Due to us using -fwrapv, the compiler can't actually assume that
> an attnum++ won't overflow. An overflow would make the loop trip counts a lot
> more complicated. Even with that I don't understand how it ends up
> generating such crappy code, but since using size_t fixes it...
Thanks. That seems to make the gcc compiled version quite a bit better.
I am still seeing a bit of register overflow as the TupleDesc is
written to the stack and reloaded back into a register a couple of
times. I've attached the objdump in question.
if (attnum < firstNonGuaranteedAttr)
1c3c: 48 39 e8 cmp rax,rbp
1c3f: 73 7f jae 1cc0 <tts_heap_getsomeattrs+0x110>
1c41: 48 89 54 24 f0 mov QWORD PTR [rsp-0x10],rdx
1c46: 48 8d 74 c2 20 lea rsi,[rdx+rax*8+0x20]
the tupledesc is put back into the register in:
off += cattr->attlen;
1f88: 48 8b 54 24 f0 mov rdx,QWORD PTR [rsp-0x10]
I've not found a way to have gcc not do this.
I've also resequenced the patches so 0002 contains the sibling call
optimisation for slot_getmissingattrs() and I've applied that tail
call optimisation that you mentioned for slot_getmissingattrs() in
0004.
I've attached benchmark results in the attached spreadsheet.
David
0000000000001bb0 <tts_heap_getsomeattrs>:
{
1bb0: f3 0f 1e fa endbr64
1bb4: 41 57 push r15
1bb6: 49 89 fb mov r11,rdi
1bb9: 41 56 push r14
1bbb: 41 55 push r13
1bbd: 41 54 push r12
1bbf: 4c 63 e6 movsxd r12,esi
1bc2: 55 push rbp
1bc3: 53 push rbx
HeapTupleHeader tup = tuple->t_data;
1bc4: 48 8b 47 40 mov rax,QWORD PTR [rdi+0x40]
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
1bc8: 48 8b 57 10 mov rdx,QWORD PTR [rdi+0x10]
isnull = slot->tts_isnull;
1bcc: 48 8b 4f 20 mov rcx,QWORD PTR [rdi+0x20]
HeapTupleHeader tup = tuple->t_data;
1bd0: 48 8b 58 10 mov rbx,QWORD PTR [rax+0x10]
if (TTS_OBEYS_NOT_NULL_CONSTRAINTS(slot))
1bd4: f6 47 04 08 test BYTE PTR [rdi+0x4],0x8
1bd8: 0f 84 02 04 00 00 je 1fe0 <tts_heap_getsomeattrs+0x430>
firstNonGuaranteedAttr = Min(reqnatts, tupleDesc->firstNonGuaranteedAttr);
1bde: 8b 42 14 mov eax,DWORD PTR [rdx+0x14]
1be1: 41 39 c4 cmp r12d,eax
1be4: 41 0f 4e c4 cmovle eax,r12d
if (attnum < firstNonGuaranteedAttr)
1be8: 48 63 e8 movsxd rbp,eax
firstNonCacheOffsetAttr = tupleDesc->firstNonCachedOffsetAttr;
1beb: 4c 63 52 10 movsxd r10,DWORD PTR [rdx+0x10]
if (HeapTupleHasNulls(tuple))
1bef: f6 43 14 01 test BYTE PTR [rbx+0x14],0x1
1bf3: 0f 84 b7 03 00 00 je 1fb0 <tts_heap_getsomeattrs+0x400>
natts = HeapTupleHeaderGetNatts(tup);
1bf9: 44 0f b7 4b 12 movzx r9d,WORD PTR [rbx+0x12]
1bfe: 41 81 e1 ff 07 00 00 and r9d,0x7ff
* Computes size of null bitmap given number of data columns.
*/
static inline int
BITMAPLEN(int NATTS)
{
return (NATTS + 7) / 8;
1c05: 45 8d 41 07 lea r8d,[r9+0x7]
1c09: 41 c1 f8 03 sar r8d,0x3
tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits) +
1c0d: 41 83 c0 1e add r8d,0x1e
1c11: 41 81 e0 f8 03 00 00 and r8d,0x3f8
1c18: 49 01 d8 add r8,rbx
natts = Min(natts, reqnatts);
1c1b: 45 39 cc cmp r12d,r9d
1c1e: 4d 0f 4e cc cmovle r9,r12
firstNullAttr = natts;
1c22: 45 89 ce mov r14d,r9d
if (natts > firstNonGuaranteedAttr)
1c25: 41 39 c1 cmp r9d,eax
1c28: 0f 8f ea 04 00 00 jg 2118 <tts_heap_getsomeattrs+0x568>
attnum = slot->tts_nvalid;
1c2e: 49 0f bf 43 06 movsx rax,WORD PTR [r11+0x6]
values = slot->tts_values;
1c33: 49 8b 7b 18 mov rdi,QWORD PTR [r11+0x18]
slot->tts_nvalid = reqnatts;
1c37: 66 45 89 63 06 mov WORD PTR [r11+0x6],r12w
if (attnum < firstNonGuaranteedAttr)
1c3c: 48 39 e8 cmp rax,rbp
1c3f: 73 7f jae 1cc0 <tts_heap_getsomeattrs+0x110>
1c41: 48 89 54 24 f0 mov QWORD PTR [rsp-0x10],rdx
1c46: 48 8d 74 c2 20 lea rsi,[rdx+rax*8+0x20]
1c4b: eb 22 jmp 1c6f <tts_heap_getsomeattrs+0xbf>
1c4d: 0f 1f 00 nop DWORD PTR [rax]
static inline Datum
fetch_att_noerr(const void *T, bool attbyval, int attlen)
{
if (attbyval)
{
switch (attlen)
1c50: 66 41 83 ff 01 cmp r15w,0x1
1c55: 74 59 je 1cb0 <tts_heap_getsomeattrs+0x100>
* Returns datum representation for a 64-bit integer.
*/
static inline Datum
Int64GetDatum(int64 X)
{
return (Datum) X;
1c57: 48 8b 12 mov rdx,QWORD PTR [rdx]
values[attnum] = fetch_att_noerr(tp + off, true, attlen);
1c5a: 48 89 14 c7 mov QWORD PTR [rdi+rax*8],rdx
attnum++;
1c5e: 48 83 c0 01 add rax,0x1
} while (attnum < firstNonGuaranteedAttr);
1c62: 48 83 c6 08 add rsi,0x8
1c66: 48 39 e8 cmp rax,rbp
1c69: 0f 83 11 03 00 00 jae 1f80 <tts_heap_getsomeattrs+0x3d0>
isnull[attnum] = false;
1c6f: c6 04 01 00 mov BYTE PTR [rcx+rax*1],0x0
off = cattr->attcacheoff;
1c73: 0f bf 16 movsx edx,WORD PTR [rsi]
cattr = &cattrs[attnum];
1c76: 49 89 f5 mov r13,rsi
attlen = cattr->attlen;
1c79: 44 0f b7 7e 02 movzx r15d,WORD PTR [rsi+0x2]
off = cattr->attcacheoff;
1c7e: 48 89 d3 mov rbx,rdx
values[attnum] = fetch_att_noerr(tp + off, true, attlen);
1c81: 4c 01 c2 add rdx,r8
1c84: 66 41 83 ff 02 cmp r15w,0x2
1c89: 74 15 je 1ca0 <tts_heap_getsomeattrs+0xf0>
1c8b: 66 41 83 ff 04 cmp r15w,0x4
1c90: 75 be jne 1c50 <tts_heap_getsomeattrs+0xa0>
return (Datum) X;
1c92: 48 63 12 movsxd rdx,DWORD PTR [rdx]
{
case sizeof(int32):
return Int32GetDatum(*((const int32 *) T));
1c95: eb c3 jmp 1c5a <tts_heap_getsomeattrs+0xaa>
1c97: 66 0f 1f 84 00 00 00 nop WORD PTR [rax+rax*1+0x0]
1c9e: 00 00
return (Datum) X;
1ca0: 48 0f bf 12 movsx rdx,WORD PTR [rdx]
case sizeof(int16):
return Int16GetDatum(*((const int16 *) T));
1ca4: eb b4 jmp 1c5a <tts_heap_getsomeattrs+0xaa>
1ca6: 66 2e 0f 1f 84 00 00 cs nop WORD PTR [rax+rax*1+0x0]
1cad: 00 00 00
return (Datum) X;
1cb0: 48 0f be 12 movsx rdx,BYTE PTR [rdx]
case sizeof(char):
return CharGetDatum(*((const char *) T));
1cb4: eb a4 jmp 1c5a <tts_heap_getsomeattrs+0xaa>
1cb6: 66 2e 0f 1f 84 00 00 cs nop WORD PTR [rax+rax*1+0x0]
1cbd: 00 00 00
off = *offp;
1cc0: 41 8b 5b 48 mov ebx,DWORD PTR [r11+0x48]
if (unlikely(attnum < reqnatts))
1cc4: 49 63 ec movsxd rbp,r12d
firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, natts);
1cc7: 45 39 ca cmp r10d,r9d
1cca: 4d 0f 4f d1 cmovg r10,r9
if (attnum < firstNonCacheOffsetAttr)
1cce: 4c 39 d0 cmp rax,r10
1cd1: 0f 82 b9 01 00 00 jb 1e90 <tts_heap_getsomeattrs+0x2e0>
for (; attnum < firstNullAttr; attnum++)
1cd7: 4d 63 d6 movsxd r10,r14d
1cda: 4c 39 d0 cmp rax,r10
1cdd: 72 5e jb 1d3d <tts_heap_getsomeattrs+0x18d>
1cdf: e9 24 05 00 00 jmp 2208 <tts_heap_getsomeattrs+0x658>
1ce4: 0f 1f 40 00 nop DWORD PTR [rax+0x0]
if (attlen > 0)
{
const char *offset_ptr;
*off = TYPEALIGN(attalignby, *off);
1ce8: 8d 5c 1e ff lea ebx,[rsi+rbx*1-0x1]
1cec: f7 de neg esi
1cee: 21 de and esi,ebx
offset_ptr = tupptr + *off;
*off += attlen;
1cf0: 41 0f bf dd movsx ebx,r13w
offset_ptr = tupptr + *off;
1cf4: 41 89 f6 mov r14d,esi
*off += attlen;
1cf7: 01 f3 add ebx,esi
offset_ptr = tupptr + *off;
1cf9: 4d 01 c6 add r14,r8
return (Datum) (uintptr_t) X;
1cfc: 4c 89 f6 mov rsi,r14
if (attbyval)
1cff: 80 7c c2 24 00 cmp BYTE PTR [rdx+rax*8+0x24],0x0
1d04: 74 2a je 1d30 <tts_heap_getsomeattrs+0x180>
{
switch (attlen)
1d06: 66 41 83 fd 02 cmp r13w,0x2
1d0b: 0f 84 ef 01 00 00 je 1f00 <tts_heap_getsomeattrs+0x350>
1d11: 66 41 83 fd 04 cmp r13w,0x4
1d16: 0f 84 d4 01 00 00 je 1ef0 <tts_heap_getsomeattrs+0x340>
1d1c: 66 41 83 fd 01 cmp r13w,0x1
1d21: 0f 85 b9 01 00 00 jne 1ee0 <tts_heap_getsomeattrs+0x330>
return (Datum) X;
1d27: 49 0f be 36 movsx rsi,BYTE PTR [r14]
1d2b: 0f 1f 44 00 00 nop DWORD PTR [rax+rax*1+0x0]
values[attnum] = align_fetch_then_add(tp,
1d30: 48 89 34 c7 mov QWORD PTR [rdi+rax*8],rsi
for (; attnum < firstNullAttr; attnum++)
1d34: 48 83 c0 01 add rax,0x1
1d38: 4c 39 d0 cmp rax,r10
1d3b: 74 73 je 1db0 <tts_heap_getsomeattrs+0x200>
isnull[attnum] = false;
1d3d: c6 04 01 00 mov BYTE PTR [rcx+rax*1],0x0
attlen = cattr->attlen;
1d41: 44 0f b7 6c c2 22 movzx r13d,WORD PTR [rdx+rax*8+0x22]
cattr->attalignby);
1d47: 0f b6 74 c2 25 movzx esi,BYTE PTR [rdx+rax*8+0x25]
if (attlen > 0)
1d4c: 66 45 85 ed test r13w,r13w
1d50: 7f 96 jg 1ce8 <tts_heap_getsomeattrs+0x138>
}
return PointerGetDatum(offset_ptr);
}
else if (attlen == -1)
{
if (!VARATT_IS_SHORT(tupptr + *off))
1d52: 41 89 dd mov r13d,ebx
1d55: 4d 01 c5 add r13,r8
1d58: 41 f6 45 00 01 test BYTE PTR [r13+0x0],0x1
1d5d: 75 0e jne 1d6d <tts_heap_getsomeattrs+0x1bd>
*off = TYPEALIGN(attalignby, *off);
1d5f: 8d 5c 1e ff lea ebx,[rsi+rbx*1-0x1]
1d63: f7 de neg esi
1d65: 21 f3 and ebx,esi
res = PointerGetDatum(tupptr + *off);
1d67: 41 89 dd mov r13d,ebx
1d6a: 4d 01 c5 add r13,r8
if (VARATT_IS_1B_E(PTR))
1d6d: 45 0f b6 75 00 movzx r14d,BYTE PTR [r13+0x0]
return (Datum) (uintptr_t) X;
1d72: 4c 89 ee mov rsi,r13
1d75: 41 80 fe 01 cmp r14b,0x1
1d79: 0f 84 59 03 00 00 je 20d8 <tts_heap_getsomeattrs+0x528>
else if (VARATT_IS_1B(PTR))
1d7f: 41 f6 c6 01 test r14b,0x1
1d83: 0f 85 87 02 00 00 jne 2010 <tts_heap_getsomeattrs+0x460>
return VARSIZE_4B(PTR);
1d89: 45 8b 75 00 mov r14d,DWORD PTR [r13+0x0]
1d8d: 41 c1 ee 02 shr r14d,0x2
values[attnum] = align_fetch_then_add(tp,
1d91: 48 89 34 c7 mov QWORD PTR [rdi+rax*8],rsi
for (; attnum < firstNullAttr; attnum++)
1d95: 48 83 c0 01 add rax,0x1
*off += VARSIZE_ANY(DatumGetPointer(res));
1d99: 44 01 f3 add ebx,r14d
1d9c: 4c 39 d0 cmp rax,r10
1d9f: 75 9c jne 1d3d <tts_heap_getsomeattrs+0x18d>
1da1: 0f 1f 40 00 nop DWORD PTR [rax+0x0]
1da5: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
1dac: 00 00 00 00
for (; attnum < natts; attnum++)
1db0: 4d 39 ca cmp r10,r9
1db3: 0f 83 57 04 00 00 jae 2210 <tts_heap_getsomeattrs+0x660>
1db9: 0f 1f 80 00 00 00 00 nop DWORD PTR [rax+0x0]
if (isnull[attnum])
1dc0: 31 c0 xor eax,eax
1dc2: 42 80 3c 11 00 cmp BYTE PTR [rcx+r10*1],0x0
1dc7: 75 57 jne 1e20 <tts_heap_getsomeattrs+0x270>
attlen = cattr->attlen;
1dc9: 42 0f b7 74 d2 22 movzx esi,WORD PTR [rdx+r10*8+0x22]
cattr->attalignby);
1dcf: 42 0f b6 44 d2 25 movzx eax,BYTE PTR [rdx+r10*8+0x25]
if (attlen > 0)
1dd5: 66 85 f6 test si,si
1dd8: 0f 8e 32 01 00 00 jle 1f10 <tts_heap_getsomeattrs+0x360>
*off = TYPEALIGN(attalignby, *off);
1dde: 8d 5c 18 ff lea ebx,[rax+rbx*1-0x1]
1de2: f7 d8 neg eax
1de4: 21 d8 and eax,ebx
*off += attlen;
1de6: 0f bf de movsx ebx,si
offset_ptr = tupptr + *off;
1de9: 41 89 c5 mov r13d,eax
*off += attlen;
1dec: 01 c3 add ebx,eax
offset_ptr = tupptr + *off;
1dee: 4d 01 c5 add r13,r8
1df1: 4c 89 e8 mov rax,r13
if (attbyval)
1df4: 42 80 7c d2 24 00 cmp BYTE PTR [rdx+r10*8+0x24],0x0
1dfa: 74 24 je 1e20 <tts_heap_getsomeattrs+0x270>
switch (attlen)
1dfc: 66 83 fe 02 cmp si,0x2
1e00: 0f 84 b2 02 00 00 je 20b8 <tts_heap_getsomeattrs+0x508>
1e06: 66 83 fe 04 cmp si,0x4
1e0a: 0f 84 88 02 00 00 je 2098 <tts_heap_getsomeattrs+0x4e8>
1e10: 66 83 fe 01 cmp si,0x1
1e14: 0f 84 d6 01 00 00 je 1ff0 <tts_heap_getsomeattrs+0x440>
return (Datum) X;
1e1a: 49 8b 45 00 mov rax,QWORD PTR [r13+0x0]
1e1e: 66 90 xchg ax,ax
values[attnum] = (Datum) 0;
1e20: 4a 89 04 d7 mov QWORD PTR [rdi+r10*8],rax
for (; attnum < natts; attnum++)
1e24: 49 83 c2 01 add r10,0x1
1e28: 4d 39 ca cmp r10,r9
1e2b: 75 93 jne 1dc0 <tts_heap_getsomeattrs+0x210>
if (unlikely(attnum < reqnatts))
1e2d: 49 39 e9 cmp r9,rbp
1e30: 0f 82 ea 03 00 00 jb 2220 <tts_heap_getsomeattrs+0x670>
*offp = off;
1e36: 41 89 5b 48 mov DWORD PTR [r11+0x48],ebx
}
1e3a: 5b pop rbx
1e3b: 5d pop rbp
1e3c: 41 5c pop r12
1e3e: 41 5d pop r13
1e40: 41 5e pop r14
1e42: 41 5f pop r15
1e44: c3 ret
1e45: 0f 1f 00 nop DWORD PTR [rax]
switch (attlen)
1e48: 66 83 fb 01 cmp bx,0x1
1e4c: 0f 84 0e 01 00 00 je 1f60 <tts_heap_getsomeattrs+0x3b0>
1e52: 48 8b 1e mov rbx,QWORD PTR [rsi]
1e55: 66 2e 0f 1f 84 00 00 cs nop WORD PTR [rax+rax*1+0x0]
1e5c: 00 00 00
1e5f: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
1e66: 00 00 00 00
1e6a: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
1e71: 00 00 00 00
1e75: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
1e7c: 00 00 00 00
} while (++attnum < firstNonCacheOffsetAttr);
1e80: 48 8d 70 01 lea rsi,[rax+0x1]
values[attnum] = fetch_att_noerr(tp + off,
1e84: 48 89 1c c7 mov QWORD PTR [rdi+rax*8],rbx
} while (++attnum < firstNonCacheOffsetAttr);
1e88: 49 39 f2 cmp r10,rsi
1e8b: 74 43 je 1ed0 <tts_heap_getsomeattrs+0x320>
1e8d: 48 89 f0 mov rax,rsi
isnull[attnum] = false;
1e90: c6 04 01 00 mov BYTE PTR [rcx+rax*1],0x0
off = cattr->attcacheoff;
1e94: 0f bf 74 c2 20 movsx esi,WORD PTR [rdx+rax*8+0x20]
1e99: 49 89 f5 mov r13,rsi
values[attnum] = fetch_att_noerr(tp + off,
1e9c: 4c 01 c6 add rsi,r8
return (Datum) (uintptr_t) X;
1e9f: 48 89 f3 mov rbx,rsi
if (attbyval)
1ea2: 80 7c c2 24 00 cmp BYTE PTR [rdx+rax*8+0x24],0x0
1ea7: 74 d7 je 1e80 <tts_heap_getsomeattrs+0x2d0>
cattr->attlen);
1ea9: 0f b7 5c c2 22 movzx ebx,WORD PTR [rdx+rax*8+0x22]
switch (attlen)
1eae: 66 83 fb 02 cmp bx,0x2
1eb2: 0f 84 b8 00 00 00 je 1f70 <tts_heap_getsomeattrs+0x3c0>
1eb8: 66 83 fb 04 cmp bx,0x4
1ebc: 75 8a jne 1e48 <tts_heap_getsomeattrs+0x298>
return (Datum) X;
1ebe: 48 63 1e movsxd rbx,DWORD PTR [rsi]
} while (++attnum < firstNonCacheOffsetAttr);
1ec1: 48 8d 70 01 lea rsi,[rax+0x1]
values[attnum] = fetch_att_noerr(tp + off,
1ec5: 48 89 1c c7 mov QWORD PTR [rdi+rax*8],rbx
} while (++attnum < firstNonCacheOffsetAttr);
1ec9: 49 39 f2 cmp r10,rsi
1ecc: 75 bf jne 1e8d <tts_heap_getsomeattrs+0x2dd>
1ece: 66 90 xchg ax,ax
off += cattr->attlen;
1ed0: 0f bf 5c c2 22 movsx ebx,WORD PTR [rdx+rax*8+0x22]
} while (++attnum < firstNonCacheOffsetAttr);
1ed5: 4c 89 d0 mov rax,r10
off += cattr->attlen;
1ed8: 44 01 eb add ebx,r13d
1edb: e9 f7 fd ff ff jmp 1cd7 <tts_heap_getsomeattrs+0x127>
return (Datum) X;
1ee0: 49 8b 36 mov rsi,QWORD PTR [r14]
return Int64GetDatum(*((const int64 *) offset_ptr));
1ee3: e9 48 fe ff ff jmp 1d30 <tts_heap_getsomeattrs+0x180>
1ee8: 0f 1f 84 00 00 00 00 nop DWORD PTR [rax+rax*1+0x0]
1eef: 00
return (Datum) X;
1ef0: 49 63 36 movsxd rsi,DWORD PTR [r14]
return Int32GetDatum(*((const int32 *) offset_ptr));
1ef3: e9 38 fe ff ff jmp 1d30 <tts_heap_getsomeattrs+0x180>
1ef8: 0f 1f 84 00 00 00 00 nop DWORD PTR [rax+rax*1+0x0]
1eff: 00
return (Datum) X;
1f00: 49 0f bf 36 movsx rsi,WORD PTR [r14]
return Int16GetDatum(*((const int16 *) offset_ptr));
1f04: e9 27 fe ff ff jmp 1d30 <tts_heap_getsomeattrs+0x180>
1f09: 0f 1f 80 00 00 00 00 nop DWORD PTR [rax+0x0]
if (!VARATT_IS_SHORT(tupptr + *off))
1f10: 89 de mov esi,ebx
1f12: 4c 01 c6 add rsi,r8
1f15: f6 06 01 test BYTE PTR [rsi],0x1
1f18: 0f 84 02 01 00 00 je 2020 <tts_heap_getsomeattrs+0x470>
if (VARATT_IS_1B_E(PTR))
1f1e: 44 0f b6 2e movzx r13d,BYTE PTR [rsi]
return (Datum) (uintptr_t) X;
1f22: 48 89 f0 mov rax,rsi
1f25: 41 80 fd 01 cmp r13b,0x1
1f29: 0f 84 0f 01 00 00 je 203e <tts_heap_getsomeattrs+0x48e>
else if (VARATT_IS_1B(PTR))
1f2f: 41 f6 c5 01 test r13b,0x1
1f33: 0f 84 3f 01 00 00 je 2078 <tts_heap_getsomeattrs+0x4c8>
return VARSIZE_1B(PTR);
1f39: 41 d0 ed shr r13b,1
*off += VARSIZE_ANY(DatumGetPointer(res));
1f3c: 45 0f b6 ed movzx r13d,r13b
values[attnum] = (Datum) 0;
1f40: 4a 89 04 d7 mov QWORD PTR [rdi+r10*8],rax
for (; attnum < natts; attnum++)
1f44: 49 83 c2 01 add r10,0x1
1f48: 44 01 eb add ebx,r13d
1f4b: 4d 39 ca cmp r10,r9
1f4e: 0f 85 6c fe ff ff jne 1dc0 <tts_heap_getsomeattrs+0x210>
1f54: e9 d4 fe ff ff jmp 1e2d <tts_heap_getsomeattrs+0x27d>
1f59: 0f 1f 80 00 00 00 00 nop DWORD PTR [rax+0x0]
return (Datum) X;
1f60: 48 0f be 1e movsx rbx,BYTE PTR [rsi]
return CharGetDatum(*((const char *) T));
1f64: e9 17 ff ff ff jmp 1e80 <tts_heap_getsomeattrs+0x2d0>
1f69: 0f 1f 80 00 00 00 00 nop DWORD PTR [rax+0x0]
return (Datum) X;
1f70: 48 0f bf 1e movsx rbx,WORD PTR [rsi]
return Int16GetDatum(*((const int16 *) T));
1f74: e9 07 ff ff ff jmp 1e80 <tts_heap_getsomeattrs+0x2d0>
1f79: 0f 1f 80 00 00 00 00 nop DWORD PTR [rax+0x0]
off += cattr->attlen;
1f80: 41 0f bf 75 02 movsx esi,WORD PTR [r13+0x2]
if (attnum == reqnatts)
1f85: 49 63 ec movsxd rbp,r12d
off += cattr->attlen;
1f88: 48 8b 54 24 f0 mov rdx,QWORD PTR [rsp-0x10]
1f8d: 01 f3 add ebx,esi
if (attnum == reqnatts)
1f8f: 48 39 e8 cmp rax,rbp
1f92: 0f 85 2f fd ff ff jne 1cc7 <tts_heap_getsomeattrs+0x117>
*offp = off;
1f98: 41 89 5b 48 mov DWORD PTR [r11+0x48],ebx
}
1f9c: 5b pop rbx
1f9d: 5d pop rbp
1f9e: 41 5c pop r12
1fa0: 41 5d pop r13
1fa2: 41 5e pop r14
1fa4: 41 5f pop r15
1fa6: c3 ret
1fa7: 66 0f 1f 84 00 00 00 nop WORD PTR [rax+rax*1+0x0]
1fae: 00 00
tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits));
1fb0: 4c 8d 43 18 lea r8,[rbx+0x18]
if (reqnatts > firstNonGuaranteedAttr)
1fb4: 41 39 c4 cmp r12d,eax
1fb7: 0f 8e cb 00 00 00 jle 2088 <tts_heap_getsomeattrs+0x4d8>
natts = Min(HeapTupleHeaderGetNatts(tup), reqnatts);
1fbd: 0f b7 43 12 movzx eax,WORD PTR [rbx+0x12]
1fc1: 25 ff 07 00 00 and eax,0x7ff
1fc6: 44 39 e0 cmp eax,r12d
1fc9: 41 0f 4f c4 cmovg eax,r12d
1fcd: 41 89 c6 mov r14d,eax
1fd0: 4c 63 c8 movsxd r9,eax
1fd3: e9 56 fc ff ff jmp 1c2e <tts_heap_getsomeattrs+0x7e>
1fd8: 0f 1f 84 00 00 00 00 nop DWORD PTR [rax+rax*1+0x0]
1fdf: 00
1fe0: 31 ed xor ebp,ebp
firstNonGuaranteedAttr = 0;
1fe2: 31 c0 xor eax,eax
1fe4: e9 02 fc ff ff jmp 1beb <tts_heap_getsomeattrs+0x3b>
1fe9: 0f 1f 80 00 00 00 00 nop DWORD PTR [rax+0x0]
return (Datum) X;
1ff0: 49 0f be 45 00 movsx rax,BYTE PTR [r13+0x0]
values[attnum] = (Datum) 0;
1ff5: 4a 89 04 d7 mov QWORD PTR [rdi+r10*8],rax
for (; attnum < natts; attnum++)
1ff9: 49 83 c2 01 add r10,0x1
1ffd: 4d 39 ca cmp r10,r9
2000: 0f 85 ba fd ff ff jne 1dc0 <tts_heap_getsomeattrs+0x210>
2006: e9 22 fe ff ff jmp 1e2d <tts_heap_getsomeattrs+0x27d>
200b: 0f 1f 44 00 00 nop DWORD PTR [rax+rax*1+0x0]
2010: 41 d0 ee shr r14b,1
*off += VARSIZE_ANY(DatumGetPointer(res));
2013: 45 0f b6 f6 movzx r14d,r14b
2017: e9 75 fd ff ff jmp 1d91 <tts_heap_getsomeattrs+0x1e1>
201c: 0f 1f 40 00 nop DWORD PTR [rax+0x0]
*off = TYPEALIGN(attalignby, *off);
2020: 8d 5c 18 ff lea ebx,[rax+rbx*1-0x1]
2024: f7 d8 neg eax
2026: 21 c3 and ebx,eax
res = PointerGetDatum(tupptr + *off);
2028: 89 de mov esi,ebx
202a: 4c 01 c6 add rsi,r8
if (VARATT_IS_1B_E(PTR))
202d: 44 0f b6 2e movzx r13d,BYTE PTR [rsi]
return (Datum) (uintptr_t) X;
2031: 48 89 f0 mov rax,rsi
2034: 41 80 fd 01 cmp r13b,0x1
2038: 0f 85 f1 fe ff ff jne 1f2f <tts_heap_getsomeattrs+0x37f>
return VARTAG_1B_E(PTR);
203e: 0f b6 76 01 movzx esi,BYTE PTR [rsi+0x1]
if (tag == VARTAG_INDIRECT)
2042: 83 fe 01 cmp esi,0x1
2045: 0f 84 12 02 00 00 je 225d <tts_heap_getsomeattrs+0x6ad>
return ((tag & ~1) == VARTAG_EXPANDED_RO);
204b: 41 89 f5 mov r13d,esi
204e: 41 83 e5 fe and r13d,0xfffffffe
else if (VARTAG_IS_EXPANDED(tag))
2052: 41 83 fd 02 cmp r13d,0x2
2056: 0f 84 01 02 00 00 je 225d <tts_heap_getsomeattrs+0x6ad>
else if (tag == VARTAG_ONDISK)
205c: 83 fe 12 cmp esi,0x12
205f: 40 0f 94 c6 sete sil
2063: 40 0f b6 f6 movzx esi,sil
2067: 48 c1 e6 04 shl rsi,0x4
*off += VARSIZE_ANY(DatumGetPointer(res));
206b: 44 8d 6e 02 lea r13d,[rsi+0x2]
206f: e9 cc fe ff ff jmp 1f40 <tts_heap_getsomeattrs+0x390>
2074: 0f 1f 40 00 nop DWORD PTR [rax+0x0]
return VARSIZE_4B(PTR);
2078: 44 8b 2e mov r13d,DWORD PTR [rsi]
207b: 41 c1 ed 02 shr r13d,0x2
207f: e9 bc fe ff ff jmp 1f40 <tts_heap_getsomeattrs+0x390>
2084: 0f 1f 40 00 nop DWORD PTR [rax+0x0]
natts = reqnatts;
2088: 4d 63 cc movsxd r9,r12d
208b: 45 89 e6 mov r14d,r12d
208e: e9 9b fb ff ff jmp 1c2e <tts_heap_getsomeattrs+0x7e>
2093: 0f 1f 44 00 00 nop DWORD PTR [rax+rax*1+0x0]
return (Datum) X;
2098: 49 63 45 00 movsxd rax,DWORD PTR [r13+0x0]
values[attnum] = (Datum) 0;
209c: 4a 89 04 d7 mov QWORD PTR [rdi+r10*8],rax
for (; attnum < natts; attnum++)
20a0: 49 83 c2 01 add r10,0x1
20a4: 4d 39 ca cmp r10,r9
20a7: 0f 85 13 fd ff ff jne 1dc0 <tts_heap_getsomeattrs+0x210>
20ad: e9 7b fd ff ff jmp 1e2d <tts_heap_getsomeattrs+0x27d>
20b2: 66 0f 1f 44 00 00 nop WORD PTR [rax+rax*1+0x0]
return (Datum) X;
20b8: 49 0f bf 45 00 movsx rax,WORD PTR [r13+0x0]
values[attnum] = (Datum) 0;
20bd: 4a 89 04 d7 mov QWORD PTR [rdi+r10*8],rax
for (; attnum < natts; attnum++)
20c1: 49 83 c2 01 add r10,0x1
20c5: 4d 39 ca cmp r10,r9
20c8: 0f 85 f2 fc ff ff jne 1dc0 <tts_heap_getsomeattrs+0x210>
20ce: e9 5a fd ff ff jmp 1e2d <tts_heap_getsomeattrs+0x27d>
20d3: 0f 1f 44 00 00 nop DWORD PTR [rax+rax*1+0x0]
return VARTAG_1B_E(PTR);
20d8: 45 0f b6 6d 01 movzx r13d,BYTE PTR [r13+0x1]
if (tag == VARTAG_INDIRECT)
20dd: 41 83 fd 01 cmp r13d,0x1
20e1: 0f 84 6b 01 00 00 je 2252 <tts_heap_getsomeattrs+0x6a2>
return ((tag & ~1) == VARTAG_EXPANDED_RO);
20e7: 45 89 ee mov r14d,r13d
20ea: 41 83 e6 fe and r14d,0xfffffffe
else if (VARTAG_IS_EXPANDED(tag))
20ee: 41 83 fe 02 cmp r14d,0x2
20f2: 0f 84 5a 01 00 00 je 2252 <tts_heap_getsomeattrs+0x6a2>
else if (tag == VARTAG_ONDISK)
20f8: 41 83 fd 12 cmp r13d,0x12
20fc: 41 0f 94 c5 sete r13b
2100: 45 0f b6 ed movzx r13d,r13b
2104: 49 c1 e5 04 shl r13,0x4
2108: 45 8d 75 02 lea r14d,[r13+0x2]
210c: e9 80 fc ff ff jmp 1d91 <tts_heap_getsomeattrs+0x1e1>
2111: 0f 1f 80 00 00 00 00 nop DWORD PTR [rax+0x0]
* case.
*/
static inline int
first_null_attr(const bits8 *bits, int natts)
{
int nattByte = natts >> 3;
2118: 45 89 cd mov r13d,r9d
211b: 41 c1 fd 03 sar r13d,0x3
}
}
#endif
/* Process all bytes up to just before the byte for the natts attribute */
for (bytenum = 0; bytenum < nattByte; bytenum++)
211f: 45 85 ed test r13d,r13d
2122: 0f 8e 40 01 00 00 jle 2268 <tts_heap_getsomeattrs+0x6b8>
2128: 48 8d 73 17 lea rsi,[rbx+0x17]
212c: 31 ff xor edi,edi
212e: eb 20 jmp 2150 <tts_heap_getsomeattrs+0x5a0>
2130: 0f 1f 44 00 00 nop DWORD PTR [rax+rax*1+0x0]
2135: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
213c: 00 00 00 00
2140: 83 c7 01 add edi,0x1
2143: 48 83 c6 01 add rsi,0x1
2147: 41 39 fd cmp r13d,edi
214a: 0f 84 ec 00 00 00 je 223c <tts_heap_getsomeattrs+0x68c>
{
/* break if there's any NULL attrs (a 0 bit) */
if (bits[bytenum] != 0xFF)
2150: 0f b6 06 movzx eax,BYTE PTR [rsi]
2153: 3c ff cmp al,0xff
2155: 74 e9 je 2140 <tts_heap_getsomeattrs+0x590>
* looking for the first 1-bit. This works even when the byte is 0xFF, as
* the bitwise NOT of 0xFF in 32 bits is 0xFFFFFF00, in which case
* pg_rightmost_one_pos32() will return 8. We may end up with a value
* higher than natts here, but we'll fix that with the Min() below.
*/
res = bytenum << 3;
2157: c1 e7 03 shl edi,0x3
res += pg_rightmost_one_pos32(~((uint32) bits[bytenum]));
215a: f7 d0 not eax
int nbytes = (natts + 7) >> 3;
215c: 45 8d 69 07 lea r13d,[r9+0x7]
pg_rightmost_one_pos32(uint32 word)
{
#ifdef HAVE__BUILTIN_CTZ
Assert(word != 0);
return __builtin_ctz(word);
2160: f3 0f bc c0 tzcnt eax,eax
res += pg_rightmost_one_pos32(~((uint32) bits[bytenum]));
2164: 01 f8 add eax,edi
/*
* Since we did no masking to mask out bits beyond the natt'th bit, we may
* have found a bit higher than natts, so we must cap res to natts
*/
res = Min(res, natts);
2166: 41 39 c1 cmp r9d,eax
2169: 41 0f 4e c1 cmovle eax,r9d
int nbytes = (natts + 7) >> 3;
216d: 41 c1 fd 03 sar r13d,0x3
res = Min(res, natts);
2171: 4c 63 f0 movsxd r14,eax
isnull_8 &= UINT64CONST(0x0101010101010101);
2174: 49 bf 01 01 01 01 01 movabs r15,0x101010101010101
217b: 01 01 01
217e: 4d 63 ed movsxd r13,r13d
for (bytenum = 0; bytenum < nattByte; bytenum++)
2181: 31 ff xor edi,edi
2183: 66 0f 1f 44 00 00 nop WORD PTR [rax+rax*1+0x0]
2189: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
2190: 00 00 00 00
2194: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
219b: 00 00 00 00
219f: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
21a6: 00 00 00 00
21aa: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
21b1: 00 00 00 00
21b5: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
21bc: 00 00 00 00
bits8 nullbyte = ~bits[i];
21c0: 0f b6 74 3b 17 movzx esi,BYTE PTR [rbx+rdi*1+0x17]
21c5: f7 d6 not esi
isnull_8 |= ((uint64) ((nullbyte >> 4) * SPREAD_BITS_MULTIPLIER_32)) << 32;
21c7: 89 f0 mov eax,esi
isnull_8 = (nullbyte & 0xf) * SPREAD_BITS_MULTIPLIER_32;
21c9: 83 e6 0f and esi,0xf
isnull_8 |= ((uint64) ((nullbyte >> 4) * SPREAD_BITS_MULTIPLIER_32)) << 32;
21cc: c0 e8 04 shr al,0x4
isnull_8 = (nullbyte & 0xf) * SPREAD_BITS_MULTIPLIER_32;
21cf: 48 69 f6 81 40 20 00 imul rsi,rsi,0x204081
isnull_8 |= ((uint64) ((nullbyte >> 4) * SPREAD_BITS_MULTIPLIER_32)) << 32;
21d6: 83 e0 0f and eax,0xf
21d9: 48 69 c0 81 40 20 00 imul rax,rax,0x204081
21e0: 48 c1 e0 20 shl rax,0x20
21e4: 48 09 f0 or rax,rsi
isnull_8 &= UINT64CONST(0x0101010101010101);
21e7: 4c 21 f8 and rax,r15
21ea: 48 89 04 f9 mov QWORD PTR [rcx+rdi*8],rax
for (int i = 0; i < nbytes; i++, isnull += 8)
21ee: 48 83 c7 01 add rdi,0x1
21f2: 4c 39 ef cmp rdi,r13
21f5: 75 c9 jne 21c0 <tts_heap_getsomeattrs+0x610>
firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, firstNullAttr);
21f7: 45 39 f2 cmp r10d,r14d
21fa: 4d 0f 4f d6 cmovg r10,r14
21fe: e9 2b fa ff ff jmp 1c2e <tts_heap_getsomeattrs+0x7e>
2203: 0f 1f 44 00 00 nop DWORD PTR [rax+rax*1+0x0]
for (; attnum < firstNullAttr; attnum++)
2208: 49 89 c2 mov r10,rax
220b: e9 a0 fb ff ff jmp 1db0 <tts_heap_getsomeattrs+0x200>
for (; attnum < natts; attnum++)
2210: 4d 89 d1 mov r9,r10
2213: e9 15 fc ff ff jmp 1e2d <tts_heap_getsomeattrs+0x27d>
2218: 0f 1f 84 00 00 00 00 nop DWORD PTR [rax+rax*1+0x0]
221f: 00
*offp = off;
2220: 41 89 5b 48 mov DWORD PTR [r11+0x48],ebx
slot_getmissingattrs(slot, attnum, reqnatts);
2224: 44 89 e2 mov edx,r12d
}
2227: 5b pop rbx
slot_getmissingattrs(slot, attnum, reqnatts);
2228: 44 89 ce mov esi,r9d
}
222b: 5d pop rbp
slot_getmissingattrs(slot, attnum, reqnatts);
222c: 4c 89 df mov rdi,r11
}
222f: 41 5c pop r12
2231: 41 5d pop r13
2233: 41 5e pop r14
2235: 41 5f pop r15
slot_getmissingattrs(slot, attnum, reqnatts);
2237: e9 b4 f8 ff ff jmp 1af0 <slot_getmissingattrs>
res = bytenum << 3;
223c: 42 8d 3c ed 00 00 00 lea edi,[r13*8+0x0]
2243: 00
res += pg_rightmost_one_pos32(~((uint32) bits[bytenum]));
2244: 4d 63 ed movsxd r13,r13d
2247: 42 0f b6 44 2b 17 movzx eax,BYTE PTR [rbx+r13*1+0x17]
224d: e9 08 ff ff ff jmp 215a <tts_heap_getsomeattrs+0x5aa>
2252: 41 be 0a 00 00 00 mov r14d,0xa
2258: e9 34 fb ff ff jmp 1d91 <tts_heap_getsomeattrs+0x1e1>
225d: 41 bd 0a 00 00 00 mov r13d,0xa
2263: e9 d8 fc ff ff jmp 1f40 <tts_heap_getsomeattrs+0x390>
2268: 0f b6 43 17 movzx eax,BYTE PTR [rbx+0x17]
int nbytes = (natts + 7) >> 3;
226c: 45 8d 69 07 lea r13d,[r9+0x7]
res += pg_rightmost_one_pos32(~((uint32) bits[bytenum]));
2270: f7 d0 not eax
2272: f3 0f bc c0 tzcnt eax,eax
res = Min(res, natts);
2276: 41 39 c1 cmp r9d,eax
2279: 41 0f 4e c1 cmovle eax,r9d
int nbytes = (natts + 7) >> 3;
227d: 41 c1 fd 03 sar r13d,0x3
res = Min(res, natts);
2281: 4c 63 f0 movsxd r14,eax
for (int i = 0; i < nbytes; i++, isnull += 8)
2284: 41 83 fd 01 cmp r13d,0x1
2288: 0f 85 69 ff ff ff jne 21f7 <tts_heap_getsomeattrs+0x647>
228e: e9 e1 fe ff ff jmp 2174 <tts_heap_getsomeattrs+0x5c4>
2293: 66 90 xchg ax,ax
2295: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
229c: 00 00 00 00
00000000000022a0 <tts_minimal_getsomeattrs>:
{
22a0: f3 0f 1e fa endbr64
22a4: 41 57 push r15
22a6: 49 89 fb mov r11,rdi
22a9: 41 56 push r14
22ab: 41 55 push r13
22ad: 41 54 push r12
22af: 4c 63 e6 movsxd r12,esi
22b2: 55 push rbp
22b3: 53 push rbx
HeapTupleHeader tup = tuple->t_data;
22b4: 48 8b 47 40 mov rax,QWORD PTR [rdi+0x40]
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
22b8: 48 8b 57 10 mov rdx,QWORD PTR [rdi+0x10]
isnull = slot->tts_isnull;
22bc: 48 8b 4f 20 mov rcx,QWORD PTR [rdi+0x20]
HeapTupleHeader tup = tuple->t_data;
22c0: 48 8b 58 10 mov rbx,QWORD PTR [rax+0x10]
if (TTS_OBEYS_NOT_NULL_CONSTRAINTS(slot))
22c4: f6 47 04 08 test BYTE PTR [rdi+0x4],0x8
22c8: 0f 84 22 04 00 00 je 26f0 <tts_minimal_getsomeattrs+0x450>
firstNonGuaranteedAttr = Min(reqnatts, tupleDesc->firstNonGuaranteedAttr);
22ce: 8b 42 14 mov eax,DWORD PTR [rdx+0x14]
22d1: 41 39 c4 cmp r12d,eax
22d4: 41 0f 4e c4 cmovle eax,r12d
if (attnum < firstNonGuaranteedAttr)
22d8: 48 63 e8 movsxd rbp,eax
firstNonCacheOffsetAttr = tupleDesc->firstNonCachedOffsetAttr;
22db: 4c 63 52 10 movsxd r10,DWORD PTR [rdx+0x10]
if (HeapTupleHasNulls(tuple))
22df: f6 43 14 01 test BYTE PTR [rbx+0x14],0x1
22e3: 0f 84 d7 03 00 00 je 26c0 <tts_minimal_getsomeattrs+0x420>
natts = HeapTupleHeaderGetNatts(tup);
22e9: 44 0f b7 4b 12 movzx r9d,WORD PTR [rbx+0x12]
22ee: 41 81 e1 ff 07 00 00 and r9d,0x7ff
22f5: 45 8d 41 07 lea r8d,[r9+0x7]
22f9: 41 c1 f8 03 sar r8d,0x3
tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits) +
22fd: 41 83 c0 1e add r8d,0x1e
2301: 41 81 e0 f8 03 00 00 and r8d,0x3f8
2308: 49 01 d8 add r8,rbx
natts = Min(natts, reqnatts);
230b: 45 39 cc cmp r12d,r9d
230e: 4d 0f 4e cc cmovle r9,r12
firstNullAttr = natts;
2312: 45 89 ce mov r14d,r9d
if (natts > firstNonGuaranteedAttr)
2315: 41 39 c1 cmp r9d,eax
2318: 0f 8f 0a 05 00 00 jg 2828 <tts_minimal_getsomeattrs+0x588>
attnum = slot->tts_nvalid;
231e: 49 0f bf 43 06 movsx rax,WORD PTR [r11+0x6]
values = slot->tts_values;
2323: 49 8b 7b 18 mov rdi,QWORD PTR [r11+0x18]
slot->tts_nvalid = reqnatts;
2327: 66 45 89 63 06 mov WORD PTR [r11+0x6],r12w
if (attnum < firstNonGuaranteedAttr)
232c: 48 39 e8 cmp rax,rbp
232f: 73 7f jae 23b0 <tts_minimal_getsomeattrs+0x110>
2331: 48 89 54 24 f0 mov QWORD PTR [rsp-0x10],rdx
2336: 48 8d 74 c2 20 lea rsi,[rdx+rax*8+0x20]
233b: eb 22 jmp 235f <tts_minimal_getsomeattrs+0xbf>
233d: 0f 1f 00 nop DWORD PTR [rax]
switch (attlen)
2340: 66 41 83 ff 01 cmp r15w,0x1
2345: 74 59 je 23a0 <tts_minimal_getsomeattrs+0x100>
return (Datum) X;
2347: 48 8b 12 mov rdx,QWORD PTR [rdx]
values[attnum] = fetch_att_noerr(tp + off, true, attlen);
234a: 48 89 14 c7 mov QWORD PTR [rdi+rax*8],rdx
attnum++;
234e: 48 83 c0 01 add rax,0x1
} while (attnum < firstNonGuaranteedAttr);
2352: 48 83 c6 08 add rsi,0x8
2356: 48 39 e8 cmp rax,rbp
2359: 0f 83 31 03 00 00 jae 2690 <tts_minimal_getsomeattrs+0x3f0>
isnull[attnum] = false;
235f: c6 04 01 00 mov BYTE PTR [rcx+rax*1],0x0
off = cattr->attcacheoff;
2363: 0f bf 16 movsx edx,WORD PTR [rsi]
cattr = &cattrs[attnum];
2366: 49 89 f5 mov r13,rsi
attlen = cattr->attlen;
2369: 44 0f b7 7e 02 movzx r15d,WORD PTR [rsi+0x2]
off = cattr->attcacheoff;
236e: 48 89 d3 mov rbx,rdx
values[attnum] = fetch_att_noerr(tp + off, true, attlen);
2371: 4c 01 c2 add rdx,r8
2374: 66 41 83 ff 02 cmp r15w,0x2
2379: 74 15 je 2390 <tts_minimal_getsomeattrs+0xf0>
237b: 66 41 83 ff 04 cmp r15w,0x4
2380: 75 be jne 2340 <tts_minimal_getsomeattrs+0xa0>
return (Datum) X;
2382: 48 63 12 movsxd rdx,DWORD PTR [rdx]
return Int32GetDatum(*((const int32 *) T));
2385: eb c3 jmp 234a <tts_minimal_getsomeattrs+0xaa>
2387: 66 0f 1f 84 00 00 00 nop WORD PTR [rax+rax*1+0x0]
238e: 00 00
return (Datum) X;
2390: 48 0f bf 12 movsx rdx,WORD PTR [rdx]
return Int16GetDatum(*((const int16 *) T));
2394: eb b4 jmp 234a <tts_minimal_getsomeattrs+0xaa>
2396: 66 2e 0f 1f 84 00 00 cs nop WORD PTR [rax+rax*1+0x0]
239d: 00 00 00
return (Datum) X;
23a0: 48 0f be 12 movsx rdx,BYTE PTR [rdx]
return CharGetDatum(*((const char *) T));
23a4: eb a4 jmp 234a <tts_minimal_getsomeattrs+0xaa>
23a6: 66 2e 0f 1f 84 00 00 cs nop WORD PTR [rax+rax*1+0x0]
23ad: 00 00 00
off = *offp;
23b0: 41 8b 5b 68 mov ebx,DWORD PTR [r11+0x68]
if (unlikely(attnum < reqnatts))
23b4: 49 63 ec movsxd rbp,r12d
firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, natts);
23b7: 45 39 ca cmp r10d,r9d
23ba: 4d 0f 4f d1 cmovg r10,r9
if (attnum < firstNonCacheOffsetAttr)
23be: 4c 39 d0 cmp rax,r10
23c1: 0f 82 c9 01 00 00 jb 2590 <tts_minimal_getsomeattrs+0x2f0>
for (; attnum < firstNullAttr; attnum++)
23c7: 4d 63 d6 movsxd r10,r14d
23ca: 4c 39 d0 cmp rax,r10
23cd: 72 5e jb 242d <tts_minimal_getsomeattrs+0x18d>
23cf: e9 34 05 00 00 jmp 2908 <tts_minimal_getsomeattrs+0x668>
23d4: 0f 1f 40 00 nop DWORD PTR [rax+0x0]
*off = TYPEALIGN(attalignby, *off);
23d8: 8d 5c 1e ff lea ebx,[rsi+rbx*1-0x1]
23dc: f7 de neg esi
23de: 21 de and esi,ebx
*off += attlen;
23e0: 41 0f bf dd movsx ebx,r13w
offset_ptr = tupptr + *off;
23e4: 41 89 f6 mov r14d,esi
*off += attlen;
23e7: 01 f3 add ebx,esi
offset_ptr = tupptr + *off;
23e9: 4d 01 c6 add r14,r8
return (Datum) (uintptr_t) X;
23ec: 4c 89 f6 mov rsi,r14
if (attbyval)
23ef: 80 7c c2 24 00 cmp BYTE PTR [rdx+rax*8+0x24],0x0
23f4: 74 2a je 2420 <tts_minimal_getsomeattrs+0x180>
switch (attlen)
23f6: 66 41 83 fd 02 cmp r13w,0x2
23fb: 0f 84 0f 02 00 00 je 2610 <tts_minimal_getsomeattrs+0x370>
2401: 66 41 83 fd 04 cmp r13w,0x4
2406: 0f 84 f4 01 00 00 je 2600 <tts_minimal_getsomeattrs+0x360>
240c: 66 41 83 fd 01 cmp r13w,0x1
2411: 0f 85 d9 01 00 00 jne 25f0 <tts_minimal_getsomeattrs+0x350>
return (Datum) X;
2417: 49 0f be 36 movsx rsi,BYTE PTR [r14]
241b: 0f 1f 44 00 00 nop DWORD PTR [rax+rax*1+0x0]
values[attnum] = align_fetch_then_add(tp,
2420: 48 89 34 c7 mov QWORD PTR [rdi+rax*8],rsi
for (; attnum < firstNullAttr; attnum++)
2424: 48 83 c0 01 add rax,0x1
2428: 4c 39 d0 cmp rax,r10
242b: 74 73 je 24a0 <tts_minimal_getsomeattrs+0x200>
isnull[attnum] = false;
242d: c6 04 01 00 mov BYTE PTR [rcx+rax*1],0x0
attlen = cattr->attlen;
2431: 44 0f b7 6c c2 22 movzx r13d,WORD PTR [rdx+rax*8+0x22]
cattr->attalignby);
2437: 0f b6 74 c2 25 movzx esi,BYTE PTR [rdx+rax*8+0x25]
if (attlen > 0)
243c: 66 45 85 ed test r13w,r13w
2440: 7f 96 jg 23d8 <tts_minimal_getsomeattrs+0x138>
if (!VARATT_IS_SHORT(tupptr + *off))
2442: 41 89 dd mov r13d,ebx
2445: 4d 01 c5 add r13,r8
2448: 41 f6 45 00 01 test BYTE PTR [r13+0x0],0x1
244d: 75 0e jne 245d <tts_minimal_getsomeattrs+0x1bd>
*off = TYPEALIGN(attalignby, *off);
244f: 8d 5c 1e ff lea ebx,[rsi+rbx*1-0x1]
2453: f7 de neg esi
2455: 21 f3 and ebx,esi
res = PointerGetDatum(tupptr + *off);
2457: 41 89 dd mov r13d,ebx
245a: 4d 01 c5 add r13,r8
if (VARATT_IS_1B_E(PTR))
245d: 45 0f b6 75 00 movzx r14d,BYTE PTR [r13+0x0]
return (Datum) (uintptr_t) X;
2462: 4c 89 ee mov rsi,r13
2465: 41 80 fe 01 cmp r14b,0x1
2469: 0f 84 79 03 00 00 je 27e8 <tts_minimal_getsomeattrs+0x548>
else if (VARATT_IS_1B(PTR))
246f: 41 f6 c6 01 test r14b,0x1
2473: 0f 85 a7 02 00 00 jne 2720 <tts_minimal_getsomeattrs+0x480>
return VARSIZE_4B(PTR);
2479: 45 8b 75 00 mov r14d,DWORD PTR [r13+0x0]
247d: 41 c1 ee 02 shr r14d,0x2
values[attnum] = align_fetch_then_add(tp,
2481: 48 89 34 c7 mov QWORD PTR [rdi+rax*8],rsi
for (; attnum < firstNullAttr; attnum++)
2485: 48 83 c0 01 add rax,0x1
*off += VARSIZE_ANY(DatumGetPointer(res));
2489: 44 01 f3 add ebx,r14d
248c: 4c 39 d0 cmp rax,r10
248f: 75 9c jne 242d <tts_minimal_getsomeattrs+0x18d>
2491: 0f 1f 40 00 nop DWORD PTR [rax+0x0]
2495: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
249c: 00 00 00 00
for (; attnum < natts; attnum++)
24a0: 4d 39 ca cmp r10,r9
24a3: 0f 83 67 04 00 00 jae 2910 <tts_minimal_getsomeattrs+0x670>
24a9: 0f 1f 80 00 00 00 00 nop DWORD PTR [rax+0x0]
if (isnull[attnum])
24b0: 31 c0 xor eax,eax
24b2: 42 80 3c 11 00 cmp BYTE PTR [rcx+r10*1],0x0
24b7: 75 57 jne 2510 <tts_minimal_getsomeattrs+0x270>
attlen = cattr->attlen;
24b9: 42 0f b7 74 d2 22 movzx esi,WORD PTR [rdx+r10*8+0x22]
cattr->attalignby);
24bf: 42 0f b6 44 d2 25 movzx eax,BYTE PTR [rdx+r10*8+0x25]
if (attlen > 0)
24c5: 66 85 f6 test si,si
24c8: 0f 8e 52 01 00 00 jle 2620 <tts_minimal_getsomeattrs+0x380>
*off = TYPEALIGN(attalignby, *off);
24ce: 8d 5c 18 ff lea ebx,[rax+rbx*1-0x1]
24d2: f7 d8 neg eax
24d4: 21 d8 and eax,ebx
*off += attlen;
24d6: 0f bf de movsx ebx,si
offset_ptr = tupptr + *off;
24d9: 41 89 c5 mov r13d,eax
*off += attlen;
24dc: 01 c3 add ebx,eax
offset_ptr = tupptr + *off;
24de: 4d 01 c5 add r13,r8
24e1: 4c 89 e8 mov rax,r13
if (attbyval)
24e4: 42 80 7c d2 24 00 cmp BYTE PTR [rdx+r10*8+0x24],0x0
24ea: 74 24 je 2510 <tts_minimal_getsomeattrs+0x270>
switch (attlen)
24ec: 66 83 fe 02 cmp si,0x2
24f0: 0f 84 d2 02 00 00 je 27c8 <tts_minimal_getsomeattrs+0x528>
24f6: 66 83 fe 04 cmp si,0x4
24fa: 0f 84 a8 02 00 00 je 27a8 <tts_minimal_getsomeattrs+0x508>
2500: 66 83 fe 01 cmp si,0x1
2504: 0f 84 f6 01 00 00 je 2700 <tts_minimal_getsomeattrs+0x460>
return (Datum) X;
250a: 49 8b 45 00 mov rax,QWORD PTR [r13+0x0]
250e: 66 90 xchg ax,ax
values[attnum] = (Datum) 0;
2510: 4a 89 04 d7 mov QWORD PTR [rdi+r10*8],rax
for (; attnum < natts; attnum++)
2514: 49 83 c2 01 add r10,0x1
2518: 4d 39 ca cmp r10,r9
251b: 75 93 jne 24b0 <tts_minimal_getsomeattrs+0x210>
if (unlikely(attnum < reqnatts))
251d: 49 39 e9 cmp r9,rbp
2520: 0f 82 fa 03 00 00 jb 2920 <tts_minimal_getsomeattrs+0x680>
*offp = off;
2526: 41 89 5b 68 mov DWORD PTR [r11+0x68],ebx
}
252a: 5b pop rbx
252b: 5d pop rbp
252c: 41 5c pop r12
252e: 41 5d pop r13
2530: 41 5e pop r14
2532: 41 5f pop r15
2534: c3 ret
2535: 0f 1f 00 nop DWORD PTR [rax]
switch (attlen)
2538: 66 83 fb 01 cmp bx,0x1
253c: 0f 84 2e 01 00 00 je 2670 <tts_minimal_getsomeattrs+0x3d0>
2542: 48 8b 1e mov rbx,QWORD PTR [rsi]
2545: 0f 1f 40 00 nop DWORD PTR [rax+0x0]
2549: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
2550: 00 00 00 00
2554: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
255b: 00 00 00 00
255f: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
2566: 00 00 00 00
256a: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
2571: 00 00 00 00
2575: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
257c: 00 00 00 00
} while (++attnum < firstNonCacheOffsetAttr);
2580: 48 8d 70 01 lea rsi,[rax+0x1]
values[attnum] = fetch_att_noerr(tp + off,
2584: 48 89 1c c7 mov QWORD PTR [rdi+rax*8],rbx
} while (++attnum < firstNonCacheOffsetAttr);
2588: 49 39 f2 cmp r10,rsi
258b: 74 53 je 25e0 <tts_minimal_getsomeattrs+0x340>
258d: 48 89 f0 mov rax,rsi
isnull[attnum] = false;
2590: c6 04 01 00 mov BYTE PTR [rcx+rax*1],0x0
off = cattr->attcacheoff;
2594: 0f bf 74 c2 20 movsx esi,WORD PTR [rdx+rax*8+0x20]
2599: 49 89 f5 mov r13,rsi
values[attnum] = fetch_att_noerr(tp + off,
259c: 4c 01 c6 add rsi,r8
return (Datum) (uintptr_t) X;
259f: 48 89 f3 mov rbx,rsi
if (attbyval)
25a2: 80 7c c2 24 00 cmp BYTE PTR [rdx+rax*8+0x24],0x0
25a7: 74 d7 je 2580 <tts_minimal_getsomeattrs+0x2e0>
cattr->attlen);
25a9: 0f b7 5c c2 22 movzx ebx,WORD PTR [rdx+rax*8+0x22]
switch (attlen)
25ae: 66 83 fb 02 cmp bx,0x2
25b2: 0f 84 c8 00 00 00 je 2680 <tts_minimal_getsomeattrs+0x3e0>
25b8: 66 83 fb 04 cmp bx,0x4
25bc: 0f 85 76 ff ff ff jne 2538 <tts_minimal_getsomeattrs+0x298>
return (Datum) X;
25c2: 48 63 1e movsxd rbx,DWORD PTR [rsi]
} while (++attnum < firstNonCacheOffsetAttr);
25c5: 48 8d 70 01 lea rsi,[rax+0x1]
values[attnum] = fetch_att_noerr(tp + off,
25c9: 48 89 1c c7 mov QWORD PTR [rdi+rax*8],rbx
} while (++attnum < firstNonCacheOffsetAttr);
25cd: 49 39 f2 cmp r10,rsi
25d0: 75 bb jne 258d <tts_minimal_getsomeattrs+0x2ed>
25d2: 0f 1f 00 nop DWORD PTR [rax]
25d5: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
25dc: 00 00 00 00
off += cattr->attlen;
25e0: 0f bf 5c c2 22 movsx ebx,WORD PTR [rdx+rax*8+0x22]
} while (++attnum < firstNonCacheOffsetAttr);
25e5: 4c 89 d0 mov rax,r10
off += cattr->attlen;
25e8: 44 01 eb add ebx,r13d
25eb: e9 d7 fd ff ff jmp 23c7 <tts_minimal_getsomeattrs+0x127>
return (Datum) X;
25f0: 49 8b 36 mov rsi,QWORD PTR [r14]
return Int64GetDatum(*((const int64 *) offset_ptr));
25f3: e9 28 fe ff ff jmp 2420 <tts_minimal_getsomeattrs+0x180>
25f8: 0f 1f 84 00 00 00 00 nop DWORD PTR [rax+rax*1+0x0]
25ff: 00
return (Datum) X;
2600: 49 63 36 movsxd rsi,DWORD PTR [r14]
return Int32GetDatum(*((const int32 *) offset_ptr));
2603: e9 18 fe ff ff jmp 2420 <tts_minimal_getsomeattrs+0x180>
2608: 0f 1f 84 00 00 00 00 nop DWORD PTR [rax+rax*1+0x0]
260f: 00
return (Datum) X;
2610: 49 0f bf 36 movsx rsi,WORD PTR [r14]
return Int16GetDatum(*((const int16 *) offset_ptr));
2614: e9 07 fe ff ff jmp 2420 <tts_minimal_getsomeattrs+0x180>
2619: 0f 1f 80 00 00 00 00 nop DWORD PTR [rax+0x0]
if (!VARATT_IS_SHORT(tupptr + *off))
2620: 89 de mov esi,ebx
2622: 4c 01 c6 add rsi,r8
2625: f6 06 01 test BYTE PTR [rsi],0x1
2628: 0f 84 02 01 00 00 je 2730 <tts_minimal_getsomeattrs+0x490>
if (VARATT_IS_1B_E(PTR))
262e: 44 0f b6 2e movzx r13d,BYTE PTR [rsi]
return (Datum) (uintptr_t) X;
2632: 48 89 f0 mov rax,rsi
2635: 41 80 fd 01 cmp r13b,0x1
2639: 0f 84 0f 01 00 00 je 274e <tts_minimal_getsomeattrs+0x4ae>
else if (VARATT_IS_1B(PTR))
263f: 41 f6 c5 01 test r13b,0x1
2643: 0f 84 3f 01 00 00 je 2788 <tts_minimal_getsomeattrs+0x4e8>
return VARSIZE_1B(PTR);
2649: 41 d0 ed shr r13b,1
*off += VARSIZE_ANY(DatumGetPointer(res));
264c: 45 0f b6 ed movzx r13d,r13b
values[attnum] = (Datum) 0;
2650: 4a 89 04 d7 mov QWORD PTR [rdi+r10*8],rax
for (; attnum < natts; attnum++)
2654: 49 83 c2 01 add r10,0x1
2658: 44 01 eb add ebx,r13d
265b: 4d 39 ca cmp r10,r9
265e: 0f 85 4c fe ff ff jne 24b0 <tts_minimal_getsomeattrs+0x210>
2664: e9 b4 fe ff ff jmp 251d <tts_minimal_getsomeattrs+0x27d>
2669: 0f 1f 80 00 00 00 00 nop DWORD PTR [rax+0x0]
return (Datum) X;
2670: 48 0f be 1e movsx rbx,BYTE PTR [rsi]
return CharGetDatum(*((const char *) T));
2674: e9 07 ff ff ff jmp 2580 <tts_minimal_getsomeattrs+0x2e0>
2679: 0f 1f 80 00 00 00 00 nop DWORD PTR [rax+0x0]
return (Datum) X;
2680: 48 0f bf 1e movsx rbx,WORD PTR [rsi]
return Int16GetDatum(*((const int16 *) T));
2684: e9 f7 fe ff ff jmp 2580 <tts_minimal_getsomeattrs+0x2e0>
2689: 0f 1f 80 00 00 00 00 nop DWORD PTR [rax+0x0]
off += cattr->attlen;
2690: 41 0f bf 75 02 movsx esi,WORD PTR [r13+0x2]
if (attnum == reqnatts)
2695: 49 63 ec movsxd rbp,r12d
off += cattr->attlen;
2698: 48 8b 54 24 f0 mov rdx,QWORD PTR [rsp-0x10]
269d: 01 f3 add ebx,esi
if (attnum == reqnatts)
269f: 48 39 e8 cmp rax,rbp
26a2: 0f 85 0f fd ff ff jne 23b7 <tts_minimal_getsomeattrs+0x117>
*offp = off;
26a8: 41 89 5b 68 mov DWORD PTR [r11+0x68],ebx
}
26ac: 5b pop rbx
26ad: 5d pop rbp
26ae: 41 5c pop r12
26b0: 41 5d pop r13
26b2: 41 5e pop r14
26b4: 41 5f pop r15
26b6: c3 ret
26b7: 66 0f 1f 84 00 00 00 nop WORD PTR [rax+rax*1+0x0]
26be: 00 00
tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits));
26c0: 4c 8d 43 18 lea r8,[rbx+0x18]
if (reqnatts > firstNonGuaranteedAttr)
26c4: 41 39 c4 cmp r12d,eax
26c7: 0f 8e cb 00 00 00 jle 2798 <tts_minimal_getsomeattrs+0x4f8>
natts = Min(HeapTupleHeaderGetNatts(tup), reqnatts);
26cd: 0f b7 43 12 movzx eax,WORD PTR [rbx+0x12]
26d1: 25 ff 07 00 00 and eax,0x7ff
26d6: 44 39 e0 cmp eax,r12d
26d9: 41 0f 4f c4 cmovg eax,r12d
26dd: 41 89 c6 mov r14d,eax
26e0: 4c 63 c8 movsxd r9,eax
26e3: e9 36 fc ff ff jmp 231e <tts_minimal_getsomeattrs+0x7e>
26e8: 0f 1f 84 00 00 00 00 nop DWORD PTR [rax+rax*1+0x0]
26ef: 00
26f0: 31 ed xor ebp,ebp
firstNonGuaranteedAttr = 0;
26f2: 31 c0 xor eax,eax
26f4: e9 e2 fb ff ff jmp 22db <tts_minimal_getsomeattrs+0x3b>
26f9: 0f 1f 80 00 00 00 00 nop DWORD PTR [rax+0x0]
return (Datum) X;
2700: 49 0f be 45 00 movsx rax,BYTE PTR [r13+0x0]
values[attnum] = (Datum) 0;
2705: 4a 89 04 d7 mov QWORD PTR [rdi+r10*8],rax
for (; attnum < natts; attnum++)
2709: 49 83 c2 01 add r10,0x1
270d: 4d 39 ca cmp r10,r9
2710: 0f 85 9a fd ff ff jne 24b0 <tts_minimal_getsomeattrs+0x210>
2716: e9 02 fe ff ff jmp 251d <tts_minimal_getsomeattrs+0x27d>
271b: 0f 1f 44 00 00 nop DWORD PTR [rax+rax*1+0x0]
2720: 41 d0 ee shr r14b,1
*off += VARSIZE_ANY(DatumGetPointer(res));
2723: 45 0f b6 f6 movzx r14d,r14b
2727: e9 55 fd ff ff jmp 2481 <tts_minimal_getsomeattrs+0x1e1>
272c: 0f 1f 40 00 nop DWORD PTR [rax+0x0]
*off = TYPEALIGN(attalignby, *off);
2730: 8d 5c 18 ff lea ebx,[rax+rbx*1-0x1]
2734: f7 d8 neg eax
2736: 21 c3 and ebx,eax
res = PointerGetDatum(tupptr + *off);
2738: 89 de mov esi,ebx
273a: 4c 01 c6 add rsi,r8
if (VARATT_IS_1B_E(PTR))
273d: 44 0f b6 2e movzx r13d,BYTE PTR [rsi]
return (Datum) (uintptr_t) X;
2741: 48 89 f0 mov rax,rsi
2744: 41 80 fd 01 cmp r13b,0x1
2748: 0f 85 f1 fe ff ff jne 263f <tts_minimal_getsomeattrs+0x39f>
return VARTAG_1B_E(PTR);
274e: 0f b6 76 01 movzx esi,BYTE PTR [rsi+0x1]
if (tag == VARTAG_INDIRECT)
2752: 83 fe 01 cmp esi,0x1
2755: 0f 84 02 02 00 00 je 295d <tts_minimal_getsomeattrs+0x6bd>
return ((tag & ~1) == VARTAG_EXPANDED_RO);
275b: 41 89 f5 mov r13d,esi
275e: 41 83 e5 fe and r13d,0xfffffffe
else if (VARTAG_IS_EXPANDED(tag))
2762: 41 83 fd 02 cmp r13d,0x2
2766: 0f 84 f1 01 00 00 je 295d <tts_minimal_getsomeattrs+0x6bd>
else if (tag == VARTAG_ONDISK)
276c: 83 fe 12 cmp esi,0x12
276f: 40 0f 94 c6 sete sil
2773: 40 0f b6 f6 movzx esi,sil
2777: 48 c1 e6 04 shl rsi,0x4
*off += VARSIZE_ANY(DatumGetPointer(res));
277b: 44 8d 6e 02 lea r13d,[rsi+0x2]
277f: e9 cc fe ff ff jmp 2650 <tts_minimal_getsomeattrs+0x3b0>
2784: 0f 1f 40 00 nop DWORD PTR [rax+0x0]
return VARSIZE_4B(PTR);
2788: 44 8b 2e mov r13d,DWORD PTR [rsi]
278b: 41 c1 ed 02 shr r13d,0x2
278f: e9 bc fe ff ff jmp 2650 <tts_minimal_getsomeattrs+0x3b0>
2794: 0f 1f 40 00 nop DWORD PTR [rax+0x0]
natts = reqnatts;
2798: 4d 63 cc movsxd r9,r12d
279b: 45 89 e6 mov r14d,r12d
279e: e9 7b fb ff ff jmp 231e <tts_minimal_getsomeattrs+0x7e>
27a3: 0f 1f 44 00 00 nop DWORD PTR [rax+rax*1+0x0]
return (Datum) X;
27a8: 49 63 45 00 movsxd rax,DWORD PTR [r13+0x0]
values[attnum] = (Datum) 0;
27ac: 4a 89 04 d7 mov QWORD PTR [rdi+r10*8],rax
for (; attnum < natts; attnum++)
27b0: 49 83 c2 01 add r10,0x1
27b4: 4d 39 ca cmp r10,r9
27b7: 0f 85 f3 fc ff ff jne 24b0 <tts_minimal_getsomeattrs+0x210>
27bd: e9 5b fd ff ff jmp 251d <tts_minimal_getsomeattrs+0x27d>
27c2: 66 0f 1f 44 00 00 nop WORD PTR [rax+rax*1+0x0]
return (Datum) X;
27c8: 49 0f bf 45 00 movsx rax,WORD PTR [r13+0x0]
values[attnum] = (Datum) 0;
27cd: 4a 89 04 d7 mov QWORD PTR [rdi+r10*8],rax
for (; attnum < natts; attnum++)
27d1: 49 83 c2 01 add r10,0x1
27d5: 4d 39 ca cmp r10,r9
27d8: 0f 85 d2 fc ff ff jne 24b0 <tts_minimal_getsomeattrs+0x210>
27de: e9 3a fd ff ff jmp 251d <tts_minimal_getsomeattrs+0x27d>
27e3: 0f 1f 44 00 00 nop DWORD PTR [rax+rax*1+0x0]
return VARTAG_1B_E(PTR);
27e8: 45 0f b6 6d 01 movzx r13d,BYTE PTR [r13+0x1]
if (tag == VARTAG_INDIRECT)
27ed: 41 83 fd 01 cmp r13d,0x1
27f1: 0f 84 5b 01 00 00 je 2952 <tts_minimal_getsomeattrs+0x6b2>
return ((tag & ~1) == VARTAG_EXPANDED_RO);
27f7: 45 89 ee mov r14d,r13d
27fa: 41 83 e6 fe and r14d,0xfffffffe
else if (VARTAG_IS_EXPANDED(tag))
27fe: 41 83 fe 02 cmp r14d,0x2
2802: 0f 84 4a 01 00 00 je 2952 <tts_minimal_getsomeattrs+0x6b2>
else if (tag == VARTAG_ONDISK)
2808: 41 83 fd 12 cmp r13d,0x12
280c: 41 0f 94 c5 sete r13b
2810: 45 0f b6 ed movzx r13d,r13b
2814: 49 c1 e5 04 shl r13,0x4
2818: 45 8d 75 02 lea r14d,[r13+0x2]
281c: e9 60 fc ff ff jmp 2481 <tts_minimal_getsomeattrs+0x1e1>
2821: 0f 1f 80 00 00 00 00 nop DWORD PTR [rax+0x0]
int nattByte = natts >> 3;
2828: 45 89 cd mov r13d,r9d
282b: 41 c1 fd 03 sar r13d,0x3
for (bytenum = 0; bytenum < nattByte; bytenum++)
282f: 45 85 ed test r13d,r13d
2832: 0f 8e 30 01 00 00 jle 2968 <tts_minimal_getsomeattrs+0x6c8>
2838: 48 8d 73 17 lea rsi,[rbx+0x17]
283c: 31 ff xor edi,edi
283e: eb 10 jmp 2850 <tts_minimal_getsomeattrs+0x5b0>
2840: 83 c7 01 add edi,0x1
2843: 48 83 c6 01 add rsi,0x1
2847: 41 39 fd cmp r13d,edi
284a: 0f 84 ec 00 00 00 je 293c <tts_minimal_getsomeattrs+0x69c>
if (bits[bytenum] != 0xFF)
2850: 0f b6 06 movzx eax,BYTE PTR [rsi]
2853: 3c ff cmp al,0xff
2855: 74 e9 je 2840 <tts_minimal_getsomeattrs+0x5a0>
res = bytenum << 3;
2857: c1 e7 03 shl edi,0x3
res += pg_rightmost_one_pos32(~((uint32) bits[bytenum]));
285a: f7 d0 not eax
int nbytes = (natts + 7) >> 3;
285c: 45 8d 69 07 lea r13d,[r9+0x7]
2860: f3 0f bc c0 tzcnt eax,eax
res += pg_rightmost_one_pos32(~((uint32) bits[bytenum]));
2864: 01 f8 add eax,edi
res = Min(res, natts);
2866: 41 39 c1 cmp r9d,eax
2869: 41 0f 4e c1 cmovle eax,r9d
int nbytes = (natts + 7) >> 3;
286d: 41 c1 fd 03 sar r13d,0x3
res = Min(res, natts);
2871: 4c 63 f0 movsxd r14,eax
isnull_8 &= UINT64CONST(0x0101010101010101);
2874: 49 bf 01 01 01 01 01 movabs r15,0x101010101010101
287b: 01 01 01
287e: 4d 63 ed movsxd r13,r13d
for (bytenum = 0; bytenum < nattByte; bytenum++)
2881: 31 ff xor edi,edi
2883: 66 0f 1f 44 00 00 nop WORD PTR [rax+rax*1+0x0]
2889: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
2890: 00 00 00 00
2894: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
289b: 00 00 00 00
289f: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
28a6: 00 00 00 00
28aa: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
28b1: 00 00 00 00
28b5: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
28bc: 00 00 00 00
bits8 nullbyte = ~bits[i];
28c0: 0f b6 74 3b 17 movzx esi,BYTE PTR [rbx+rdi*1+0x17]
28c5: f7 d6 not esi
isnull_8 |= ((uint64) ((nullbyte >> 4) * SPREAD_BITS_MULTIPLIER_32)) << 32;
28c7: 89 f0 mov eax,esi
isnull_8 = (nullbyte & 0xf) * SPREAD_BITS_MULTIPLIER_32;
28c9: 83 e6 0f and esi,0xf
isnull_8 |= ((uint64) ((nullbyte >> 4) * SPREAD_BITS_MULTIPLIER_32)) << 32;
28cc: c0 e8 04 shr al,0x4
isnull_8 = (nullbyte & 0xf) * SPREAD_BITS_MULTIPLIER_32;
28cf: 48 69 f6 81 40 20 00 imul rsi,rsi,0x204081
isnull_8 |= ((uint64) ((nullbyte >> 4) * SPREAD_BITS_MULTIPLIER_32)) << 32;
28d6: 83 e0 0f and eax,0xf
28d9: 48 69 c0 81 40 20 00 imul rax,rax,0x204081
28e0: 48 c1 e0 20 shl rax,0x20
28e4: 48 09 f0 or rax,rsi
isnull_8 &= UINT64CONST(0x0101010101010101);
28e7: 4c 21 f8 and rax,r15
28ea: 48 89 04 f9 mov QWORD PTR [rcx+rdi*8],rax
for (int i = 0; i < nbytes; i++, isnull += 8)
28ee: 48 83 c7 01 add rdi,0x1
28f2: 4c 39 ef cmp rdi,r13
28f5: 75 c9 jne 28c0 <tts_minimal_getsomeattrs+0x620>
firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, firstNullAttr);
28f7: 45 39 f2 cmp r10d,r14d
28fa: 4d 0f 4f d6 cmovg r10,r14
28fe: e9 1b fa ff ff jmp 231e <tts_minimal_getsomeattrs+0x7e>
2903: 0f 1f 44 00 00 nop DWORD PTR [rax+rax*1+0x0]
for (; attnum < firstNullAttr; attnum++)
2908: 49 89 c2 mov r10,rax
290b: e9 90 fb ff ff jmp 24a0 <tts_minimal_getsomeattrs+0x200>
for (; attnum < natts; attnum++)
2910: 4d 89 d1 mov r9,r10
2913: e9 05 fc ff ff jmp 251d <tts_minimal_getsomeattrs+0x27d>
2918: 0f 1f 84 00 00 00 00 nop DWORD PTR [rax+rax*1+0x0]
291f: 00
*offp = off;
2920: 41 89 5b 68 mov DWORD PTR [r11+0x68],ebx
slot_getmissingattrs(slot, attnum, reqnatts);
2924: 44 89 e2 mov edx,r12d
}
2927: 5b pop rbx
slot_getmissingattrs(slot, attnum, reqnatts);
2928: 44 89 ce mov esi,r9d
}
292b: 5d pop rbp
slot_getmissingattrs(slot, attnum, reqnatts);
292c: 4c 89 df mov rdi,r11
}
292f: 41 5c pop r12
2931: 41 5d pop r13
2933: 41 5e pop r14
2935: 41 5f pop r15
slot_getmissingattrs(slot, attnum, reqnatts);
2937: e9 b4 f1 ff ff jmp 1af0 <slot_getmissingattrs>
res = bytenum << 3;
293c: 42 8d 3c ed 00 00 00 lea edi,[r13*8+0x0]
2943: 00
res += pg_rightmost_one_pos32(~((uint32) bits[bytenum]));
2944: 4d 63 ed movsxd r13,r13d
2947: 42 0f b6 44 2b 17 movzx eax,BYTE PTR [rbx+r13*1+0x17]
294d: e9 08 ff ff ff jmp 285a <tts_minimal_getsomeattrs+0x5ba>
2952: 41 be 0a 00 00 00 mov r14d,0xa
2958: e9 24 fb ff ff jmp 2481 <tts_minimal_getsomeattrs+0x1e1>
295d: 41 bd 0a 00 00 00 mov r13d,0xa
2963: e9 e8 fc ff ff jmp 2650 <tts_minimal_getsomeattrs+0x3b0>
2968: 0f b6 43 17 movzx eax,BYTE PTR [rbx+0x17]
int nbytes = (natts + 7) >> 3;
296c: 45 8d 69 07 lea r13d,[r9+0x7]
res += pg_rightmost_one_pos32(~((uint32) bits[bytenum]));
2970: f7 d0 not eax
2972: f3 0f bc c0 tzcnt eax,eax
res = Min(res, natts);
2976: 41 39 c1 cmp r9d,eax
2979: 41 0f 4e c1 cmovle eax,r9d
int nbytes = (natts + 7) >> 3;
297d: 41 c1 fd 03 sar r13d,0x3
res = Min(res, natts);
2981: 4c 63 f0 movsxd r14,eax
for (int i = 0; i < nbytes; i++, isnull += 8)
2984: 41 83 fd 01 cmp r13d,0x1
2988: 0f 85 69 ff ff ff jne 28f7 <tts_minimal_getsomeattrs+0x657>
298e: e9 e1 fe ff ff jmp 2874 <tts_minimal_getsomeattrs+0x5d4>
2993: 66 90 xchg ax,ax
2995: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
299c: 00 00 00 00
From 46c83290a6ed1256cbefd9fa62de808424601d70 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Tue, 27 Jan 2026 15:08:09 +1300
Subject: [PATCH v11 1/5] Introduce deform_bench test module
For benchmarking tuple deformation.
---
src/test/modules/deform_bench/.gitignore | 4 +
src/test/modules/deform_bench/Makefile | 21 ++++
.../deform_bench/deform_bench--1.0.sql | 8 ++
src/test/modules/deform_bench/deform_bench.c | 107 ++++++++++++++++++
.../modules/deform_bench/deform_bench.control | 4 +
src/test/modules/deform_bench/meson.build | 22 ++++
src/test/modules/meson.build | 1 +
7 files changed, 167 insertions(+)
create mode 100644 src/test/modules/deform_bench/.gitignore
create mode 100644 src/test/modules/deform_bench/Makefile
create mode 100644 src/test/modules/deform_bench/deform_bench--1.0.sql
create mode 100644 src/test/modules/deform_bench/deform_bench.c
create mode 100644 src/test/modules/deform_bench/deform_bench.control
create mode 100644 src/test/modules/deform_bench/meson.build
diff --git a/src/test/modules/deform_bench/.gitignore b/src/test/modules/deform_bench/.gitignore
new file mode 100644
index 00000000000..5dcb3ff9723
--- /dev/null
+++ b/src/test/modules/deform_bench/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/src/test/modules/deform_bench/Makefile b/src/test/modules/deform_bench/Makefile
new file mode 100644
index 00000000000..b5fc0f7a583
--- /dev/null
+++ b/src/test/modules/deform_bench/Makefile
@@ -0,0 +1,21 @@
+# src/test/modules/deform_bench/Makefile
+
+MODULE_big = deform_bench
+OBJS = deform_bench.o
+
+EXTENSION = deform_bench
+DATA = deform_bench--1.0.sql
+PGFILEDESC = "deform_bench - tuple deform benchmarking"
+
+REGRESS = deform_bench
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = src/test/modules/deform_bench
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/src/test/modules/deform_bench/deform_bench--1.0.sql b/src/test/modules/deform_bench/deform_bench--1.0.sql
new file mode 100644
index 00000000000..492b71dba3b
--- /dev/null
+++ b/src/test/modules/deform_bench/deform_bench--1.0.sql
@@ -0,0 +1,8 @@
+/* deform_bench--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION deform_bench" to load this file. \quit
+
+CREATE FUNCTION deform_bench(tableoid Oid, attnum int[]) RETURNS FLOAT
+AS 'MODULE_PATHNAME', 'deform_bench'
+LANGUAGE C VOLATILE STRICT;
diff --git a/src/test/modules/deform_bench/deform_bench.c b/src/test/modules/deform_bench/deform_bench.c
new file mode 100644
index 00000000000..7838f639bef
--- /dev/null
+++ b/src/test/modules/deform_bench/deform_bench.c
@@ -0,0 +1,107 @@
+/*-------------------------------------------------------------------------
+ *
+ * deform_bench.c
+ *
+ * for benchmarking tuple deformation routines
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include <time.h>
+#include <sys/time.h>
+
+#include "access/heapam.h"
+#include "access/relscan.h"
+#include "catalog/pg_am_d.h"
+#include "catalog/pg_type_d.h"
+#include "funcapi.h"
+#include "miscadmin.h"
+#include "utils/array.h"
+#include "utils/arrayaccess.h"
+#include "utils/builtins.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(deform_bench);
+
+Datum
+deform_bench(PG_FUNCTION_ARGS)
+{
+ Oid tableoid = PG_GETARG_OID(0);
+ ArrayType *array = PG_GETARG_ARRAYTYPE_P(1);
+ TableScanDesc scan;
+ Relation rel;
+ TupleDesc tupdesc;
+ TupleTableSlot *slot;
+ Datum *elem_datums = NULL;
+ bool *elem_nulls = NULL;
+ int elem_count;
+ int *attnums;
+ clock_t start,
+ end;
+
+ rel = relation_open(tableoid, AccessShareLock);
+
+ if (rel->rd_rel->relam != HEAP_TABLE_AM_OID)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("only heap AM is supported")));
+
+ tupdesc = RelationGetDescr(rel);
+ slot = MakeTupleTableSlot(tupdesc, &TTSOpsBufferHeapTuple);
+ scan = table_beginscan_strat(rel, GetActiveSnapshot(), 0, NULL, true, false);
+
+ /*
+ * The array is used to allow callers to define how many atts to deform.
+ * e.g: '{1,10}'::int[] would deform attnum=1, then in a 2nd pass deform
+ * the remainder up to attnum=10. Passing an element as NULL means all
+ * attnums. This allows simulation of incremental deformation. Generally
+ * if you're passing an array with more than 1 element, then the array
+ * should be in ascending order. Doing something like '{10,1}' would mean
+ * we've already deformed 10 attributes and on the 2nd pass there's
+ * nothing to do since attnum=1 was already deformed in the first pass.
+ *
+ * You'll get an ERROR if you pass a number higher than the number of
+ * attributes in the table.
+ */
+ deconstruct_array(array,
+ INT4OID,
+ sizeof(int32),
+ true,
+ 'i',
+ &elem_datums,
+ &elem_nulls,
+ &elem_count);
+
+ attnums = palloc_array(int, elem_count);
+
+ for (int i = 0; i < elem_count; i++)
+ {
+ /* Make a NULL element mean all attributes */
+ if (elem_nulls[i])
+ attnums[i] = tupdesc->natts;
+ else
+ attnums[i] = DatumGetInt32(elem_datums[i]);
+ }
+
+ start = clock();
+
+ while (heap_getnextslot(scan, ForwardScanDirection, slot))
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ /* Deform in stages according to the attnums array */
+ for (int i = 0; i < elem_count; i++)
+ slot_getsomeattrs(slot, attnums[i]);
+ }
+
+ end = clock();
+
+ ExecDropSingleTupleTableSlot(slot);
+ table_endscan(scan);
+ relation_close(rel, AccessShareLock);
+
+
+ /* Returns the number of milliseconds to run the test */
+ PG_RETURN_FLOAT8((double) (end - start) / (CLOCKS_PER_SEC / 1000));
+}
diff --git a/src/test/modules/deform_bench/deform_bench.control b/src/test/modules/deform_bench/deform_bench.control
new file mode 100644
index 00000000000..a2023f9d738
--- /dev/null
+++ b/src/test/modules/deform_bench/deform_bench.control
@@ -0,0 +1,4 @@
+# deform_bench extension
+comment = 'functions for benchmarking tuple deformation'
+default_version = '1.0'
+module_pathname = '$libdir/deform_bench'
diff --git a/src/test/modules/deform_bench/meson.build b/src/test/modules/deform_bench/meson.build
new file mode 100644
index 00000000000..82049585244
--- /dev/null
+++ b/src/test/modules/deform_bench/meson.build
@@ -0,0 +1,22 @@
+# Copyright (c) 2026, PostgreSQL Global Development Group
+
+deform_bench_sources = files(
+ 'deform_bench.c',
+)
+
+if host_system == 'windows'
+ deform_bench_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+ '--NAME', 'deform_bench',
+ '--FILEDESC', 'deform_bench - benchmarking tuple deformation',])
+endif
+
+deform_bench = shared_module('deform_bench',
+ deform_bench_sources,
+ kwargs: pg_test_mod_args,
+)
+test_install_libs += deform_bench
+
+test_install_data += files(
+ 'deform_bench--1.0.sql',
+ 'deform_bench.control',
+)
diff --git a/src/test/modules/meson.build b/src/test/modules/meson.build
index 2634a519935..ef2b0af4581 100644
--- a/src/test/modules/meson.build
+++ b/src/test/modules/meson.build
@@ -2,6 +2,7 @@
subdir('brin')
subdir('commit_ts')
+subdir('deform_bench')
subdir('delay_execution')
subdir('dummy_index_am')
subdir('dummy_seclabel')
--
2.51.0
From 5d372c316557406e319b26dcf381d896aecea226 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Mon, 16 Feb 2026 14:20:19 +1300
Subject: [PATCH v11 2/5] Allow sibling call optimization in
slot_getsomeattrs_int()
This changes the TupleTableSlotOps contract to make it so the
getsomeattrs() function is in charge of calling
slot_getmissingattrs().
Since this removes all code from slot_getsomeattrs_int() aside from the
getsomeattrs() call itself, we may as well adjust slot_getsomeattrs() so
that it calls getsomeattrs() directly. We leave slot_getsomeattrs_int()
intact as this is still called from the JIT code.
---
src/backend/executor/execTuples.c | 57 ++++++++++++++++---------------
src/include/executor/tuptable.h | 13 ++++---
2 files changed, 37 insertions(+), 33 deletions(-)
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index b768eae9e53..5b9bb21fa7b 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -73,7 +73,7 @@
static TupleDesc ExecTypeFromTLInternal(List *targetList,
bool skipjunk);
static pg_attribute_always_inline void slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
- int natts);
+ int reqnatts);
static inline void tts_buffer_heap_store_tuple(TupleTableSlot *slot,
HeapTuple tuple,
Buffer buffer,
@@ -1108,7 +1108,10 @@ slot_deform_heap_tuple_internal(TupleTableSlot *slot, HeapTuple tuple,
* slot_deform_heap_tuple
* Given a TupleTableSlot, extract data from the slot's physical tuple
* into its Datum/isnull arrays. Data is extracted up through the
- * natts'th column (caller must ensure this is a legal column number).
+ * reqnatts'th column. If there are insufficient attributes in the given
+ * tuple, then slot_getmissingattrs() is called to populate the
+ * remainder. If reqnatts is above the number of attributes in the
+ * slot's TupleDesc, an error is raised.
*
* This is essentially an incremental version of heap_deform_tuple:
* on each call we extract attributes up to the one needed, without
@@ -1120,7 +1123,7 @@ slot_deform_heap_tuple_internal(TupleTableSlot *slot, HeapTuple tuple,
*/
static pg_attribute_always_inline void
slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
- int natts)
+ int reqnatts)
{
bool hasnulls = HeapTupleHasNulls(tuple);
int attnum;
@@ -1128,13 +1131,14 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
bool slow; /* can we use/set attcacheoff? */
/* We can only fetch as many attributes as the tuple has. */
- natts = Min(HeapTupleHeaderGetNatts(tuple->t_data), natts);
+ natts = Min(HeapTupleHeaderGetNatts(tuple->t_data), reqnatts);
/*
* Check whether the first call for this tuple, and initialize or restore
* loop state.
*/
attnum = slot->tts_nvalid;
+ slot->tts_nvalid = reqnatts;
if (attnum == 0)
{
/* Start from the first attribute */
@@ -1199,12 +1203,15 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
/*
* Save state for next execution
*/
- slot->tts_nvalid = attnum;
*offp = off;
if (slow)
slot->tts_flags |= TTS_FLAG_SLOW;
else
slot->tts_flags &= ~TTS_FLAG_SLOW;
+
+ /* Fetch any missing attrs and raise an error if reqnatts is invalid. */
+ if (unlikely(attnum < reqnatts))
+ slot_getmissingattrs(slot, attnum, reqnatts);
}
const TupleTableSlotOps TTSOpsVirtual = {
@@ -2058,34 +2065,36 @@ slot_getmissingattrs(TupleTableSlot *slot, int startAttNum, int lastAttNum)
{
AttrMissing *attrmiss = NULL;
+ /* Check for invalid attnums */
+ if (unlikely(lastAttNum > slot->tts_tupleDescriptor->natts))
+ elog(ERROR, "invalid attribute number %d", lastAttNum);
+
if (slot->tts_tupleDescriptor->constr)
attrmiss = slot->tts_tupleDescriptor->constr->missing;
if (!attrmiss)
{
/* no missing values array at all, so just fill everything in as NULL */
- memset(slot->tts_values + startAttNum, 0,
- (lastAttNum - startAttNum) * sizeof(Datum));
- memset(slot->tts_isnull + startAttNum, 1,
- (lastAttNum - startAttNum) * sizeof(bool));
+ for (int attnum = startAttNum; attnum < lastAttNum; attnum++)
+ {
+ slot->tts_values[attnum] = (Datum) 0;
+ slot->tts_isnull[attnum] = true;
+ }
}
else
{
- int missattnum;
-
- /* if there is a missing values array we must process them one by one */
- for (missattnum = startAttNum;
- missattnum < lastAttNum;
- missattnum++)
+ /* use attrmiss to set the missing values */
+ for (int attnum = startAttNum; attnum < lastAttNum; attnum++)
{
- slot->tts_values[missattnum] = attrmiss[missattnum].am_value;
- slot->tts_isnull[missattnum] = !attrmiss[missattnum].am_present;
+ slot->tts_values[attnum] = attrmiss[attnum].am_value;
+ slot->tts_isnull[attnum] = !attrmiss[attnum].am_present;
}
}
}
/*
- * slot_getsomeattrs_int - workhorse for slot_getsomeattrs()
+ * slot_getsomeattrs_int
+ * external function to call getsomeattrs() for use in JIT
*/
void
slot_getsomeattrs_int(TupleTableSlot *slot, int attnum)
@@ -2094,21 +2103,13 @@ slot_getsomeattrs_int(TupleTableSlot *slot, int attnum)
Assert(slot->tts_nvalid < attnum); /* checked in slot_getsomeattrs */
Assert(attnum > 0);
- if (unlikely(attnum > slot->tts_tupleDescriptor->natts))
- elog(ERROR, "invalid attribute number %d", attnum);
-
/* Fetch as many attributes as possible from the underlying tuple. */
slot->tts_ops->getsomeattrs(slot, attnum);
/*
- * If the underlying tuple doesn't have enough attributes, tuple
- * descriptor must have the missing attributes.
+ * Avoid putting new code here as that would prevent the compiler from
+ * using the sibling call optimization for the above function.
*/
- if (unlikely(slot->tts_nvalid < attnum))
- {
- slot_getmissingattrs(slot, slot->tts_nvalid, attnum);
- slot->tts_nvalid = attnum;
- }
}
/* ----------------------------------------------------------------
diff --git a/src/include/executor/tuptable.h b/src/include/executor/tuptable.h
index a2dfd707e78..3b09abbf99f 100644
--- a/src/include/executor/tuptable.h
+++ b/src/include/executor/tuptable.h
@@ -151,10 +151,12 @@ struct TupleTableSlotOps
/*
* Fill up first natts entries of tts_values and tts_isnull arrays with
- * values from the tuple contained in the slot. The function may be called
- * with natts more than the number of attributes available in the tuple,
- * in which case it should set tts_nvalid to the number of returned
- * columns.
+ * values from the tuple contained in the slot and set the slot's
+ * tts_nvalid to natts. The function may be called with an natts value
+ * more than the number of attributes available in the tuple, in which
+ * case the function must call slot_getmissingattrs() to populate the
+ * remaining attributes. The function must raise an ERROR if 'natts' is
+ * higher than the number of attributes in the slot's TupleDesc.
*/
void (*getsomeattrs) (TupleTableSlot *slot, int natts);
@@ -357,8 +359,9 @@ extern void slot_getsomeattrs_int(TupleTableSlot *slot, int attnum);
static inline void
slot_getsomeattrs(TupleTableSlot *slot, int attnum)
{
+ /* Populate slot with attributes up to 'attnum', if it's not already */
if (slot->tts_nvalid < attnum)
- slot_getsomeattrs_int(slot, attnum);
+ slot->tts_ops->getsomeattrs(slot, attnum);
}
/*
--
2.51.0
From 099a6186e1886432ed24653178ab1ce9113900c9 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Mon, 23 Feb 2026 09:39:37 +1300
Subject: [PATCH v11 5/5] Reduce size of CompactAttribute struct to 8 bytes
Previously, this was 16 bytes. With the use of some bitflags and by
reducing the attcacheoff field size to a 16-bit type, we can halve the
size of the struct.
It's unlikely that caching the offsets for offsets larger than what will
fit in a 16-bit int will help much as the tuple is very likely to have
some non-fixed-width types anyway, the offsets of which we cannot cache.
---
src/backend/access/common/tupdesc.c | 10 ++++++++++
src/backend/executor/execTuples.c | 16 ++++++++++++----
src/include/access/tupdesc.h | 16 ++++++++--------
3 files changed, 30 insertions(+), 12 deletions(-)
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index c68561337d7..71461ba6096 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -530,6 +530,16 @@ TupleDescFinalize(TupleDesc tupdesc)
off = att_nominal_alignby(off, cattr->attalignby);
+ /*
+ * attcacheoff is an int16, so don't try to cache any offsets larger
+ * than will fit in that type. Any attributes which are offset more
+ * than 2^15 are likely due to variable-length attributes. Since we
+ * don't cache offsets for or beyond variable-length attributes, using
+ * an int16 rather than an int32 here is unlikely to cost us anything.
+ */
+ if (off > PG_INT16_MAX)
+ break;
+
cattr->attcacheoff = off;
off += cattr->attlen;
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 83a8c02894d..345b22ca932 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -1013,6 +1013,7 @@ static pg_attribute_always_inline void
slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int reqnatts)
{
+ CompactAttribute *cattrs;
CompactAttribute *cattr;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
HeapTupleHeader tup = tuple->t_data;
@@ -1101,6 +1102,13 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
values = slot->tts_values;
slot->tts_nvalid = reqnatts;
+ /*
+ * We store the tupleDesc's CompactAttribute array in 'cattrs' as gcc
+ * seems to be unwilling to optimize accessing the CompactAttribute
+ * element efficiently when accessing it via TupleDescCompactAttr().
+ */
+ cattrs = tupleDesc->compact_attrs;
+
/* Ensure we calculated tp correctly */
Assert(tp == (char *) tup + tup->t_hoff);
@@ -1111,7 +1119,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int attlen;
isnull[attnum] = false;
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
attlen = cattr->attlen;
/* We don't expect any non-byval types */
@@ -1156,7 +1164,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
do
{
isnull[attnum] = false;
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
off = cattr->attcacheoff;
values[attnum] = fetch_att_noerr(tp + off,
@@ -1183,7 +1191,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int attlen;
isnull[attnum] = false;
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
attlen = cattr->attlen;
/*
@@ -1216,7 +1224,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
continue;
}
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
attlen = cattr->attlen;
/* As above, we don't expect cstrings */
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index ad7bc013812..e98036b58bf 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -55,7 +55,7 @@ typedef struct TupleConstr
* directly after the FormData_pg_attribute struct is populated or
* altered in any way.
*
- * Currently, this struct is 16 bytes. Any code changes which enlarge this
+ * Currently, this struct is 8 bytes. Any code changes which enlarge this
* struct should be considered very carefully.
*
* Code which must access a TupleDesc's attribute data should always make use
@@ -67,17 +67,17 @@ typedef struct TupleConstr
*/
typedef struct CompactAttribute
{
- int32 attcacheoff; /* fixed offset into tuple, if known, or -1 */
+ int16 attcacheoff; /* fixed offset into tuple, if known, or -1 */
int16 attlen; /* attr len in bytes or -1 = varlen, -2 =
* cstring */
bool attbyval; /* as FormData_pg_attribute.attbyval */
- bool attispackable; /* FormData_pg_attribute.attstorage !=
- * TYPSTORAGE_PLAIN */
- bool atthasmissing; /* as FormData_pg_attribute.atthasmissing */
- bool attisdropped; /* as FormData_pg_attribute.attisdropped */
- bool attgenerated; /* FormData_pg_attribute.attgenerated != '\0' */
- char attnullability; /* status of not-null constraint, see below */
uint8 attalignby; /* alignment requirement in bytes */
+ bool attispackable:1; /* FormData_pg_attribute.attstorage !=
+ * TYPSTORAGE_PLAIN */
+ bool atthasmissing:1; /* as FormData_pg_attribute.atthasmissing */
+ bool attisdropped:1; /* as FormData_pg_attribute.attisdropped */
+ bool attgenerated:1; /* FormData_pg_attribute.attgenerated != '\0' */
+ char attnullability; /* status of not-null constraint, see below */
} CompactAttribute;
/* Valid values for CompactAttribute->attnullability */
--
2.51.0
From 3fa14f2411303b5433dd2e3434c840a77395e213 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Wed, 21 Jan 2026 15:41:37 +1300
Subject: [PATCH v11 3/5] Add empty TupleDescFinalize() function
Currently does nothing, but will in a future commit.
---
contrib/dblink/dblink.c | 4 ++++
contrib/pg_buffercache/pg_buffercache_pages.c | 2 ++
contrib/pg_visibility/pg_visibility.c | 2 ++
src/backend/access/brin/brin_tuple.c | 1 +
src/backend/access/common/tupdesc.c | 13 +++++++++++++
src/backend/access/gin/ginutil.c | 1 +
src/backend/access/gist/gistscan.c | 1 +
src/backend/access/spgist/spgutils.c | 1 +
src/backend/access/transam/twophase.c | 1 +
src/backend/access/transam/xlogfuncs.c | 1 +
src/backend/backup/basebackup_copy.c | 3 +++
src/backend/catalog/index.c | 2 ++
src/backend/catalog/pg_publication.c | 1 +
src/backend/catalog/toasting.c | 6 ++++++
src/backend/commands/explain.c | 1 +
src/backend/commands/functioncmds.c | 1 +
src/backend/commands/sequence.c | 1 +
src/backend/commands/tablecmds.c | 4 ++++
src/backend/commands/wait.c | 1 +
src/backend/executor/execSRF.c | 2 ++
src/backend/executor/execTuples.c | 4 ++++
src/backend/executor/nodeFunctionscan.c | 2 ++
src/backend/parser/parse_relation.c | 4 +++-
src/backend/parser/parse_target.c | 2 ++
.../replication/libpqwalreceiver/libpqwalreceiver.c | 1 +
src/backend/replication/walsender.c | 5 +++++
src/backend/utils/adt/acl.c | 1 +
src/backend/utils/adt/genfile.c | 1 +
src/backend/utils/adt/lockfuncs.c | 1 +
src/backend/utils/adt/orderedsetaggs.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 5 +++++
src/backend/utils/adt/tsvector_op.c | 1 +
src/backend/utils/cache/relcache.c | 8 ++++++++
src/backend/utils/fmgr/funcapi.c | 6 ++++++
src/backend/utils/misc/guc_funcs.c | 5 +++++
src/include/access/tupdesc.h | 1 +
src/pl/plpgsql/src/pl_comp.c | 2 ++
.../test_custom_stats/test_custom_fixed_stats.c | 1 +
src/test/modules/test_predtest/test_predtest.c | 1 +
39 files changed, 100 insertions(+), 1 deletion(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 2498d80c8e7..4038950a6ef 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -881,6 +881,7 @@ materializeResult(FunctionCallInfo fcinfo, PGconn *conn, PGresult *res)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "status",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
ntuples = 1;
nfields = 1;
}
@@ -1044,6 +1045,7 @@ materializeQueryResult(FunctionCallInfo fcinfo,
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "status",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
attinmeta = TupleDescGetAttInMetadata(tupdesc);
oldcontext = MemoryContextSwitchTo(rsinfo->econtext->ecxt_per_query_memory);
@@ -1529,6 +1531,8 @@ dblink_get_pkey(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 2, "colname",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+
/*
* Generate attribute metadata needed later to produce tuples from raw
* C strings
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 89b86855243..a6b4fb5252b 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -174,6 +174,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
INT4OID, -1, 0);
+ TupleDescFinalize(tupledesc);
fctx->tupdesc = BlessTupleDesc(tupledesc);
/* Allocate NBuffers worth of BufferCachePagesRec records. */
@@ -442,6 +443,7 @@ pg_buffercache_os_pages_internal(FunctionCallInfo fcinfo, bool include_numa)
TupleDescInitEntry(tupledesc, (AttrNumber) 3, "numa_node",
INT4OID, -1, 0);
+ TupleDescFinalize(tupledesc);
fctx->tupdesc = BlessTupleDesc(tupledesc);
fctx->include_numa = include_numa;
diff --git a/contrib/pg_visibility/pg_visibility.c b/contrib/pg_visibility/pg_visibility.c
index 9bc3a784bf7..dfab0b64cf5 100644
--- a/contrib/pg_visibility/pg_visibility.c
+++ b/contrib/pg_visibility/pg_visibility.c
@@ -469,6 +469,8 @@ pg_visibility_tupdesc(bool include_blkno, bool include_pd)
TupleDescInitEntry(tupdesc, ++a, "pd_all_visible", BOOLOID, -1, 0);
Assert(a == maxattr);
+ TupleDescFinalize(tupdesc);
+
return BlessTupleDesc(tupdesc);
}
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 69c233c62eb..742ac089a28 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -84,6 +84,7 @@ brtuple_disk_tupdesc(BrinDesc *brdesc)
MemoryContextSwitchTo(oldcxt);
+ TupleDescFinalize(tupdesc);
brdesc->bd_disktdesc = tupdesc;
}
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index b69d10f0a45..2137385a833 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -221,6 +221,9 @@ CreateTupleDesc(int natts, Form_pg_attribute *attrs)
memcpy(TupleDescAttr(desc, i), attrs[i], ATTRIBUTE_FIXED_PART_SIZE);
populate_compact_attribute(desc, i);
}
+
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -265,6 +268,8 @@ CreateTupleDescCopy(TupleDesc tupdesc)
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -311,6 +316,8 @@ CreateTupleDescTruncatedCopy(TupleDesc tupdesc, int natts)
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -396,6 +403,8 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -438,6 +447,8 @@ TupleDescCopy(TupleDesc dst, TupleDesc src)
* source's refcount would be wrong in any case.)
*/
dst->tdrefcount = -1;
+
+ TupleDescFinalize(dst);
}
/*
@@ -1065,6 +1076,8 @@ BuildDescFromLists(const List *names, const List *types, const List *typmods, co
TupleDescInitEntryCollation(desc, attnum, attcollation);
}
+ TupleDescFinalize(desc);
+
return desc;
}
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index ff927279cc3..fe7b984ff32 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -129,6 +129,7 @@ initGinState(GinState *state, Relation index)
attr->attndims);
TupleDescInitEntryCollation(state->tupdesc[i], (AttrNumber) 2,
attr->attcollation);
+ TupleDescFinalize(state->tupdesc[i]);
}
/*
diff --git a/src/backend/access/gist/gistscan.c b/src/backend/access/gist/gistscan.c
index f23bc4a6757..c65f93abdae 100644
--- a/src/backend/access/gist/gistscan.c
+++ b/src/backend/access/gist/gistscan.c
@@ -201,6 +201,7 @@ gistrescan(IndexScanDesc scan, ScanKey key, int nkeys,
attno - 1)->atttypid,
-1, 0);
}
+ TupleDescFinalize(so->giststate->fetchTupdesc);
scan->xs_hitupdesc = so->giststate->fetchTupdesc;
/* Also create a memory context that will hold the returned tuples */
diff --git a/src/backend/access/spgist/spgutils.c b/src/backend/access/spgist/spgutils.c
index 9f5379b87ac..b246e8127db 100644
--- a/src/backend/access/spgist/spgutils.c
+++ b/src/backend/access/spgist/spgutils.c
@@ -340,6 +340,7 @@ getSpGistTupleDesc(Relation index, SpGistTypeDesc *keyType)
TupleDescCompactAttr(outTupDesc, i)->attcacheoff = -1;
populate_compact_attribute(outTupDesc, spgKeyColumn);
+ TupleDescFinalize(outTupDesc);
}
return outTupDesc;
}
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index e4340b59640..7f4ed02a6b9 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -744,6 +744,7 @@ pg_prepared_xact(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 5, "dbid",
OIDOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
/*
diff --git a/src/backend/access/transam/xlogfuncs.c b/src/backend/access/transam/xlogfuncs.c
index 2efe4105efb..b6bc616c74c 100644
--- a/src/backend/access/transam/xlogfuncs.c
+++ b/src/backend/access/transam/xlogfuncs.c
@@ -400,6 +400,7 @@ pg_walfile_name_offset(PG_FUNCTION_ARGS)
TupleDescInitEntry(resultTupleDesc, (AttrNumber) 2, "file_offset",
INT4OID, -1, 0);
+ TupleDescFinalize(resultTupleDesc);
resultTupleDesc = BlessTupleDesc(resultTupleDesc);
/*
diff --git a/src/backend/backup/basebackup_copy.c b/src/backend/backup/basebackup_copy.c
index 07f58b39d8c..6c3453efd80 100644
--- a/src/backend/backup/basebackup_copy.c
+++ b/src/backend/backup/basebackup_copy.c
@@ -357,6 +357,8 @@ SendXlogRecPtrResult(XLogRecPtr ptr, TimeLineID tli)
*/
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "tli", INT8OID, -1, 0);
+ TupleDescFinalize(tupdesc);
+
/* send RowDescription */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -388,6 +390,7 @@ SendTablespaceList(List *tablespaces)
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 1, "spcoid", OIDOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "spclocation", TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 3, "size", INT8OID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* send RowDescription */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 43de42ce39e..75e97fb394a 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -481,6 +481,8 @@ ConstructTupleDescriptor(Relation heapRelation,
populate_compact_attribute(indexTupDesc, i);
}
+ TupleDescFinalize(indexTupDesc);
+
return indexTupDesc;
}
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index 9a4791c573e..fa353a0dd37 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -1230,6 +1230,7 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 4, "qual",
PG_NODE_TREEOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
funcctx->user_fctx = table_infos;
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index c78dcea98c1..078a1cf5127 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -229,6 +229,12 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attcompression = InvalidCompressionMethod;
TupleDescAttr(tupdesc, 2)->attcompression = InvalidCompressionMethod;
+ populate_compact_attribute(tupdesc, 0);
+ populate_compact_attribute(tupdesc, 1);
+ populate_compact_attribute(tupdesc, 2);
+
+ TupleDescFinalize(tupdesc);
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 93918a223b8..5f922c3f5c2 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -281,6 +281,7 @@ ExplainResultDesc(ExplainStmt *stmt)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "QUERY PLAN",
result_type, -1, 0);
+ TupleDescFinalize(tupdesc);
return tupdesc;
}
diff --git a/src/backend/commands/functioncmds.c b/src/backend/commands/functioncmds.c
index 242372b1e68..3afd762e9dc 100644
--- a/src/backend/commands/functioncmds.c
+++ b/src/backend/commands/functioncmds.c
@@ -2424,6 +2424,7 @@ CallStmtResultDesc(CallStmt *stmt)
-1,
0);
}
+ TupleDescFinalize(tupdesc);
}
return tupdesc;
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index e1b808bbb60..551667650ba 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -1808,6 +1808,7 @@ pg_get_sequence_data(PG_FUNCTION_ARGS)
BOOLOID, -1, 0);
TupleDescInitEntry(resultTupleDesc, (AttrNumber) 3, "page_lsn",
LSNOID, -1, 0);
+ TupleDescFinalize(resultTupleDesc);
resultTupleDesc = BlessTupleDesc(resultTupleDesc);
seqrel = try_relation_open(relid, AccessShareLock);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index b04b0dbd2a0..8678cecd53f 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -1030,6 +1030,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
}
}
+ TupleDescFinalize(descriptor);
+
/*
* For relations with table AM and partitioned tables, select access
* method to use: an explicitly indicated one, or (in the case of a
@@ -1458,6 +1460,8 @@ BuildDescForRelation(const List *columns)
populate_compact_attribute(desc, attnum - 1);
}
+ TupleDescFinalize(desc);
+
return desc;
}
diff --git a/src/backend/commands/wait.c b/src/backend/commands/wait.c
index 1290df10c6f..8e920a72372 100644
--- a/src/backend/commands/wait.c
+++ b/src/backend/commands/wait.c
@@ -338,5 +338,6 @@ WaitStmtResultDesc(WaitStmt *stmt)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "status",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
return tupdesc;
}
diff --git a/src/backend/executor/execSRF.c b/src/backend/executor/execSRF.c
index a0b111dc0e4..b481e50acfb 100644
--- a/src/backend/executor/execSRF.c
+++ b/src/backend/executor/execSRF.c
@@ -272,6 +272,7 @@ ExecMakeTableFunctionResult(SetExprState *setexpr,
funcrettype,
-1,
0);
+ TupleDescFinalize(tupdesc);
rsinfo.setDesc = tupdesc;
}
MemoryContextSwitchTo(oldcontext);
@@ -776,6 +777,7 @@ init_sexpr(Oid foid, Oid input_collation, Expr *node,
funcrettype,
-1,
0);
+ TupleDescFinalize(tupdesc);
sexpr->funcResultDesc = tupdesc;
sexpr->funcReturnsTuple = false;
}
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 5b9bb21fa7b..bb997182481 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -2174,6 +2174,8 @@ ExecTypeFromTLInternal(List *targetList, bool skipjunk)
cur_resno++;
}
+ TupleDescFinalize(typeInfo);
+
return typeInfo;
}
@@ -2208,6 +2210,8 @@ ExecTypeFromExprList(List *exprList)
cur_resno++;
}
+ TupleDescFinalize(typeInfo);
+
return typeInfo;
}
diff --git a/src/backend/executor/nodeFunctionscan.c b/src/backend/executor/nodeFunctionscan.c
index 63e605e1f81..feb82d64967 100644
--- a/src/backend/executor/nodeFunctionscan.c
+++ b/src/backend/executor/nodeFunctionscan.c
@@ -414,6 +414,7 @@ ExecInitFunctionScan(FunctionScan *node, EState *estate, int eflags)
TupleDescInitEntryCollation(tupdesc,
(AttrNumber) 1,
exprCollation(funcexpr));
+ TupleDescFinalize(tupdesc);
}
else
{
@@ -485,6 +486,7 @@ ExecInitFunctionScan(FunctionScan *node, EState *estate, int eflags)
0);
}
+ TupleDescFinalize(scan_tupdesc);
Assert(attno == natts);
}
diff --git a/src/backend/parser/parse_relation.c b/src/backend/parser/parse_relation.c
index e003db520de..9c415e166ee 100644
--- a/src/backend/parser/parse_relation.c
+++ b/src/backend/parser/parse_relation.c
@@ -1883,6 +1883,7 @@ addRangeTableEntryForFunction(ParseState *pstate,
TupleDescInitEntryCollation(tupdesc,
(AttrNumber) 1,
exprCollation(funcexpr));
+ TupleDescFinalize(tupdesc);
}
else if (functypclass == TYPEFUNC_RECORD)
{
@@ -1940,6 +1941,7 @@ addRangeTableEntryForFunction(ParseState *pstate,
i++;
}
+ TupleDescFinalize(tupdesc);
/*
* Ensure that the coldeflist defines a legal set of names (no
@@ -2008,7 +2010,7 @@ addRangeTableEntryForFunction(ParseState *pstate,
0);
/* no need to set collation */
}
-
+ TupleDescFinalize(tupdesc);
Assert(natts == totalatts);
}
else
diff --git a/src/backend/parser/parse_target.c b/src/backend/parser/parse_target.c
index dbf5b2b5c01..a03d82c0540 100644
--- a/src/backend/parser/parse_target.c
+++ b/src/backend/parser/parse_target.c
@@ -1572,6 +1572,8 @@ expandRecordVariable(ParseState *pstate, Var *var, int levelsup)
}
Assert(lname == NULL && lvar == NULL); /* lists same length? */
+ TupleDescFinalize(tupleDesc);
+
return tupleDesc;
}
diff --git a/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c b/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
index 7c8639b32e9..9f04c9ed25d 100644
--- a/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
+++ b/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
@@ -1073,6 +1073,7 @@ libpqrcv_processTuples(PGresult *pgres, WalRcvExecResult *walres,
for (coln = 0; coln < nRetTypes; coln++)
TupleDescInitEntry(walres->tupledesc, (AttrNumber) coln + 1,
PQfname(pgres, coln), retTypes[coln], -1, 0);
+ TupleDescFinalize(walres->tupledesc);
attinmeta = TupleDescGetAttInMetadata(walres->tupledesc);
/* No point in doing more here if there were no tuples returned. */
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c
index 2cde8ebc729..33a9e8d7f21 100644
--- a/src/backend/replication/walsender.c
+++ b/src/backend/replication/walsender.c
@@ -451,6 +451,7 @@ IdentifySystem(void)
TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 4, "dbname",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -496,6 +497,7 @@ ReadReplicationSlot(ReadReplicationSlotCmd *cmd)
/* TimeLineID is unsigned, so int4 is not wide enough. */
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 3, "restart_tli",
INT8OID, -1, 0);
+ TupleDescFinalize(tupdesc);
memset(nulls, true, READ_REPLICATION_SLOT_COLS * sizeof(bool));
@@ -598,6 +600,7 @@ SendTimeLineHistory(TimeLineHistoryCmd *cmd)
tupdesc = CreateTemplateTupleDesc(2);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 1, "filename", TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "content", TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
TLHistoryFileName(histfname, cmd->timeline);
TLHistoryFilePath(path, cmd->timeline);
@@ -1015,6 +1018,7 @@ StartReplication(StartReplicationCmd *cmd)
INT8OID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "next_tli_startpos",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuple */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -1369,6 +1373,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd)
TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 4, "output_plugin",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
diff --git a/src/backend/utils/adt/acl.c b/src/backend/utils/adt/acl.c
index 641673f0b0e..ce07f2bc046 100644
--- a/src/backend/utils/adt/acl.c
+++ b/src/backend/utils/adt/acl.c
@@ -1819,6 +1819,7 @@ aclexplode(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 4, "is_grantable",
BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
/* allocate memory for user context */
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index c083608b1d5..bfb949401d0 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -454,6 +454,7 @@ pg_stat_file(PG_FUNCTION_ARGS)
"creation", TIMESTAMPTZOID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 6,
"isdir", BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
memset(isnull, false, sizeof(isnull));
diff --git a/src/backend/utils/adt/lockfuncs.c b/src/backend/utils/adt/lockfuncs.c
index 9dadd6da672..4481c354fd6 100644
--- a/src/backend/utils/adt/lockfuncs.c
+++ b/src/backend/utils/adt/lockfuncs.c
@@ -146,6 +146,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 16, "waitstart",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
/*
diff --git a/src/backend/utils/adt/orderedsetaggs.c b/src/backend/utils/adt/orderedsetaggs.c
index 3b6da8e36ac..fd8b8676470 100644
--- a/src/backend/utils/adt/orderedsetaggs.c
+++ b/src/backend/utils/adt/orderedsetaggs.c
@@ -233,6 +233,7 @@ ordered_set_startup(FunctionCallInfo fcinfo, bool use_tuples)
-1,
0);
+ TupleDescFinalize(newdesc);
FreeTupleDesc(qstate->tupdesc);
qstate->tupdesc = newdesc;
}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index b1df96e7b0b..0b10da3b180 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -769,6 +769,7 @@ pg_stat_get_backend_subxact(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 2, "subxact_overflow",
BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
if ((local_beentry = pgstat_get_local_beentry_by_proc_number(procNumber)) != NULL)
@@ -1670,6 +1671,7 @@ pg_stat_wal_build_tuple(PgStat_WalCounters wal_counters,
TupleDescInitEntry(tupdesc, (AttrNumber) 6, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
/* Fill values and NULLs */
@@ -2097,6 +2099,7 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 7, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
/* Get statistics about the archiver process */
@@ -2178,6 +2181,7 @@ pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
TIMESTAMPTZOID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 13, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
namestrcpy(&slotname, text_to_cstring(slotname_text));
@@ -2265,6 +2269,7 @@ pg_stat_get_subscription_stats(PG_FUNCTION_ARGS)
INT8OID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 13, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
if (!subentry)
diff --git a/src/backend/utils/adt/tsvector_op.c b/src/backend/utils/adt/tsvector_op.c
index 71c7c7d3b3c..d8dece42b9b 100644
--- a/src/backend/utils/adt/tsvector_op.c
+++ b/src/backend/utils/adt/tsvector_op.c
@@ -651,6 +651,7 @@ tsvector_unnest(PG_FUNCTION_ARGS)
TEXTARRAYOID, -1, 0);
if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
elog(ERROR, "return type must be a row type");
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = tupdesc;
funcctx->user_fctx = PG_GETARG_TSVECTOR_COPY(0);
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 6b634c9fff1..770edb34e08 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -729,6 +729,8 @@ RelationBuildTupleDesc(Relation relation)
pfree(constr);
relation->rd_att->constr = NULL;
}
+
+ TupleDescFinalize(relation->rd_att);
}
/*
@@ -1985,6 +1987,7 @@ formrdesc(const char *relationName, Oid relationReltype,
/* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
TupleDescCompactAttr(relation->rd_att, 0)->attcacheoff = 0;
+ TupleDescFinalize(relation->rd_att);
/* mark not-null status */
if (has_not_null)
@@ -3688,6 +3691,8 @@ RelationBuildLocalRelation(const char *relname,
for (i = 0; i < natts; i++)
TupleDescAttr(rel->rd_att, i)->attrelid = relid;
+ TupleDescFinalize(rel->rd_att);
+
rel->rd_rel->reltablespace = reltablespace;
if (mapped_relation)
@@ -4443,6 +4448,7 @@ BuildHardcodedDescriptor(int natts, const FormData_pg_attribute *attrs)
/* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
TupleDescCompactAttr(result, 0)->attcacheoff = 0;
+ TupleDescFinalize(result);
/* Note: we don't bother to set up a TupleConstr entry */
@@ -6268,6 +6274,8 @@ load_relcache_init_file(bool shared)
populate_compact_attribute(rel->rd_att, i);
}
+ TupleDescFinalize(rel->rd_att);
+
/* next read the access method specific field */
if (fread(&len, 1, sizeof(len), fp) != sizeof(len))
goto read_failed;
diff --git a/src/backend/utils/fmgr/funcapi.c b/src/backend/utils/fmgr/funcapi.c
index 8a934ea8dca..516d02cfb82 100644
--- a/src/backend/utils/fmgr/funcapi.c
+++ b/src/backend/utils/fmgr/funcapi.c
@@ -340,6 +340,8 @@ get_expr_result_type(Node *expr,
exprCollation(col));
i++;
}
+ TupleDescFinalize(tupdesc);
+
if (resultTypeId)
*resultTypeId = rexpr->row_typeid;
if (resultTupleDesc)
@@ -1044,6 +1046,7 @@ resolve_polymorphic_tupdesc(TupleDesc tupdesc, oidvector *declared_args,
}
}
+ TupleDescFinalize(tupdesc);
return true;
}
@@ -1853,6 +1856,8 @@ build_function_result_tupdesc_d(char prokind,
0);
}
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -1970,6 +1975,7 @@ TypeGetTupleDesc(Oid typeoid, List *colaliases)
typeoid,
-1,
0);
+ TupleDescFinalize(tupdesc);
}
else if (functypclass == TYPEFUNC_RECORD)
{
diff --git a/src/backend/utils/misc/guc_funcs.c b/src/backend/utils/misc/guc_funcs.c
index 8524dd3a981..472cb5393ce 100644
--- a/src/backend/utils/misc/guc_funcs.c
+++ b/src/backend/utils/misc/guc_funcs.c
@@ -444,6 +444,7 @@ GetPGVariableResultDesc(const char *name)
TupleDescInitEntry(tupdesc, (AttrNumber) 1, varname,
TEXTOID, -1, 0);
}
+ TupleDescFinalize(tupdesc);
return tupdesc;
}
@@ -465,6 +466,7 @@ ShowGUCConfigOption(const char *name, DestReceiver *dest)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 1, varname,
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -499,6 +501,7 @@ ShowAllGUCConfig(DestReceiver *dest)
TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 3, "description",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -934,6 +937,8 @@ show_all_settings(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 17, "pending_restart",
BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+
/*
* Generate attribute metadata needed later to produce tuples from raw
* C strings
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index d46cdbf7a3c..595413dbbc5 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -195,6 +195,7 @@ extern TupleDesc CreateTupleDescTruncatedCopy(TupleDesc tupdesc, int natts);
extern TupleDesc CreateTupleDescCopyConstr(TupleDesc tupdesc);
+#define TupleDescFinalize(d) ((void) 0)
#define TupleDescSize(src) \
(offsetof(struct TupleDescData, compact_attrs) + \
(src)->natts * sizeof(CompactAttribute) + \
diff --git a/src/pl/plpgsql/src/pl_comp.c b/src/pl/plpgsql/src/pl_comp.c
index 5ecc7766757..b72c963b3be 100644
--- a/src/pl/plpgsql/src/pl_comp.c
+++ b/src/pl/plpgsql/src/pl_comp.c
@@ -1912,6 +1912,8 @@ build_row_from_vars(PLpgSQL_variable **vars, int numvars)
TupleDescInitEntryCollation(row->rowtupdesc, i + 1, typcoll);
}
+ TupleDescFinalize(row->rowtupdesc);
+
return row;
}
diff --git a/src/test/modules/test_custom_stats/test_custom_fixed_stats.c b/src/test/modules/test_custom_stats/test_custom_fixed_stats.c
index 485e08e5c19..f9e7c717280 100644
--- a/src/test/modules/test_custom_stats/test_custom_fixed_stats.c
+++ b/src/test/modules/test_custom_stats/test_custom_fixed_stats.c
@@ -206,6 +206,7 @@ test_custom_stats_fixed_report(PG_FUNCTION_ARGS)
INT8OID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 2, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
values[0] = Int64GetDatum(stats->numcalls);
diff --git a/src/test/modules/test_predtest/test_predtest.c b/src/test/modules/test_predtest/test_predtest.c
index 679a5de456d..48ca2a4ea70 100644
--- a/src/test/modules/test_predtest/test_predtest.c
+++ b/src/test/modules/test_predtest/test_predtest.c
@@ -230,6 +230,7 @@ test_predtest(PG_FUNCTION_ARGS)
"s_r_holds", BOOLOID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 8,
"w_r_holds", BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
tupdesc = BlessTupleDesc(tupdesc);
values[0] = BoolGetDatum(strong_implied_by);
--
2.51.0
From 0c4bc383f1deae72103063a7e912f276dfd4a1c5 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Tue, 31 Dec 2024 09:19:24 +1300
Subject: [PATCH v11 4/5] Optimize tuple deformation
This commit includes various optimizations to improve the performance of
tuple deformation.
We now precalculate CompactAttribute's attcacheoff, which allows us to
remove the code from the deform routines which was setting the
attcacheoff. Setting the attcacheoff is handled by TupleDescFinalize(),
which must be called before the TupleDesc is used for anything. Having
this TupleDescFinalize() function means we can store the first
attribute in the TupleDesc which does not have an offset cached. That
allows us to add a dedicated deforming loop to deform all attributes up
to the final one with an attcacheoff set, or up to the first NULL
attribute, whichever comes first.
We also record the maximum attribute number which is guaranteed to exist
in the tuple, that is, has a NOT NULL constraint and isn't an
atthasmissing attribute. When deforming only attributes prior to the
guaranteed attnum, we've no need to access the tuple's natt count. As an
additional optimization, we only count fixed-width columns when
calculating the maximum guaranteed column as this eliminates the need to
emit code to fetch byref types in the deformation loop for guaranteed
attributes.
Some locations in the code deform tuples that have yet to go through NOT
NULL constraint validation. We're unable to perform the guaranteed
attribute optimization when that's the case. The optimization is opt-in
via the TupleTableSlot using the TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS
flag.
This commit also adds a more efficient way of populating the isnull
array by using a bit-wise trick which performs multiplication on the
inverse of the tuple's bitmap byte and masking out all but the lower bit
of each of the boolean's byte. This results in much more optimal code
when compared to determining the NULLness via att_isnull(). 8 isnull
elements are processed at once using this method, which means we need to
round the tts_isnull array size up to the next 8 bytes. The palloc code
does this anyway, but the round-up needed to be formalized so as not to
overwrite the sentinel byte in debug builds.
---
src/backend/access/common/heaptuple.c | 360 ++++++++---------
src/backend/access/common/indextuple.c | 363 +++++++----------
src/backend/access/common/tupdesc.c | 51 +++
src/backend/access/spgist/spgutils.c | 3 -
src/backend/executor/execTuples.c | 392 +++++++++++--------
src/backend/executor/nodeBitmapHeapscan.c | 3 +
src/backend/executor/nodeIndexonlyscan.c | 3 +
src/backend/executor/nodeIndexscan.c | 3 +
src/backend/executor/nodeSamplescan.c | 3 +
src/backend/executor/nodeSeqscan.c | 3 +
src/backend/executor/nodeTidrangescan.c | 3 +
src/backend/executor/nodeTidscan.c | 3 +
src/backend/jit/llvm/llvmjit_deform.c | 6 -
src/backend/utils/cache/relcache.c | 12 -
src/include/access/tupdesc.h | 20 +-
src/include/access/tupmacs.h | 224 ++++++++++-
src/include/executor/tuptable.h | 17 +-
src/test/modules/deform_bench/deform_bench.c | 1 +
18 files changed, 846 insertions(+), 624 deletions(-)
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 11bec20e82e..b2ac7fef35b 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -498,19 +498,7 @@ heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc)
* nocachegetattr
*
* This only gets called from fastgetattr(), in cases where we
- * can't use a cacheoffset and the value is not null.
- *
- * This caches attribute offsets in the attribute descriptor.
- *
- * An alternative way to speed things up would be to cache offsets
- * with the tuple, but that seems more difficult unless you take
- * the storage hit of actually putting those offsets into the
- * tuple you send to disk. Yuck.
- *
- * This scheme will be slightly slower than that, but should
- * perform well for queries which hit large #'s of tuples. After
- * you cache the offsets once, examining all the other tuples using
- * the same attribute descriptor will go much quicker. -cim 5/4/91
+ * can't use the attcacheoff and the value is not null.
*
* NOTE: if you need to change this code, see also heap_deform_tuple.
* Also see nocache_index_getattr, which is the same code for index
@@ -522,194 +510,125 @@ nocachegetattr(HeapTuple tup,
int attnum,
TupleDesc tupleDesc)
{
+ CompactAttribute *cattr;
HeapTupleHeader td = tup->t_data;
char *tp; /* ptr to data part of tuple */
bits8 *bp = td->t_bits; /* ptr to null bitmap in tuple */
- bool slow = false; /* do we have to walk attrs? */
int off; /* current offset within data */
+ int startAttr;
+ int firstNullAttr;
+ int i;
+ bool hasnulls = HeapTupleHasNulls(tup);
- /* ----------------
- * Three cases:
- *
- * 1: No nulls and no variable-width attributes.
- * 2: Has a null or a var-width AFTER att.
- * 3: Has nulls or var-widths BEFORE att.
- * ----------------
- */
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
attnum--;
- if (!HeapTupleNoNulls(tup))
+ /*
+ * To minimize the number of attributes we need to look at, start walking
+ * the tuple at the attribute with the highest attcacheoff prior to attnum
+ * or the first NULL attribute prior to attnum, whichever comes first.
+ */
+ if (hasnulls)
+ firstNullAttr = first_null_attr(bp, attnum);
+ else
+ firstNullAttr = attnum;
+
+ if (tupleDesc->firstNonCachedOffsetAttr > 0)
{
/*
- * there's a null somewhere in the tuple
- *
- * check to see if any preceding bits are null...
+ * Start at the highest attcacheoff attribute with no NULLs in prior
+ * attributes.
*/
- int byte = attnum >> 3;
- int finalbit = attnum & 0x07;
-
- /* check for nulls "before" final bit of last byte */
- if ((~bp[byte]) & ((1 << finalbit) - 1))
- slow = true;
- else
- {
- /* check for nulls in any "earlier" bytes */
- int i;
-
- for (i = 0; i < byte; i++)
- {
- if (bp[i] != 0xFF)
- {
- slow = true;
- break;
- }
- }
- }
+ startAttr = Min(tupleDesc->firstNonCachedOffsetAttr - 1, firstNullAttr);
+ off = TupleDescCompactAttr(tupleDesc, startAttr)->attcacheoff;
+ }
+ else
+ {
+ /* Otherwise, start at the beginning... */
+ startAttr = 0;
+ off = 0;
}
tp = (char *) td + td->t_hoff;
- if (!slow)
+ /*
+ * Calculate 'off' up to the first NULL attr. We use two cheaper loops
+ * when the tuple has no variable-width columns. When variable-width
+ * columns exists, we use att_addlength_pointer() to move the offset
+ * beyond the current attribute.
+ */
+ if (!HeapTupleHasVarWidth(tup))
{
- CompactAttribute *att;
-
- /*
- * If we get here, there are no nulls up to and including the target
- * attribute. If we have a cached offset, we can use it.
- */
- att = TupleDescCompactAttr(tupleDesc, attnum);
- if (att->attcacheoff >= 0)
- return fetchatt(att, tp + att->attcacheoff);
-
- /*
- * Otherwise, check for non-fixed-length attrs up to and including
- * target. If there aren't any, it's safe to cheaply initialize the
- * cached offsets for these attrs.
- */
- if (HeapTupleHasVarWidth(tup))
+ for (i = startAttr; i < firstNullAttr; i++)
{
- int j;
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- for (j = 0; j <= attnum; j++)
- {
- if (TupleDescCompactAttr(tupleDesc, j)->attlen <= 0)
- {
- slow = true;
- break;
- }
- }
+ off = att_nominal_alignby(off, cattr->attalignby);
+ off += cattr->attlen;
}
- }
-
- if (!slow)
- {
- int natts = tupleDesc->natts;
- int j = 1;
-
- /*
- * If we get here, we have a tuple with no nulls or var-widths up to
- * and including the target attribute, so we can use the cached offset
- * ... only we don't have it yet, or we'd not have got here. Since
- * it's cheap to compute offsets for fixed-width columns, we take the
- * opportunity to initialize the cached offsets for *all* the leading
- * fixed-width columns, in hope of avoiding future visits to this
- * routine.
- */
- TupleDescCompactAttr(tupleDesc, 0)->attcacheoff = 0;
-
- /* we might have set some offsets in the slow path previously */
- while (j < natts && TupleDescCompactAttr(tupleDesc, j)->attcacheoff > 0)
- j++;
-
- off = TupleDescCompactAttr(tupleDesc, j - 1)->attcacheoff +
- TupleDescCompactAttr(tupleDesc, j - 1)->attlen;
- for (; j < natts; j++)
+ for (; i < attnum; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, j);
+ if (att_isnull(i, bp))
+ continue;
- if (att->attlen <= 0)
- break;
-
- off = att_nominal_alignby(off, att->attalignby);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- att->attcacheoff = off;
-
- off += att->attlen;
+ off = att_nominal_alignby(off, cattr->attalignby);
+ off += cattr->attlen;
}
-
- Assert(j > attnum);
-
- off = TupleDescCompactAttr(tupleDesc, attnum)->attcacheoff;
}
else
{
- bool usecache = true;
- int i;
-
- /*
- * Now we know that we have to walk the tuple CAREFULLY. But we still
- * might be able to cache some offsets for next time.
- *
- * Note - This loop is a little tricky. For each non-null attribute,
- * we have to first account for alignment padding before the attr,
- * then advance over the attr based on its length. Nulls have no
- * storage and no alignment padding either. We can use/set
- * attcacheoff until we reach either a null or a var-width attribute.
- */
- off = 0;
- for (i = 0;; i++) /* loop exit is at "break" */
+ for (i = startAttr; i < firstNullAttr; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, i);
+ int attlen;
- if (HeapTupleHasNulls(tup) && att_isnull(i, bp))
- {
- usecache = false;
- continue; /* this cannot be the target att */
- }
+ cattr = TupleDescCompactAttr(tupleDesc, i);
+ attlen = cattr->attlen;
- /* If we know the next offset, we can skip the rest */
- if (usecache && att->attcacheoff >= 0)
- off = att->attcacheoff;
- else if (att->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be
- * no pad bytes in any case: then the offset will be valid for
- * either an aligned or unaligned value.
- */
- if (usecache &&
- off == att_nominal_alignby(off, att->attalignby))
- att->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, att->attalignby, -1,
- tp + off);
- usecache = false;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, att->attalignby);
+ /*
+ * cstrings don't exist in heap tuples. Use pg_assume to instruct
+ * the compiler not to emit the cstring-related code in
+ * att_addlength_pointer().
+ */
+ pg_assume(attlen > 0 || attlen == -1);
- if (usecache)
- att->attcacheoff = off;
- }
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ attlen,
+ tp + off);
+ off = att_addlength_pointer(off, attlen, tp + off);
+ }
- if (i == attnum)
- break;
+ for (; i < attnum; i++)
+ {
+ int attlen;
- off = att_addlength_pointer(off, att->attlen, tp + off);
+ if (att_isnull(i, bp))
+ continue;
- if (usecache && att->attlen <= 0)
- usecache = false;
+ cattr = TupleDescCompactAttr(tupleDesc, i);
+ attlen = cattr->attlen;
+
+ /* As above, heaptuples have no cstrings */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ off = att_pointer_alignby(off, cattr->attalignby, attlen,
+ tp + off);
+ off = att_addlength_pointer(off, attlen, tp + off);
}
}
- return fetchatt(TupleDescCompactAttr(tupleDesc, attnum), tp + off);
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+
+ return fetchatt(cattr, tp + off);
}
/* ----------------
@@ -1347,6 +1266,7 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,
Datum *values, bool *isnull)
{
HeapTupleHeader tup = tuple->t_data;
+ CompactAttribute *cattr;
bool hasnulls = HeapTupleHasNulls(tuple);
int tdesc_natts = tupleDesc->natts;
int natts; /* number of atts to extract */
@@ -1354,70 +1274,98 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,
char *tp; /* ptr to tuple data */
uint32 off; /* offset in tuple data */
bits8 *bp = tup->t_bits; /* ptr to null bitmap in tuple */
- bool slow = false; /* can we use/set attcacheoff? */
+ int firstNonCacheOffsetAttr;
+ int firstNullAttr;
natts = HeapTupleHeaderGetNatts(tup);
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
+
/*
* In inheritance situations, it is possible that the given tuple actually
* has more fields than the caller is expecting. Don't run off the end of
* the caller's arrays.
*/
natts = Min(natts, tdesc_natts);
+ firstNonCacheOffsetAttr = Min(tupleDesc->firstNonCachedOffsetAttr, natts);
+
+ if (hasnulls)
+ {
+ firstNullAttr = first_null_attr(bp, natts);
+
+ /*
+ * XXX: it'd be nice to use populate_isnull_array() here, but that
+ * requires that the isnull array's size is rounded up to the next
+ * multiple of 8. Doing that would require adjusting many locations
+ * that allocate the array.
+ */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, firstNullAttr);
+ }
+ else
+ firstNullAttr = natts;
tp = (char *) tup + tup->t_hoff;
+ attnum = 0;
- off = 0;
+ if (firstNonCacheOffsetAttr > 0)
+ {
+#ifdef USE_ASSERT_CHECKING
+ /* In Assert enabled builds, verify attcacheoff is correct */
+ int offcheck = 0;
+#endif
+ do
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ off = cattr->attcacheoff;
- for (attnum = 0; attnum < natts; attnum++)
+#ifdef USE_ASSERT_CHECKING
+ offcheck = att_nominal_alignby(offcheck, cattr->attalignby);
+ Assert(offcheck == cattr->attcacheoff);
+ offcheck += cattr->attlen;
+#endif
+
+ values[attnum] = fetch_att_noerr(tp + off,
+ cattr->attbyval,
+ cattr->attlen);
+ } while (++attnum < firstNonCacheOffsetAttr);
+ off += cattr->attlen;
+ }
+ else
+ off = 0;
+
+ for (; attnum < firstNullAttr; attnum++)
{
- CompactAttribute *thisatt = TupleDescCompactAttr(tupleDesc, attnum);
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
+ }
+
+ for (; attnum < natts; attnum++)
+ {
+ Assert(hasnulls);
- if (hasnulls && att_isnull(attnum, bp))
+ if (att_isnull(attnum, bp))
{
values[attnum] = (Datum) 0;
isnull[attnum] = true;
- slow = true; /* can't use attcacheoff anymore */
continue;
}
isnull[attnum] = false;
-
- if (!slow && thisatt->attcacheoff >= 0)
- off = thisatt->attcacheoff;
- else if (thisatt->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be no
- * pad bytes in any case: then the offset will be valid for either
- * an aligned or unaligned value.
- */
- if (!slow &&
- off == att_nominal_alignby(off, thisatt->attalignby))
- thisatt->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, thisatt->attalignby, -1,
- tp + off);
- slow = true;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, thisatt->attalignby);
-
- if (!slow)
- thisatt->attcacheoff = off;
- }
-
- values[attnum] = fetchatt(thisatt, tp + off);
-
- off = att_addlength_pointer(off, thisatt->attlen, tp + off);
-
- if (thisatt->attlen <= 0)
- slow = true; /* can't use attcacheoff anymore */
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
}
/*
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index d6350201e01..8c410853191 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -223,18 +223,6 @@ index_form_tuple_context(TupleDesc tupleDescriptor,
*
* This gets called from index_getattr() macro, and only in cases
* where we can't use cacheoffset and the value is not null.
- *
- * This caches attribute offsets in the attribute descriptor.
- *
- * An alternative way to speed things up would be to cache offsets
- * with the tuple, but that seems more difficult unless you take
- * the storage hit of actually putting those offsets into the
- * tuple you send to disk. Yuck.
- *
- * This scheme will be slightly slower than that, but should
- * perform well for queries which hit large #'s of tuples. After
- * you cache the offsets once, examining all the other tuples using
- * the same attribute descriptor will go much quicker. -cim 5/4/91
* ----------------
*/
Datum
@@ -242,205 +230,124 @@ nocache_index_getattr(IndexTuple tup,
int attnum,
TupleDesc tupleDesc)
{
+ CompactAttribute *cattr;
char *tp; /* ptr to data part of tuple */
bits8 *bp = NULL; /* ptr to null bitmap in tuple */
- bool slow = false; /* do we have to walk attrs? */
int data_off; /* tuple data offset */
int off; /* current offset within data */
+ int startAttr;
+ int firstNullAttr;
+ bool hasnulls = IndexTupleHasNulls(tup);
+ int i;
- /* ----------------
- * Three cases:
- *
- * 1: No nulls and no variable-width attributes.
- * 2: Has a null or a var-width AFTER att.
- * 3: Has nulls or var-widths BEFORE att.
- * ----------------
- */
-
- data_off = IndexInfoFindDataOffset(tup->t_info);
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
attnum--;
- if (IndexTupleHasNulls(tup))
- {
- /*
- * there's a null somewhere in the tuple
- *
- * check to see if desired att is null
- */
+ data_off = IndexInfoFindDataOffset(tup->t_info);
+ tp = (char *) tup + data_off;
- /* XXX "knows" t_bits are just after fixed tuple header! */
+ /*
+ * To minimize the number of attributes we need to look at, start walking
+ * the tuple at the attribute with the highest attcacheoff prior to attnum
+ * or the first NULL attribute prior to attnum, whichever comes first.
+ */
+ if (hasnulls)
+ {
bp = (bits8 *) ((char *) tup + sizeof(IndexTupleData));
-
- /*
- * Now check to see if any preceding bits are null...
- */
- {
- int byte = attnum >> 3;
- int finalbit = attnum & 0x07;
-
- /* check for nulls "before" final bit of last byte */
- if ((~bp[byte]) & ((1 << finalbit) - 1))
- slow = true;
- else
- {
- /* check for nulls in any "earlier" bytes */
- int i;
-
- for (i = 0; i < byte; i++)
- {
- if (bp[i] != 0xFF)
- {
- slow = true;
- break;
- }
- }
- }
- }
+ firstNullAttr = first_null_attr(bp, attnum);
}
+ else
+ firstNullAttr = attnum;
- tp = (char *) tup + data_off;
-
- if (!slow)
+ if (tupleDesc->firstNonCachedOffsetAttr > 0)
{
- CompactAttribute *att;
-
- /*
- * If we get here, there are no nulls up to and including the target
- * attribute. If we have a cached offset, we can use it.
- */
- att = TupleDescCompactAttr(tupleDesc, attnum);
- if (att->attcacheoff >= 0)
- return fetchatt(att, tp + att->attcacheoff);
-
/*
- * Otherwise, check for non-fixed-length attrs up to and including
- * target. If there aren't any, it's safe to cheaply initialize the
- * cached offsets for these attrs.
+ * Start at the highest attcacheoff attribute with no NULLs in prior
+ * attributes.
*/
- if (IndexTupleHasVarwidths(tup))
- {
- int j;
-
- for (j = 0; j <= attnum; j++)
- {
- if (TupleDescCompactAttr(tupleDesc, j)->attlen <= 0)
- {
- slow = true;
- break;
- }
- }
- }
+ startAttr = Min(tupleDesc->firstNonCachedOffsetAttr - 1, firstNullAttr);
+ off = TupleDescCompactAttr(tupleDesc, startAttr)->attcacheoff;
}
-
- if (!slow)
+ else
{
- int natts = tupleDesc->natts;
- int j = 1;
-
- /*
- * If we get here, we have a tuple with no nulls or var-widths up to
- * and including the target attribute, so we can use the cached offset
- * ... only we don't have it yet, or we'd not have got here. Since
- * it's cheap to compute offsets for fixed-width columns, we take the
- * opportunity to initialize the cached offsets for *all* the leading
- * fixed-width columns, in hope of avoiding future visits to this
- * routine.
- */
- TupleDescCompactAttr(tupleDesc, 0)->attcacheoff = 0;
+ /* Otherwise, start at the beginning... */
+ startAttr = 0;
+ off = 0;
+ }
- /* we might have set some offsets in the slow path previously */
- while (j < natts && TupleDescCompactAttr(tupleDesc, j)->attcacheoff > 0)
- j++;
+ /*
+ * Calculate 'off' up to the first NULL attr. We use two cheaper loops
+ * when the tuple has no variable-width columns. When variable-width
+ * columns exists, we use att_addlength_pointer() to move the offset
+ * beyond the current attribute.
+ */
+ if (IndexTupleHasVarwidths(tup))
+ {
+ /* Calculate the offset up until the first NULL */
+ for (i = startAttr; i < firstNullAttr; i++)
+ {
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- off = TupleDescCompactAttr(tupleDesc, j - 1)->attcacheoff +
- TupleDescCompactAttr(tupleDesc, j - 1)->attlen;
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+ off = att_addlength_pointer(off, cattr->attlen, tp + off);
+ }
- for (; j < natts; j++)
+ /* Calculate the offset for any remaining columns. */
+ for (; i < attnum; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, j);
+ Assert(hasnulls);
- if (att->attlen <= 0)
- break;
+ if (att_isnull(i, bp))
+ continue;
- off = att_nominal_alignby(off, att->attalignby);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- att->attcacheoff = off;
-
- off += att->attlen;
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+ off = att_addlength_pointer(off, cattr->attlen, tp + off);
}
-
- Assert(j > attnum);
-
- off = TupleDescCompactAttr(tupleDesc, attnum)->attcacheoff;
}
else
{
- bool usecache = true;
- int i;
+ /* Handle tuples with only fixed-width attributes */
- /*
- * Now we know that we have to walk the tuple CAREFULLY. But we still
- * might be able to cache some offsets for next time.
- *
- * Note - This loop is a little tricky. For each non-null attribute,
- * we have to first account for alignment padding before the attr,
- * then advance over the attr based on its length. Nulls have no
- * storage and no alignment padding either. We can use/set
- * attcacheoff until we reach either a null or a var-width attribute.
- */
- off = 0;
- for (i = 0;; i++) /* loop exit is at "break" */
+ /* Calculate the offset up until the first NULL */
+ for (i = startAttr; i < firstNullAttr; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, i);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- if (IndexTupleHasNulls(tup) && att_isnull(i, bp))
- {
- usecache = false;
- continue; /* this cannot be the target att */
- }
-
- /* If we know the next offset, we can skip the rest */
- if (usecache && att->attcacheoff >= 0)
- off = att->attcacheoff;
- else if (att->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be
- * no pad bytes in any case: then the offset will be valid for
- * either an aligned or unaligned value.
- */
- if (usecache &&
- off == att_nominal_alignby(off, att->attalignby))
- att->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, att->attalignby, -1,
- tp + off);
- usecache = false;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, att->attalignby);
+ Assert(cattr->attlen > 0);
+ off = att_nominal_alignby(off, cattr->attalignby);
+ off += cattr->attlen;
+ }
- if (usecache)
- att->attcacheoff = off;
- }
+ /* Calculate the offset for any remaining columns. */
+ for (; i < attnum; i++)
+ {
+ Assert(hasnulls);
- if (i == attnum)
- break;
+ if (att_isnull(i, bp))
+ continue;
- off = att_addlength_pointer(off, att->attlen, tp + off);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- if (usecache && att->attlen <= 0)
- usecache = false;
+ Assert(cattr->attlen > 0);
+ off = att_nominal_alignby(off, cattr->attalignby);
+ off += cattr->attlen;
}
}
- return fetchatt(TupleDescCompactAttr(tupleDesc, attnum), tp + off);
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ off = att_pointer_alignby(off, cattr->attalignby,
+ cattr->attlen, tp + off);
+ return fetchatt(cattr, tp + off);
}
/*
@@ -480,63 +387,87 @@ index_deform_tuple_internal(TupleDesc tupleDescriptor,
Datum *values, bool *isnull,
char *tp, bits8 *bp, int hasnulls)
{
+ CompactAttribute *cattr;
int natts = tupleDescriptor->natts; /* number of atts to extract */
- int attnum;
- int off = 0; /* offset in tuple data */
- bool slow = false; /* can we use/set attcacheoff? */
+ int attnum = 0;
+ uint32 off = 0; /* offset in tuple data */
+ int firstNonCacheOffsetAttr;
+ int firstNullAttr;
/* Assert to protect callers who allocate fixed-size arrays */
Assert(natts <= INDEX_MAX_KEYS);
- for (attnum = 0; attnum < natts; attnum++)
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDescriptor->firstNonCachedOffsetAttr >= 0);
+
+ firstNonCacheOffsetAttr = Min(tupleDescriptor->firstNonCachedOffsetAttr, natts);
+
+ if (hasnulls)
+ {
+ firstNullAttr = first_null_attr(bp, natts);
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, firstNullAttr);
+ }
+ else
+ firstNullAttr = natts;
+
+ if (firstNonCacheOffsetAttr > 0)
{
- CompactAttribute *thisatt = TupleDescCompactAttr(tupleDescriptor, attnum);
+#ifdef USE_ASSERT_CHECKING
+ /* In Assert enabled builds, verify attcacheoff is correct */
+ off = 0;
+#endif
- if (hasnulls && att_isnull(attnum, bp))
+ do
{
- values[attnum] = (Datum) 0;
- isnull[attnum] = true;
- slow = true; /* can't use attcacheoff anymore */
- continue;
- }
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDescriptor, attnum);
- isnull[attnum] = false;
+#ifdef USE_ASSERT_CHECKING
+ off = att_nominal_alignby(off, cattr->attalignby);
+ Assert(off == cattr->attcacheoff);
+ off += cattr->attlen;
+#endif
- if (!slow && thisatt->attcacheoff >= 0)
- off = thisatt->attcacheoff;
- else if (thisatt->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be no
- * pad bytes in any case: then the offset will be valid for either
- * an aligned or unaligned value.
- */
- if (!slow &&
- off == att_nominal_alignby(off, thisatt->attalignby))
- thisatt->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, thisatt->attalignby, -1,
- tp + off);
- slow = true;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, thisatt->attalignby);
+ values[attnum] = fetch_att_noerr(tp + cattr->attcacheoff, cattr->attbyval,
+ cattr->attlen);
+ } while (++attnum < firstNonCacheOffsetAttr);
- if (!slow)
- thisatt->attcacheoff = off;
- }
+ off = cattr->attcacheoff + cattr->attlen;
+ }
- values[attnum] = fetchatt(thisatt, tp + off);
+ for (; attnum < firstNullAttr; attnum++)
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDescriptor, attnum);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
+ }
- off = att_addlength_pointer(off, thisatt->attlen, tp + off);
+ for (; attnum < natts; attnum++)
+ {
+ Assert(hasnulls);
- if (thisatt->attlen <= 0)
- slow = true; /* can't use attcacheoff anymore */
+ if (att_isnull(attnum, bp))
+ {
+ values[attnum] = (Datum) 0;
+ isnull[attnum] = true;
+ continue;
+ }
+
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDescriptor, attnum);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
}
}
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 2137385a833..c68561337d7 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -197,6 +197,10 @@ CreateTemplateTupleDesc(int natts)
desc->tdtypmod = -1;
desc->tdrefcount = -1; /* assume not reference-counted */
+ /* This will be set to the correct value by TupleDescFinalize() */
+ desc->firstNonCachedOffsetAttr = -1;
+ desc->firstNonGuaranteedAttr = -1;
+
return desc;
}
@@ -457,6 +461,9 @@ TupleDescCopy(TupleDesc dst, TupleDesc src)
* descriptor to another.
*
* !!! Constraints and defaults are not copied !!!
+ *
+ * The caller must take care of calling TupleDescFinalize() on 'dst' once all
+ * TupleDesc changes have been made.
*/
void
TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
@@ -489,6 +496,50 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
populate_compact_attribute(dst, dstAttno - 1);
}
+/*
+ * TupleDescFinalize
+ * Finalize the given TupleDesc. This must be called after the
+ * attributes arrays have been populated or adjusted by any code.
+ *
+ * Must be called after populate_compact_attribute() and before
+ * BlessTupleDesc().
+ */
+void
+TupleDescFinalize(TupleDesc tupdesc)
+{
+ int firstNonCachedOffsetAttr = 0;
+ int firstNonGuaranteedAttr = tupdesc->natts;
+ int off = 0;
+
+ for (int i = 0; i < tupdesc->natts; i++)
+ {
+ CompactAttribute *cattr = TupleDescCompactAttr(tupdesc, i);
+
+ /*
+ * Find the highest attnum which is guaranteed to exist in all tuples
+ * in the table. We currently only pay attention to byval attributes
+ * to allow additional optimizations during tuple deformation.
+ */
+ if (firstNonGuaranteedAttr == tupdesc->natts &&
+ (cattr->attnullability != ATTNULLABLE_VALID || !cattr->attbyval ||
+ cattr->atthasmissing || cattr->attisdropped || cattr->attlen <= 0))
+ firstNonGuaranteedAttr = i;
+
+ if (cattr->attlen <= 0)
+ break;
+
+ off = att_nominal_alignby(off, cattr->attalignby);
+
+ cattr->attcacheoff = off;
+
+ off += cattr->attlen;
+ firstNonCachedOffsetAttr = i + 1;
+ }
+
+ tupdesc->firstNonCachedOffsetAttr = firstNonCachedOffsetAttr;
+ tupdesc->firstNonGuaranteedAttr = firstNonGuaranteedAttr;
+}
+
/*
* Free a TupleDesc including all substructure
*/
diff --git a/src/backend/access/spgist/spgutils.c b/src/backend/access/spgist/spgutils.c
index b246e8127db..a4694bd8065 100644
--- a/src/backend/access/spgist/spgutils.c
+++ b/src/backend/access/spgist/spgutils.c
@@ -335,9 +335,6 @@ getSpGistTupleDesc(Relation index, SpGistTypeDesc *keyType)
/* We shouldn't need to bother with making these valid: */
att->attcompression = InvalidCompressionMethod;
att->attcollation = InvalidOid;
- /* In case we changed typlen, we'd better reset following offsets */
- for (int i = spgFirstIncludeColumn; i < outTupDesc->natts; i++)
- TupleDescCompactAttr(outTupDesc, i)->attcacheoff = -1;
populate_compact_attribute(outTupDesc, spgKeyColumn);
TupleDescFinalize(outTupDesc);
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index bb997182481..83a8c02894d 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -993,225 +993,254 @@ tts_buffer_heap_store_tuple(TupleTableSlot *slot, HeapTuple tuple,
}
/*
- * slot_deform_heap_tuple_internal
- * An always inline helper function for use in slot_deform_heap_tuple to
- * allow the compiler to emit specialized versions of this function for
- * various combinations of "slow" and "hasnulls". For example, if a
- * given tuple has no nulls, then we needn't check "hasnulls" for every
- * attribute that we're deforming. The caller can just call this
- * function with hasnulls set to constant-false and have the compiler
- * remove the constant-false branches and emit more optimal code.
- *
- * Returns the next attnum to deform, which can be equal to natts when the
- * function manages to deform all requested attributes. *offp is an input and
- * output parameter which is the byte offset within the tuple to start deforming
- * from which, on return, gets set to the offset where the next attribute
- * should be deformed from. *slowp is set to true when subsequent deforming
- * of this tuple must use a version of this function with "slow" passed as
- * true.
- *
- * Callers cannot assume when we return "attnum" (i.e. all requested
- * attributes have been deformed) that slow mode isn't required for any
- * additional deforming as the final attribute may have caused a switch to
- * slow mode.
+ * slot_deform_heap_tuple
+ * Given a TupleTableSlot, extract data from the slot's physical tuple
+ * into its Datum/isnull arrays. Data is extracted up through the
+ * reqnatts'th column. If there are insufficient attributes in the given
+ * tuple, then slot_getmissingattrs() is called to populate the
+ * remainder. If reqnatts is above the number of attributes in the
+ * slot's TupleDesc, an error is raised.
+ *
+ * This is essentially an incremental version of heap_deform_tuple:
+ * on each call we extract attributes up to the one needed, without
+ * re-computing information about previously extracted attributes.
+ * slot->tts_nvalid is the number of attributes already extracted.
+ *
+ * This is marked as always inline, so the different offp for different types
+ * of slots gets optimized away.
*/
-static pg_attribute_always_inline int
-slot_deform_heap_tuple_internal(TupleTableSlot *slot, HeapTuple tuple,
- int attnum, int natts, bool slow,
- bool hasnulls, uint32 *offp, bool *slowp)
+static pg_attribute_always_inline void
+slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
+ int reqnatts)
{
+ CompactAttribute *cattr;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
- Datum *values = slot->tts_values;
- bool *isnull = slot->tts_isnull;
HeapTupleHeader tup = tuple->t_data;
+ size_t attnum;
+ int firstNonCacheOffsetAttr;
+ int firstNonGuaranteedAttr;
+ int firstNullAttr;
+ int natts;
+ Datum *values;
+ bool *isnull;
char *tp; /* ptr to tuple data */
- bits8 *bp = tup->t_bits; /* ptr to null bitmap in tuple */
- bool slownext = false;
+ uint32 off; /* offset in tuple data */
- tp = (char *) tup + tup->t_hoff;
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
- for (; attnum < natts; attnum++)
+ isnull = slot->tts_isnull;
+
+ /*
+ * Some callers may form and deform tuples prior to NOT NULL constraints
+ * being checked. Here we'd like to optimize the case where we only need
+ * to fetch attributes before or up to the point where the attribute is
+ * guaranteed to exist in the tuple. We rely on the slot flag being set
+ * correctly to only enable this optimization when it's valid to do so.
+ * This optimization allows us to save fetching the number of attributes
+ * from the tuple and saves the additional cost of handling non-byval
+ * attrs.
+ */
+ if (TTS_OBEYS_NOT_NULL_CONSTRAINTS(slot))
+ firstNonGuaranteedAttr = Min(reqnatts, tupleDesc->firstNonGuaranteedAttr);
+ else
+ firstNonGuaranteedAttr = 0;
+
+ firstNonCacheOffsetAttr = tupleDesc->firstNonCachedOffsetAttr;
+
+ if (HeapTupleHasNulls(tuple))
{
- CompactAttribute *thisatt = TupleDescCompactAttr(tupleDesc, attnum);
+ natts = HeapTupleHeaderGetNatts(tup);
+ tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits) +
+ BITMAPLEN(natts));
- if (hasnulls && att_isnull(attnum, bp))
+ natts = Min(natts, reqnatts);
+ if (natts > firstNonGuaranteedAttr)
{
- values[attnum] = (Datum) 0;
- isnull[attnum] = true;
- if (!slow)
- {
- *slowp = true;
- return attnum + 1;
- }
- else
- continue;
- }
+ bits8 *bp = tup->t_bits;
- isnull[attnum] = false;
+ /* Find the first NULL attr */
+ firstNullAttr = first_null_attr(bp, natts);
- /* calculate the offset of this attribute */
- if (!slow && thisatt->attcacheoff >= 0)
- *offp = thisatt->attcacheoff;
- else if (thisatt->attlen == -1)
- {
/*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be no
- * pad bytes in any case: then the offset will be valid for either
- * an aligned or unaligned value.
+ * And populate the isnull array for all attributes being fetched
+ * from the tuple.
*/
- if (!slow && *offp == att_nominal_alignby(*offp, thisatt->attalignby))
- thisatt->attcacheoff = *offp;
- else
- {
- *offp = att_pointer_alignby(*offp,
- thisatt->attalignby,
- -1,
- tp + *offp);
+ populate_isnull_array(bp, natts, isnull);
- if (!slow)
- slownext = true;
- }
+ /* We can only use any cached offsets until the first NULL attr */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, firstNullAttr);
}
else
{
- /* not varlena, so safe to use att_nominal_alignby */
- *offp = att_nominal_alignby(*offp, thisatt->attalignby);
+ /* Otherwise all required columns are guaranteed to exist */
+ firstNullAttr = natts;
+ }
+ }
+ else
+ {
+ tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits));
- if (!slow)
- thisatt->attcacheoff = *offp;
+ /*
+ * We only need to look at the tuple's natts if we need more than the
+ * guaranteed number of columns
+ */
+ if (reqnatts > firstNonGuaranteedAttr)
+ natts = Min(HeapTupleHeaderGetNatts(tup), reqnatts);
+ else
+ {
+ /* No need to access the number of attributes in the tuple */
+ natts = reqnatts;
}
- values[attnum] = fetchatt(thisatt, tp + *offp);
+ /* All attrs can be fetched without checking for NULLs */
+ firstNullAttr = natts;
+ }
+
+ attnum = slot->tts_nvalid;
+ values = slot->tts_values;
+ slot->tts_nvalid = reqnatts;
- *offp = att_addlength_pointer(*offp, thisatt->attlen, tp + *offp);
+ /* Ensure we calculated tp correctly */
+ Assert(tp == (char *) tup + tup->t_hoff);
- /* check if we need to switch to slow mode */
- if (!slow)
+ if (attnum < firstNonGuaranteedAttr)
+ {
+ do
{
+ int attlen;
+
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ attlen = cattr->attlen;
+
+ /* We don't expect any non-byval types */
+ pg_assume(attlen > 0);
+
/*
- * We're unable to deform any further if the above code set
- * 'slownext', or if this isn't a fixed-width attribute.
+ * Technically we could support non-byval fixed-width types, but
+ * not doing so allows us to pass true to fetch_att_noerr() which
+ * eliminates the !attbyval branch.
*/
- if (slownext || thisatt->attlen <= 0)
- {
- *slowp = true;
- return attnum + 1;
- }
- }
- }
+ Assert(cattr->attbyval == true);
- return natts;
-}
-
-/*
- * slot_deform_heap_tuple
- * Given a TupleTableSlot, extract data from the slot's physical tuple
- * into its Datum/isnull arrays. Data is extracted up through the
- * reqnatts'th column. If there are insufficient attributes in the given
- * tuple, then slot_getmissingattrs() is called to populate the
- * remainder. If reqnatts is above the number of attributes in the
- * slot's TupleDesc, an error is raised.
- *
- * This is essentially an incremental version of heap_deform_tuple:
- * on each call we extract attributes up to the one needed, without
- * re-computing information about previously extracted attributes.
- * slot->tts_nvalid is the number of attributes already extracted.
- *
- * This is marked as always inline, so the different offp for different types
- * of slots gets optimized away.
- */
-static pg_attribute_always_inline void
-slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
- int reqnatts)
-{
- bool hasnulls = HeapTupleHasNulls(tuple);
- int attnum;
- uint32 off; /* offset in tuple data */
- bool slow; /* can we use/set attcacheoff? */
+ off = cattr->attcacheoff;
+ values[attnum] = fetch_att_noerr(tp + off, true, attlen);
+ attnum++;
+ } while (attnum < firstNonGuaranteedAttr);
- /* We can only fetch as many attributes as the tuple has. */
- natts = Min(HeapTupleHeaderGetNatts(tuple->t_data), reqnatts);
+ off += cattr->attlen;
- /*
- * Check whether the first call for this tuple, and initialize or restore
- * loop state.
- */
- attnum = slot->tts_nvalid;
- slot->tts_nvalid = reqnatts;
- if (attnum == 0)
- {
- /* Start from the first attribute */
- off = 0;
- slow = false;
+ if (attnum == reqnatts)
+ goto done;
}
else
{
/* Restore state from previous execution */
off = *offp;
- slow = TTS_SLOW(slot);
+
+ /* We expect *offp to be set to 0 when attnum == 0 */
+ Assert(off == 0 || attnum > 0);
}
+ /* We can only fetch as many attributes as the tuple has. */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, natts);
+
/*
- * If 'slow' isn't set, try deforming using deforming code that does not
- * contain any of the extra checks required for non-fixed offset
- * deforming. During deforming, if or when we find a NULL or a variable
- * length attribute, we'll switch to a deforming method which includes the
- * extra code required for non-fixed offset deforming, a.k.a slow mode.
- * Because this is performance critical, we inline
- * slot_deform_heap_tuple_internal passing the 'slow' and 'hasnull'
- * parameters as constants to allow the compiler to emit specialized code
- * with the known-const false comparisons and subsequent branches removed.
+ * Handle the portion of the tuple that we have cached the offset for up
+ * to the first NULL attribute. The offset is effectively fixed for
+ * these, so we can use the CompactAttribute's attcacheoff.
*/
- if (!slow)
+ if (attnum < firstNonCacheOffsetAttr)
{
- /* Tuple without any NULLs? We can skip doing any NULL checking */
- if (!hasnulls)
- attnum = slot_deform_heap_tuple_internal(slot,
- tuple,
- attnum,
- natts,
- false, /* slow */
- false, /* hasnulls */
- &off,
- &slow);
- else
- attnum = slot_deform_heap_tuple_internal(slot,
- tuple,
- attnum,
- natts,
- false, /* slow */
- true, /* hasnulls */
- &off,
- &slow);
+ do
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+
+ off = cattr->attcacheoff;
+ values[attnum] = fetch_att_noerr(tp + off,
+ cattr->attbyval,
+ cattr->attlen);
+ } while (++attnum < firstNonCacheOffsetAttr);
+
+ /*
+ * Point the offset after the end of the last attribute with a cached
+ * offset. We expect the final cached offset attribute to have a
+ * fixed width, so just add the attlen to the attcacheoff
+ */
+ Assert(cattr->attlen > 0);
+ off += cattr->attlen;
}
- /* If there's still work to do then we must be in slow mode */
- if (attnum < natts)
+ /*
+ * Handle any portion of the tuple that doesn't have a fixed offset up
+ * until the first NULL attribute. This loop only differs from the one
+ * after it by the NULL checks.
+ */
+ for (; attnum < firstNullAttr; attnum++)
{
- /* XXX is it worth adding a separate call when hasnulls is false? */
- attnum = slot_deform_heap_tuple_internal(slot,
- tuple,
- attnum,
- natts,
- true, /* slow */
- hasnulls,
- &off,
- &slow);
+ int attlen;
+
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ attlen = cattr->attlen;
+
+ /*
+ * cstrings don't exist in heap tuples. Use pg_assume to instruct the
+ * compiler not to emit the cstring-related code in
+ * align_fetch_then_add().
+ */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ attlen,
+ cattr->attalignby);
}
/*
- * Save state for next execution
+ * Now handle any remaining attributes in the tuple up to the requested
+ * attnum. This time, include NULL checks as we're now at the first NULL
+ * attribute.
*/
- *offp = off;
- if (slow)
- slot->tts_flags |= TTS_FLAG_SLOW;
- else
- slot->tts_flags &= ~TTS_FLAG_SLOW;
+ for (; attnum < natts; attnum++)
+ {
+ int attlen;
+
+ if (isnull[attnum])
+ {
+ values[attnum] = (Datum) 0;
+ continue;
+ }
+
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ attlen = cattr->attlen;
- /* Fetch any missing attrs and raise an error if reqnatts is invalid. */
+ /* As above, we don't expect cstrings */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ attlen,
+ cattr->attalignby);
+ }
+
+ /* Fetch any missing attrs and raise an error if reqnatts is invalid */
if (unlikely(attnum < reqnatts))
+ {
+ *offp = off;
slot_getmissingattrs(slot, attnum, reqnatts);
+ return;
+ }
+done:
+
+ /* Save current offset for next execution */
+ *offp = off;
}
const TupleTableSlotOps TTSOpsVirtual = {
@@ -1341,10 +1370,17 @@ MakeTupleTableSlot(TupleDesc tupleDesc,
slot->tts_values = (Datum *)
(((char *) slot)
+ MAXALIGN(basesz));
+
+ /*
+ * We round the size of tts_isnull up to the next highest multiple of
+ * 8. This is needed as populate_isnull_array() operates on 8
+ * elements at a time when converting a tuple's NULL bitmap into a
+ * boolean array.
+ */
slot->tts_isnull = (bool *)
(((char *) slot)
+ MAXALIGN(basesz)
- + MAXALIGN(tupleDesc->natts * sizeof(Datum)));
+ + TYPEALIGN(8, tupleDesc->natts * sizeof(Datum)));
PinTupleDesc(tupleDesc);
}
@@ -1514,8 +1550,14 @@ ExecSetSlotDescriptor(TupleTableSlot *slot, /* slot to change */
*/
slot->tts_values = (Datum *)
MemoryContextAlloc(slot->tts_mcxt, tupdesc->natts * sizeof(Datum));
+
+ /*
+ * We round the size of tts_isnull up to the next highest multiple of 8.
+ * This is needed as populate_isnull_array() operates on 8 elements at a
+ * time when converting a tuple's NULL bitmap into a boolean array.
+ */
slot->tts_isnull = (bool *)
- MemoryContextAlloc(slot->tts_mcxt, tupdesc->natts * sizeof(bool));
+ MemoryContextAlloc(slot->tts_mcxt, TYPEALIGN(8, tupdesc->natts * sizeof(bool)));
}
/* --------------------------------
@@ -2260,10 +2302,16 @@ ExecTypeSetColNames(TupleDesc typeInfo, List *namesList)
* This happens "for free" if the tupdesc came from a relcache entry, but
* not if we have manufactured a tupdesc for a transient RECORD datatype.
* In that case we have to notify typcache.c of the existence of the type.
+ *
+ * TupleDescFinalize() must be called on the TupleDesc before calling this
+ * function.
*/
TupleDesc
BlessTupleDesc(TupleDesc tupdesc)
{
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupdesc->firstNonCachedOffsetAttr >= 0);
+
if (tupdesc->tdtypeid == RECORDOID &&
tupdesc->tdtypmod < 0)
assign_record_type_typmod(tupdesc);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index c68c26cbf38..b17c4e721b3 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -383,6 +383,9 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
RelationGetDescr(currentRelation),
table_slot_callbacks(currentRelation));
+ scanstate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
+
/*
* Initialize result type and projection.
*/
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index c2d09374517..506fdf446d2 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -569,6 +569,9 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
ExecInitScanTupleSlot(estate, &indexstate->ss, tupDesc,
&TTSOpsVirtual);
+ indexstate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
+
/*
* We need another slot, in a format that's suitable for the table AM, for
* when we need to fetch a tuple from the table for rechecking visibility.
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index a616abff04c..c77746ab9f5 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -940,6 +940,9 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
RelationGetDescr(currentRelation),
table_slot_callbacks(currentRelation));
+ indexstate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
+
/*
* Initialize result type and projection.
*/
diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c
index 1b0af70fd7a..d29ef2872f7 100644
--- a/src/backend/executor/nodeSamplescan.c
+++ b/src/backend/executor/nodeSamplescan.c
@@ -130,6 +130,9 @@ ExecInitSampleScan(SampleScan *node, EState *estate, int eflags)
RelationGetDescr(scanstate->ss.ss_currentRelation),
table_slot_callbacks(scanstate->ss.ss_currentRelation));
+ scanstate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
+
/*
* Initialize result type and projection.
*/
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index af3c788ce8b..3ff2a2843eb 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -246,6 +246,9 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags)
RelationGetDescr(scanstate->ss.ss_currentRelation),
table_slot_callbacks(scanstate->ss.ss_currentRelation));
+ scanstate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
+
/*
* Initialize result type and projection.
*/
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index 503817da65b..2ece0255e7d 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -396,6 +396,9 @@ ExecInitTidRangeScan(TidRangeScan *node, EState *estate, int eflags)
RelationGetDescr(currentRelation),
table_slot_callbacks(currentRelation));
+ tidrangestate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
+
/*
* Initialize result type and projection.
*/
diff --git a/src/backend/executor/nodeTidscan.c b/src/backend/executor/nodeTidscan.c
index 4eddb0828b5..484e3306e0b 100644
--- a/src/backend/executor/nodeTidscan.c
+++ b/src/backend/executor/nodeTidscan.c
@@ -538,6 +538,9 @@ ExecInitTidScan(TidScan *node, EState *estate, int eflags)
RelationGetDescr(currentRelation),
table_slot_callbacks(currentRelation));
+ tidstate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
+
/*
* Initialize result type and projection.
*/
diff --git a/src/backend/jit/llvm/llvmjit_deform.c b/src/backend/jit/llvm/llvmjit_deform.c
index 3eb087eb56b..12521e3e46a 100644
--- a/src/backend/jit/llvm/llvmjit_deform.c
+++ b/src/backend/jit/llvm/llvmjit_deform.c
@@ -62,7 +62,6 @@ slot_compile_deform(LLVMJitContext *context, TupleDesc desc,
LLVMValueRef v_tts_values;
LLVMValueRef v_tts_nulls;
LLVMValueRef v_slotoffp;
- LLVMValueRef v_flagsp;
LLVMValueRef v_nvalidp;
LLVMValueRef v_nvalid;
LLVMValueRef v_maxatt;
@@ -178,7 +177,6 @@ slot_compile_deform(LLVMJitContext *context, TupleDesc desc,
v_tts_nulls =
l_load_struct_gep(b, StructTupleTableSlot, v_slot, FIELDNO_TUPLETABLESLOT_ISNULL,
"tts_ISNULL");
- v_flagsp = l_struct_gep(b, StructTupleTableSlot, v_slot, FIELDNO_TUPLETABLESLOT_FLAGS, "");
v_nvalidp = l_struct_gep(b, StructTupleTableSlot, v_slot, FIELDNO_TUPLETABLESLOT_NVALID, "");
if (ops == &TTSOpsHeapTuple || ops == &TTSOpsBufferHeapTuple)
@@ -747,14 +745,10 @@ slot_compile_deform(LLVMJitContext *context, TupleDesc desc,
{
LLVMValueRef v_off = l_load(b, TypeSizeT, v_offp, "");
- LLVMValueRef v_flags;
LLVMBuildStore(b, l_int16_const(lc, natts), v_nvalidp);
v_off = LLVMBuildTrunc(b, v_off, LLVMInt32TypeInContext(lc), "");
LLVMBuildStore(b, v_off, v_slotoffp);
- v_flags = l_load(b, LLVMInt16TypeInContext(lc), v_flagsp, "tts_flags");
- v_flags = LLVMBuildOr(b, v_flags, l_int16_const(lc, TTS_FLAG_SLOW), "");
- LLVMBuildStore(b, v_flags, v_flagsp);
LLVMBuildRetVoid(b);
}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 770edb34e08..998be24ac41 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -666,14 +666,6 @@ RelationBuildTupleDesc(Relation relation)
elog(ERROR, "pg_attribute catalog is missing %d attribute(s) for relation OID %u",
need, RelationGetRelid(relation));
- /*
- * We can easily set the attcacheoff value for the first attribute: it
- * must be zero. This eliminates the need for special cases for attnum=1
- * that used to exist in fastgetattr() and index_getattr().
- */
- if (RelationGetNumberOfAttributes(relation) > 0)
- TupleDescCompactAttr(relation->rd_att, 0)->attcacheoff = 0;
-
/*
* Set up constraint/default info
*/
@@ -1985,8 +1977,6 @@ formrdesc(const char *relationName, Oid relationReltype,
populate_compact_attribute(relation->rd_att, i);
}
- /* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
- TupleDescCompactAttr(relation->rd_att, 0)->attcacheoff = 0;
TupleDescFinalize(relation->rd_att);
/* mark not-null status */
@@ -4446,8 +4436,6 @@ BuildHardcodedDescriptor(int natts, const FormData_pg_attribute *attrs)
populate_compact_attribute(result, i);
}
- /* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
- TupleDescCompactAttr(result, 0)->attcacheoff = 0;
TupleDescFinalize(result);
/* Note: we don't bother to set up a TupleConstr entry */
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 595413dbbc5..ad7bc013812 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -131,6 +131,19 @@ typedef struct CompactAttribute
* Any code making changes manually to and fields in the FormData_pg_attribute
* array must subsequently call populate_compact_attribute() to flush the
* changes out to the corresponding 'compact_attrs' element.
+ *
+ * firstNonCachedOffsetAttr stores the index into the compact_attrs array for
+ * the first attribute that we don't have a known attcacheoff for.
+ *
+ * firstNonGuaranteedAttr stores the index to into the compact_attrs array for
+ * the first attribute that is either NULLable, missing, or !attbyval. This
+ * can be used in locations as a guarantee that attributes before this will
+ * always exist in tuples. The !attbyval part isn't required for this, but
+ * including this allows various tuple deforming routines to forego any checks
+ * for !attbyval.
+ *
+ * Once a TupleDesc has been populated, before it is used for any purpose,
+ * TupleDescFinalize() must be called on it.
*/
typedef struct TupleDescData
{
@@ -138,6 +151,11 @@ typedef struct TupleDescData
Oid tdtypeid; /* composite type ID for tuple type */
int32 tdtypmod; /* typmod for tuple type */
int tdrefcount; /* reference count, or -1 if not counting */
+ int firstNonCachedOffsetAttr; /* index of the first att without
+ * an attcacheoff */
+ int firstNonGuaranteedAttr; /* index of the first nullable,
+ * missing, dropped, or !attbyval
+ * attribute. */
TupleConstr *constr; /* constraints, or NULL if none */
/* compact_attrs[N] is the compact metadata of Attribute Number N+1 */
CompactAttribute compact_attrs[FLEXIBLE_ARRAY_MEMBER];
@@ -195,7 +213,6 @@ extern TupleDesc CreateTupleDescTruncatedCopy(TupleDesc tupdesc, int natts);
extern TupleDesc CreateTupleDescCopyConstr(TupleDesc tupdesc);
-#define TupleDescFinalize(d) ((void) 0)
#define TupleDescSize(src) \
(offsetof(struct TupleDescData, compact_attrs) + \
(src)->natts * sizeof(CompactAttribute) + \
@@ -206,6 +223,7 @@ extern void TupleDescCopy(TupleDesc dst, TupleDesc src);
extern void TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
TupleDesc src, AttrNumber srcAttno);
+extern void TupleDescFinalize(TupleDesc tupdesc);
extern void FreeTupleDesc(TupleDesc tupdesc);
extern void IncrTupleDescRefCount(TupleDesc tupdesc);
diff --git a/src/include/access/tupmacs.h b/src/include/access/tupmacs.h
index d64c18b950b..87dbeb76618 100644
--- a/src/include/access/tupmacs.h
+++ b/src/include/access/tupmacs.h
@@ -15,7 +15,9 @@
#define TUPMACS_H
#include "catalog/pg_type_d.h" /* for TYPALIGN macros */
-
+#include "port/pg_bitutils.h"
+#include "port/pg_bswap.h"
+#include "varatt.h"
/*
* Check a tuple's null bitmap to determine whether the attribute is null.
@@ -28,6 +30,62 @@ att_isnull(int ATT, const bits8 *BITS)
return !(BITS[ATT >> 3] & (1 << (ATT & 0x07)));
}
+/*
+ * populate_isnull_array
+ * Transform a tuple's null bitmap into a boolean array.
+ *
+ * Caller must ensure that the isnull array is sized so it contains
+ * at least as many elements as there are bits in the 'bits' array.
+ * Callers should be aware that isnull is populated 8 elements at a time,
+ * effectively as if natts is rounded up to the next multiple of 8.
+ */
+static inline void
+populate_isnull_array(const bits8 *bits, int natts, bool *isnull)
+{
+ int nbytes = (natts + 7) >> 3;
+
+ /*
+ * Multiplying the inverted NULL bitmap byte by this value results in the
+ * lowest bit in each byte being set the same as each bit of the inverted
+ * byte. We perform this as 2 32-bit operations rather than a single
+ * 64-bit operation as multiplying by the required value to do this in
+ * 64-bits would result in overflowing a uint64 in some cases.
+ *
+ * XXX if we ever require BMI2 (-march=x86-64-v3), then this could be done
+ * more efficiently on most X86-64 CPUs with the PDEP instruction. Beware
+ * that some chips (e.g. AMD's Zen2) are horribly inefficient at PDEP.
+ */
+#define SPREAD_BITS_MULTIPLIER_32 0x204081U
+
+ for (int i = 0; i < nbytes; i++, isnull += 8)
+ {
+ uint64 isnull_8;
+ bits8 nullbyte = ~bits[i];
+
+ /* Convert the lower 4 bits of NULL bitmap word into a 64 bit int */
+ isnull_8 = (nullbyte & 0xf) * SPREAD_BITS_MULTIPLIER_32;
+
+ /*
+ * Convert the upper 4 bits of null bitmap word into a 64 bit int,
+ * shift into the upper 32 bit and bitwise-OR with the result of the
+ * lower 4 bits.
+ */
+ isnull_8 |= ((uint64) ((nullbyte >> 4) * SPREAD_BITS_MULTIPLIER_32)) << 32;
+
+ /* Mask out all other bits apart from the lowest bit of each byte. */
+ isnull_8 &= UINT64CONST(0x0101010101010101);
+
+#ifdef WORDS_BIGENDIAN
+
+ /*
+ * Fix byte order on big-endian machines before copying to the array.
+ */
+ isnull_8 = pg_bswap64(isnull_8);
+#endif
+ memcpy(isnull, &isnull_8, sizeof(uint64));
+ }
+}
+
#ifndef FRONTEND
/*
* Given an attbyval and an attlen from either a Form_pg_attribute or
@@ -69,6 +127,170 @@ fetch_att(const void *T, bool attbyval, int attlen)
else
return PointerGetDatum(T);
}
+
+/*
+ * Same, but no error checking for invalid attlens for byval types. This
+ * is safe to use when attlen comes from CompactAttribute as we validate the
+ * length when populating that struct.
+ */
+static inline Datum
+fetch_att_noerr(const void *T, bool attbyval, int attlen)
+{
+ if (attbyval)
+ {
+ switch (attlen)
+ {
+ case sizeof(int32):
+ return Int32GetDatum(*((const int32 *) T));
+ case sizeof(int16):
+ return Int16GetDatum(*((const int16 *) T));
+ case sizeof(char):
+ return CharGetDatum(*((const char *) T));
+ default:
+ Assert(attlen == sizeof(int64));
+ return Int64GetDatum(*((const int64 *) T));
+ }
+ }
+ else
+ return PointerGetDatum(T);
+}
+
+
+/*
+ * align_fetch_then_add
+ * Applies all the functionality of att_pointer_alignby(),
+ * fetch_att_noerr() and att_addlength_pointer(), resulting in the *off
+ * pointer to the perhaps unaligned number of bytes into 'tupptr', ready
+ * to deform the next attribute.
+ *
+ * tupptr: pointer to the beginning of the tuple, after the header and any
+ * NULL bitmask.
+ * off: offset in bytes for reading tuple data, possibly unaligned.
+ * attbyval, attlen and attalignby are values from CompactAttribute.
+ */
+static inline Datum
+align_fetch_then_add(const char *tupptr, uint32 *off, bool attbyval, int attlen,
+ uint8 attalignby)
+{
+ Datum res;
+
+ if (attlen > 0)
+ {
+ const char *offset_ptr;
+
+ *off = TYPEALIGN(attalignby, *off);
+ offset_ptr = tupptr + *off;
+ *off += attlen;
+ if (attbyval)
+ {
+ switch (attlen)
+ {
+ case sizeof(char):
+ return CharGetDatum(*((const char *) offset_ptr));
+ case sizeof(int16):
+ return Int16GetDatum(*((const int16 *) offset_ptr));
+ case sizeof(int32):
+ return Int32GetDatum(*((const int32 *) offset_ptr));
+ default:
+
+ /*
+ * populate_compact_attribute_internal() should have
+ * checked
+ */
+ Assert(attlen == sizeof(int64));
+ return Int64GetDatum(*((const int64 *) offset_ptr));
+ }
+ }
+ return PointerGetDatum(offset_ptr);
+ }
+ else if (attlen == -1)
+ {
+ if (!VARATT_IS_SHORT(tupptr + *off))
+ *off = TYPEALIGN(attalignby, *off);
+
+ res = PointerGetDatum(tupptr + *off);
+ *off += VARSIZE_ANY(DatumGetPointer(res));
+ return res;
+ }
+ else
+ {
+ Assert(attlen == -2);
+ *off = TYPEALIGN(attalignby, *off);
+ res = PointerGetDatum(tupptr + *off);
+ *off += strlen(tupptr + *off) + 1;
+ return res;
+ }
+}
+
+/*
+ * first_null_attr
+ * Inspect a NULL bitmap from a tuple and return the 0-based attnum of the
+ * first NULL attribute. Returns natts if no NULLs were found.
+ *
+ * This is coded to expect that 'bits' contains at least one 0 bit somewhere
+ * in the array, but not necessarily < natts. Note that natts may be passed
+ * as a value lower than the number of bits physically stored in the tuple's
+ * NULL bitmap, in which case we may not find a NULL and return natts.
+ *
+ * The reason we require at least one 0 bit somewhere in the NULL bitmap is
+ * that the for loop that checks 0xFF bytes would loop to the last byte in
+ * the array if all bytes were 0xFF, and the subsequent code that finds the
+ * right-most 0 bit would access the first byte beyond the bitmap. Provided
+ * we find a 0 bit before then, that won't happen. Since tuples which have no
+ * NULLs don't have a NULL bitmap, this function won't get called for that
+ * case.
+ */
+static inline int
+first_null_attr(const bits8 *bits, int natts)
+{
+ int nattByte = natts >> 3;
+ int bytenum;
+ int res;
+
+#ifdef USE_ASSERT_CHECKING
+ int firstnull_check = natts;
+
+ /* Do it the slow way and check we get the same answer. */
+ for (int i = 0; i < natts; i++)
+ {
+ if (att_isnull(i, bits))
+ {
+ firstnull_check = i;
+ break;
+ }
+ }
+#endif
+
+ /* Process all bytes up to just before the byte for the natts attribute */
+ for (bytenum = 0; bytenum < nattByte; bytenum++)
+ {
+ /* break if there's any NULL attrs (a 0 bit) */
+ if (bits[bytenum] != 0xFF)
+ break;
+ }
+
+ /*
+ * Look for the highest 0-bit in the 'bytenum' element. To do this, we
+ * promote the uint8 to uint32 before performing the bitwise NOT and
+ * looking for the first 1-bit. This works even when the byte is 0xFF, as
+ * the bitwise NOT of 0xFF in 32 bits is 0xFFFFFF00, in which case
+ * pg_rightmost_one_pos32() will return 8. We may end up with a value
+ * higher than natts here, but we'll fix that with the Min() below.
+ */
+ res = bytenum << 3;
+ res += pg_rightmost_one_pos32(~((uint32) bits[bytenum]));
+
+ /*
+ * Since we did no masking to mask out bits beyond the natt'th bit, we may
+ * have found a bit higher than natts, so we must cap res to natts
+ */
+ res = Min(res, natts);
+
+ /* Ensure we got the same answer as the att_isnull() loop got */
+ Assert(res == firstnull_check);
+
+ return res;
+}
#endif /* FRONTEND */
/*
diff --git a/src/include/executor/tuptable.h b/src/include/executor/tuptable.h
index 3b09abbf99f..ff4572a29ae 100644
--- a/src/include/executor/tuptable.h
+++ b/src/include/executor/tuptable.h
@@ -84,9 +84,6 @@
* tts_values/tts_isnull are allocated either when the slot is created (when
* the descriptor is provided), or when a descriptor is assigned to the slot;
* they are of length equal to the descriptor's natts.
- *
- * The TTS_FLAG_SLOW flag is saved state for
- * slot_deform_heap_tuple, and should not be touched by any other code.
*----------
*/
@@ -98,9 +95,13 @@
#define TTS_FLAG_SHOULDFREE (1 << 2)
#define TTS_SHOULDFREE(slot) (((slot)->tts_flags & TTS_FLAG_SHOULDFREE) != 0)
-/* saved state for slot_deform_heap_tuple */
-#define TTS_FLAG_SLOW (1 << 3)
-#define TTS_SLOW(slot) (((slot)->tts_flags & TTS_FLAG_SLOW) != 0)
+/*
+ * true = slot's formed tuple guaranteed to not have NULLs in NOT NULLable
+ * columns.
+ */
+#define TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS (1 << 3)
+#define TTS_OBEYS_NOT_NULL_CONSTRAINTS(slot) \
+ (((slot)->tts_flags & TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS) != 0)
/* fixed tuple descriptor */
#define TTS_FLAG_FIXED (1 << 4)
@@ -123,7 +124,9 @@ typedef struct TupleTableSlot
#define FIELDNO_TUPLETABLESLOT_VALUES 5
Datum *tts_values; /* current per-attribute values */
#define FIELDNO_TUPLETABLESLOT_ISNULL 6
- bool *tts_isnull; /* current per-attribute isnull flags */
+ bool *tts_isnull; /* current per-attribute isnull flags. Array
+ * size must always be rounded up to the next
+ * multiple of 8 elements. */
MemoryContext tts_mcxt; /* slot itself is in this context */
ItemPointerData tts_tid; /* stored tuple's tid */
Oid tts_tableOid; /* table oid of tuple */
diff --git a/src/test/modules/deform_bench/deform_bench.c b/src/test/modules/deform_bench/deform_bench.c
index 7838f639bef..de39fecf8fd 100644
--- a/src/test/modules/deform_bench/deform_bench.c
+++ b/src/test/modules/deform_bench/deform_bench.c
@@ -49,6 +49,7 @@ deform_bench(PG_FUNCTION_ARGS)
tupdesc = RelationGetDescr(rel);
slot = MakeTupleTableSlot(tupdesc, &TTSOpsBufferHeapTuple);
+ slot->tts_flags |= TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
scan = table_beginscan_strat(rel, GetActiveSnapshot(), 0, NULL, true, false);
/*
--
2.51.0
Attachments:
[text/plain] tts_heap_getsomeattrs_objdump_Mintel.txt (63.8K, 2-tts_heap_getsomeattrs_objdump_Mintel.txt)
download | inline:
0000000000001bb0 <tts_heap_getsomeattrs>:
{
1bb0: f3 0f 1e fa endbr64
1bb4: 41 57 push r15
1bb6: 49 89 fb mov r11,rdi
1bb9: 41 56 push r14
1bbb: 41 55 push r13
1bbd: 41 54 push r12
1bbf: 4c 63 e6 movsxd r12,esi
1bc2: 55 push rbp
1bc3: 53 push rbx
HeapTupleHeader tup = tuple->t_data;
1bc4: 48 8b 47 40 mov rax,QWORD PTR [rdi+0x40]
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
1bc8: 48 8b 57 10 mov rdx,QWORD PTR [rdi+0x10]
isnull = slot->tts_isnull;
1bcc: 48 8b 4f 20 mov rcx,QWORD PTR [rdi+0x20]
HeapTupleHeader tup = tuple->t_data;
1bd0: 48 8b 58 10 mov rbx,QWORD PTR [rax+0x10]
if (TTS_OBEYS_NOT_NULL_CONSTRAINTS(slot))
1bd4: f6 47 04 08 test BYTE PTR [rdi+0x4],0x8
1bd8: 0f 84 02 04 00 00 je 1fe0 <tts_heap_getsomeattrs+0x430>
firstNonGuaranteedAttr = Min(reqnatts, tupleDesc->firstNonGuaranteedAttr);
1bde: 8b 42 14 mov eax,DWORD PTR [rdx+0x14]
1be1: 41 39 c4 cmp r12d,eax
1be4: 41 0f 4e c4 cmovle eax,r12d
if (attnum < firstNonGuaranteedAttr)
1be8: 48 63 e8 movsxd rbp,eax
firstNonCacheOffsetAttr = tupleDesc->firstNonCachedOffsetAttr;
1beb: 4c 63 52 10 movsxd r10,DWORD PTR [rdx+0x10]
if (HeapTupleHasNulls(tuple))
1bef: f6 43 14 01 test BYTE PTR [rbx+0x14],0x1
1bf3: 0f 84 b7 03 00 00 je 1fb0 <tts_heap_getsomeattrs+0x400>
natts = HeapTupleHeaderGetNatts(tup);
1bf9: 44 0f b7 4b 12 movzx r9d,WORD PTR [rbx+0x12]
1bfe: 41 81 e1 ff 07 00 00 and r9d,0x7ff
* Computes size of null bitmap given number of data columns.
*/
static inline int
BITMAPLEN(int NATTS)
{
return (NATTS + 7) / 8;
1c05: 45 8d 41 07 lea r8d,[r9+0x7]
1c09: 41 c1 f8 03 sar r8d,0x3
tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits) +
1c0d: 41 83 c0 1e add r8d,0x1e
1c11: 41 81 e0 f8 03 00 00 and r8d,0x3f8
1c18: 49 01 d8 add r8,rbx
natts = Min(natts, reqnatts);
1c1b: 45 39 cc cmp r12d,r9d
1c1e: 4d 0f 4e cc cmovle r9,r12
firstNullAttr = natts;
1c22: 45 89 ce mov r14d,r9d
if (natts > firstNonGuaranteedAttr)
1c25: 41 39 c1 cmp r9d,eax
1c28: 0f 8f ea 04 00 00 jg 2118 <tts_heap_getsomeattrs+0x568>
attnum = slot->tts_nvalid;
1c2e: 49 0f bf 43 06 movsx rax,WORD PTR [r11+0x6]
values = slot->tts_values;
1c33: 49 8b 7b 18 mov rdi,QWORD PTR [r11+0x18]
slot->tts_nvalid = reqnatts;
1c37: 66 45 89 63 06 mov WORD PTR [r11+0x6],r12w
if (attnum < firstNonGuaranteedAttr)
1c3c: 48 39 e8 cmp rax,rbp
1c3f: 73 7f jae 1cc0 <tts_heap_getsomeattrs+0x110>
1c41: 48 89 54 24 f0 mov QWORD PTR [rsp-0x10],rdx
1c46: 48 8d 74 c2 20 lea rsi,[rdx+rax*8+0x20]
1c4b: eb 22 jmp 1c6f <tts_heap_getsomeattrs+0xbf>
1c4d: 0f 1f 00 nop DWORD PTR [rax]
static inline Datum
fetch_att_noerr(const void *T, bool attbyval, int attlen)
{
if (attbyval)
{
switch (attlen)
1c50: 66 41 83 ff 01 cmp r15w,0x1
1c55: 74 59 je 1cb0 <tts_heap_getsomeattrs+0x100>
* Returns datum representation for a 64-bit integer.
*/
static inline Datum
Int64GetDatum(int64 X)
{
return (Datum) X;
1c57: 48 8b 12 mov rdx,QWORD PTR [rdx]
values[attnum] = fetch_att_noerr(tp + off, true, attlen);
1c5a: 48 89 14 c7 mov QWORD PTR [rdi+rax*8],rdx
attnum++;
1c5e: 48 83 c0 01 add rax,0x1
} while (attnum < firstNonGuaranteedAttr);
1c62: 48 83 c6 08 add rsi,0x8
1c66: 48 39 e8 cmp rax,rbp
1c69: 0f 83 11 03 00 00 jae 1f80 <tts_heap_getsomeattrs+0x3d0>
isnull[attnum] = false;
1c6f: c6 04 01 00 mov BYTE PTR [rcx+rax*1],0x0
off = cattr->attcacheoff;
1c73: 0f bf 16 movsx edx,WORD PTR [rsi]
cattr = &cattrs[attnum];
1c76: 49 89 f5 mov r13,rsi
attlen = cattr->attlen;
1c79: 44 0f b7 7e 02 movzx r15d,WORD PTR [rsi+0x2]
off = cattr->attcacheoff;
1c7e: 48 89 d3 mov rbx,rdx
values[attnum] = fetch_att_noerr(tp + off, true, attlen);
1c81: 4c 01 c2 add rdx,r8
1c84: 66 41 83 ff 02 cmp r15w,0x2
1c89: 74 15 je 1ca0 <tts_heap_getsomeattrs+0xf0>
1c8b: 66 41 83 ff 04 cmp r15w,0x4
1c90: 75 be jne 1c50 <tts_heap_getsomeattrs+0xa0>
return (Datum) X;
1c92: 48 63 12 movsxd rdx,DWORD PTR [rdx]
{
case sizeof(int32):
return Int32GetDatum(*((const int32 *) T));
1c95: eb c3 jmp 1c5a <tts_heap_getsomeattrs+0xaa>
1c97: 66 0f 1f 84 00 00 00 nop WORD PTR [rax+rax*1+0x0]
1c9e: 00 00
return (Datum) X;
1ca0: 48 0f bf 12 movsx rdx,WORD PTR [rdx]
case sizeof(int16):
return Int16GetDatum(*((const int16 *) T));
1ca4: eb b4 jmp 1c5a <tts_heap_getsomeattrs+0xaa>
1ca6: 66 2e 0f 1f 84 00 00 cs nop WORD PTR [rax+rax*1+0x0]
1cad: 00 00 00
return (Datum) X;
1cb0: 48 0f be 12 movsx rdx,BYTE PTR [rdx]
case sizeof(char):
return CharGetDatum(*((const char *) T));
1cb4: eb a4 jmp 1c5a <tts_heap_getsomeattrs+0xaa>
1cb6: 66 2e 0f 1f 84 00 00 cs nop WORD PTR [rax+rax*1+0x0]
1cbd: 00 00 00
off = *offp;
1cc0: 41 8b 5b 48 mov ebx,DWORD PTR [r11+0x48]
if (unlikely(attnum < reqnatts))
1cc4: 49 63 ec movsxd rbp,r12d
firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, natts);
1cc7: 45 39 ca cmp r10d,r9d
1cca: 4d 0f 4f d1 cmovg r10,r9
if (attnum < firstNonCacheOffsetAttr)
1cce: 4c 39 d0 cmp rax,r10
1cd1: 0f 82 b9 01 00 00 jb 1e90 <tts_heap_getsomeattrs+0x2e0>
for (; attnum < firstNullAttr; attnum++)
1cd7: 4d 63 d6 movsxd r10,r14d
1cda: 4c 39 d0 cmp rax,r10
1cdd: 72 5e jb 1d3d <tts_heap_getsomeattrs+0x18d>
1cdf: e9 24 05 00 00 jmp 2208 <tts_heap_getsomeattrs+0x658>
1ce4: 0f 1f 40 00 nop DWORD PTR [rax+0x0]
if (attlen > 0)
{
const char *offset_ptr;
*off = TYPEALIGN(attalignby, *off);
1ce8: 8d 5c 1e ff lea ebx,[rsi+rbx*1-0x1]
1cec: f7 de neg esi
1cee: 21 de and esi,ebx
offset_ptr = tupptr + *off;
*off += attlen;
1cf0: 41 0f bf dd movsx ebx,r13w
offset_ptr = tupptr + *off;
1cf4: 41 89 f6 mov r14d,esi
*off += attlen;
1cf7: 01 f3 add ebx,esi
offset_ptr = tupptr + *off;
1cf9: 4d 01 c6 add r14,r8
return (Datum) (uintptr_t) X;
1cfc: 4c 89 f6 mov rsi,r14
if (attbyval)
1cff: 80 7c c2 24 00 cmp BYTE PTR [rdx+rax*8+0x24],0x0
1d04: 74 2a je 1d30 <tts_heap_getsomeattrs+0x180>
{
switch (attlen)
1d06: 66 41 83 fd 02 cmp r13w,0x2
1d0b: 0f 84 ef 01 00 00 je 1f00 <tts_heap_getsomeattrs+0x350>
1d11: 66 41 83 fd 04 cmp r13w,0x4
1d16: 0f 84 d4 01 00 00 je 1ef0 <tts_heap_getsomeattrs+0x340>
1d1c: 66 41 83 fd 01 cmp r13w,0x1
1d21: 0f 85 b9 01 00 00 jne 1ee0 <tts_heap_getsomeattrs+0x330>
return (Datum) X;
1d27: 49 0f be 36 movsx rsi,BYTE PTR [r14]
1d2b: 0f 1f 44 00 00 nop DWORD PTR [rax+rax*1+0x0]
values[attnum] = align_fetch_then_add(tp,
1d30: 48 89 34 c7 mov QWORD PTR [rdi+rax*8],rsi
for (; attnum < firstNullAttr; attnum++)
1d34: 48 83 c0 01 add rax,0x1
1d38: 4c 39 d0 cmp rax,r10
1d3b: 74 73 je 1db0 <tts_heap_getsomeattrs+0x200>
isnull[attnum] = false;
1d3d: c6 04 01 00 mov BYTE PTR [rcx+rax*1],0x0
attlen = cattr->attlen;
1d41: 44 0f b7 6c c2 22 movzx r13d,WORD PTR [rdx+rax*8+0x22]
cattr->attalignby);
1d47: 0f b6 74 c2 25 movzx esi,BYTE PTR [rdx+rax*8+0x25]
if (attlen > 0)
1d4c: 66 45 85 ed test r13w,r13w
1d50: 7f 96 jg 1ce8 <tts_heap_getsomeattrs+0x138>
}
return PointerGetDatum(offset_ptr);
}
else if (attlen == -1)
{
if (!VARATT_IS_SHORT(tupptr + *off))
1d52: 41 89 dd mov r13d,ebx
1d55: 4d 01 c5 add r13,r8
1d58: 41 f6 45 00 01 test BYTE PTR [r13+0x0],0x1
1d5d: 75 0e jne 1d6d <tts_heap_getsomeattrs+0x1bd>
*off = TYPEALIGN(attalignby, *off);
1d5f: 8d 5c 1e ff lea ebx,[rsi+rbx*1-0x1]
1d63: f7 de neg esi
1d65: 21 f3 and ebx,esi
res = PointerGetDatum(tupptr + *off);
1d67: 41 89 dd mov r13d,ebx
1d6a: 4d 01 c5 add r13,r8
if (VARATT_IS_1B_E(PTR))
1d6d: 45 0f b6 75 00 movzx r14d,BYTE PTR [r13+0x0]
return (Datum) (uintptr_t) X;
1d72: 4c 89 ee mov rsi,r13
1d75: 41 80 fe 01 cmp r14b,0x1
1d79: 0f 84 59 03 00 00 je 20d8 <tts_heap_getsomeattrs+0x528>
else if (VARATT_IS_1B(PTR))
1d7f: 41 f6 c6 01 test r14b,0x1
1d83: 0f 85 87 02 00 00 jne 2010 <tts_heap_getsomeattrs+0x460>
return VARSIZE_4B(PTR);
1d89: 45 8b 75 00 mov r14d,DWORD PTR [r13+0x0]
1d8d: 41 c1 ee 02 shr r14d,0x2
values[attnum] = align_fetch_then_add(tp,
1d91: 48 89 34 c7 mov QWORD PTR [rdi+rax*8],rsi
for (; attnum < firstNullAttr; attnum++)
1d95: 48 83 c0 01 add rax,0x1
*off += VARSIZE_ANY(DatumGetPointer(res));
1d99: 44 01 f3 add ebx,r14d
1d9c: 4c 39 d0 cmp rax,r10
1d9f: 75 9c jne 1d3d <tts_heap_getsomeattrs+0x18d>
1da1: 0f 1f 40 00 nop DWORD PTR [rax+0x0]
1da5: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
1dac: 00 00 00 00
for (; attnum < natts; attnum++)
1db0: 4d 39 ca cmp r10,r9
1db3: 0f 83 57 04 00 00 jae 2210 <tts_heap_getsomeattrs+0x660>
1db9: 0f 1f 80 00 00 00 00 nop DWORD PTR [rax+0x0]
if (isnull[attnum])
1dc0: 31 c0 xor eax,eax
1dc2: 42 80 3c 11 00 cmp BYTE PTR [rcx+r10*1],0x0
1dc7: 75 57 jne 1e20 <tts_heap_getsomeattrs+0x270>
attlen = cattr->attlen;
1dc9: 42 0f b7 74 d2 22 movzx esi,WORD PTR [rdx+r10*8+0x22]
cattr->attalignby);
1dcf: 42 0f b6 44 d2 25 movzx eax,BYTE PTR [rdx+r10*8+0x25]
if (attlen > 0)
1dd5: 66 85 f6 test si,si
1dd8: 0f 8e 32 01 00 00 jle 1f10 <tts_heap_getsomeattrs+0x360>
*off = TYPEALIGN(attalignby, *off);
1dde: 8d 5c 18 ff lea ebx,[rax+rbx*1-0x1]
1de2: f7 d8 neg eax
1de4: 21 d8 and eax,ebx
*off += attlen;
1de6: 0f bf de movsx ebx,si
offset_ptr = tupptr + *off;
1de9: 41 89 c5 mov r13d,eax
*off += attlen;
1dec: 01 c3 add ebx,eax
offset_ptr = tupptr + *off;
1dee: 4d 01 c5 add r13,r8
1df1: 4c 89 e8 mov rax,r13
if (attbyval)
1df4: 42 80 7c d2 24 00 cmp BYTE PTR [rdx+r10*8+0x24],0x0
1dfa: 74 24 je 1e20 <tts_heap_getsomeattrs+0x270>
switch (attlen)
1dfc: 66 83 fe 02 cmp si,0x2
1e00: 0f 84 b2 02 00 00 je 20b8 <tts_heap_getsomeattrs+0x508>
1e06: 66 83 fe 04 cmp si,0x4
1e0a: 0f 84 88 02 00 00 je 2098 <tts_heap_getsomeattrs+0x4e8>
1e10: 66 83 fe 01 cmp si,0x1
1e14: 0f 84 d6 01 00 00 je 1ff0 <tts_heap_getsomeattrs+0x440>
return (Datum) X;
1e1a: 49 8b 45 00 mov rax,QWORD PTR [r13+0x0]
1e1e: 66 90 xchg ax,ax
values[attnum] = (Datum) 0;
1e20: 4a 89 04 d7 mov QWORD PTR [rdi+r10*8],rax
for (; attnum < natts; attnum++)
1e24: 49 83 c2 01 add r10,0x1
1e28: 4d 39 ca cmp r10,r9
1e2b: 75 93 jne 1dc0 <tts_heap_getsomeattrs+0x210>
if (unlikely(attnum < reqnatts))
1e2d: 49 39 e9 cmp r9,rbp
1e30: 0f 82 ea 03 00 00 jb 2220 <tts_heap_getsomeattrs+0x670>
*offp = off;
1e36: 41 89 5b 48 mov DWORD PTR [r11+0x48],ebx
}
1e3a: 5b pop rbx
1e3b: 5d pop rbp
1e3c: 41 5c pop r12
1e3e: 41 5d pop r13
1e40: 41 5e pop r14
1e42: 41 5f pop r15
1e44: c3 ret
1e45: 0f 1f 00 nop DWORD PTR [rax]
switch (attlen)
1e48: 66 83 fb 01 cmp bx,0x1
1e4c: 0f 84 0e 01 00 00 je 1f60 <tts_heap_getsomeattrs+0x3b0>
1e52: 48 8b 1e mov rbx,QWORD PTR [rsi]
1e55: 66 2e 0f 1f 84 00 00 cs nop WORD PTR [rax+rax*1+0x0]
1e5c: 00 00 00
1e5f: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
1e66: 00 00 00 00
1e6a: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
1e71: 00 00 00 00
1e75: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
1e7c: 00 00 00 00
} while (++attnum < firstNonCacheOffsetAttr);
1e80: 48 8d 70 01 lea rsi,[rax+0x1]
values[attnum] = fetch_att_noerr(tp + off,
1e84: 48 89 1c c7 mov QWORD PTR [rdi+rax*8],rbx
} while (++attnum < firstNonCacheOffsetAttr);
1e88: 49 39 f2 cmp r10,rsi
1e8b: 74 43 je 1ed0 <tts_heap_getsomeattrs+0x320>
1e8d: 48 89 f0 mov rax,rsi
isnull[attnum] = false;
1e90: c6 04 01 00 mov BYTE PTR [rcx+rax*1],0x0
off = cattr->attcacheoff;
1e94: 0f bf 74 c2 20 movsx esi,WORD PTR [rdx+rax*8+0x20]
1e99: 49 89 f5 mov r13,rsi
values[attnum] = fetch_att_noerr(tp + off,
1e9c: 4c 01 c6 add rsi,r8
return (Datum) (uintptr_t) X;
1e9f: 48 89 f3 mov rbx,rsi
if (attbyval)
1ea2: 80 7c c2 24 00 cmp BYTE PTR [rdx+rax*8+0x24],0x0
1ea7: 74 d7 je 1e80 <tts_heap_getsomeattrs+0x2d0>
cattr->attlen);
1ea9: 0f b7 5c c2 22 movzx ebx,WORD PTR [rdx+rax*8+0x22]
switch (attlen)
1eae: 66 83 fb 02 cmp bx,0x2
1eb2: 0f 84 b8 00 00 00 je 1f70 <tts_heap_getsomeattrs+0x3c0>
1eb8: 66 83 fb 04 cmp bx,0x4
1ebc: 75 8a jne 1e48 <tts_heap_getsomeattrs+0x298>
return (Datum) X;
1ebe: 48 63 1e movsxd rbx,DWORD PTR [rsi]
} while (++attnum < firstNonCacheOffsetAttr);
1ec1: 48 8d 70 01 lea rsi,[rax+0x1]
values[attnum] = fetch_att_noerr(tp + off,
1ec5: 48 89 1c c7 mov QWORD PTR [rdi+rax*8],rbx
} while (++attnum < firstNonCacheOffsetAttr);
1ec9: 49 39 f2 cmp r10,rsi
1ecc: 75 bf jne 1e8d <tts_heap_getsomeattrs+0x2dd>
1ece: 66 90 xchg ax,ax
off += cattr->attlen;
1ed0: 0f bf 5c c2 22 movsx ebx,WORD PTR [rdx+rax*8+0x22]
} while (++attnum < firstNonCacheOffsetAttr);
1ed5: 4c 89 d0 mov rax,r10
off += cattr->attlen;
1ed8: 44 01 eb add ebx,r13d
1edb: e9 f7 fd ff ff jmp 1cd7 <tts_heap_getsomeattrs+0x127>
return (Datum) X;
1ee0: 49 8b 36 mov rsi,QWORD PTR [r14]
return Int64GetDatum(*((const int64 *) offset_ptr));
1ee3: e9 48 fe ff ff jmp 1d30 <tts_heap_getsomeattrs+0x180>
1ee8: 0f 1f 84 00 00 00 00 nop DWORD PTR [rax+rax*1+0x0]
1eef: 00
return (Datum) X;
1ef0: 49 63 36 movsxd rsi,DWORD PTR [r14]
return Int32GetDatum(*((const int32 *) offset_ptr));
1ef3: e9 38 fe ff ff jmp 1d30 <tts_heap_getsomeattrs+0x180>
1ef8: 0f 1f 84 00 00 00 00 nop DWORD PTR [rax+rax*1+0x0]
1eff: 00
return (Datum) X;
1f00: 49 0f bf 36 movsx rsi,WORD PTR [r14]
return Int16GetDatum(*((const int16 *) offset_ptr));
1f04: e9 27 fe ff ff jmp 1d30 <tts_heap_getsomeattrs+0x180>
1f09: 0f 1f 80 00 00 00 00 nop DWORD PTR [rax+0x0]
if (!VARATT_IS_SHORT(tupptr + *off))
1f10: 89 de mov esi,ebx
1f12: 4c 01 c6 add rsi,r8
1f15: f6 06 01 test BYTE PTR [rsi],0x1
1f18: 0f 84 02 01 00 00 je 2020 <tts_heap_getsomeattrs+0x470>
if (VARATT_IS_1B_E(PTR))
1f1e: 44 0f b6 2e movzx r13d,BYTE PTR [rsi]
return (Datum) (uintptr_t) X;
1f22: 48 89 f0 mov rax,rsi
1f25: 41 80 fd 01 cmp r13b,0x1
1f29: 0f 84 0f 01 00 00 je 203e <tts_heap_getsomeattrs+0x48e>
else if (VARATT_IS_1B(PTR))
1f2f: 41 f6 c5 01 test r13b,0x1
1f33: 0f 84 3f 01 00 00 je 2078 <tts_heap_getsomeattrs+0x4c8>
return VARSIZE_1B(PTR);
1f39: 41 d0 ed shr r13b,1
*off += VARSIZE_ANY(DatumGetPointer(res));
1f3c: 45 0f b6 ed movzx r13d,r13b
values[attnum] = (Datum) 0;
1f40: 4a 89 04 d7 mov QWORD PTR [rdi+r10*8],rax
for (; attnum < natts; attnum++)
1f44: 49 83 c2 01 add r10,0x1
1f48: 44 01 eb add ebx,r13d
1f4b: 4d 39 ca cmp r10,r9
1f4e: 0f 85 6c fe ff ff jne 1dc0 <tts_heap_getsomeattrs+0x210>
1f54: e9 d4 fe ff ff jmp 1e2d <tts_heap_getsomeattrs+0x27d>
1f59: 0f 1f 80 00 00 00 00 nop DWORD PTR [rax+0x0]
return (Datum) X;
1f60: 48 0f be 1e movsx rbx,BYTE PTR [rsi]
return CharGetDatum(*((const char *) T));
1f64: e9 17 ff ff ff jmp 1e80 <tts_heap_getsomeattrs+0x2d0>
1f69: 0f 1f 80 00 00 00 00 nop DWORD PTR [rax+0x0]
return (Datum) X;
1f70: 48 0f bf 1e movsx rbx,WORD PTR [rsi]
return Int16GetDatum(*((const int16 *) T));
1f74: e9 07 ff ff ff jmp 1e80 <tts_heap_getsomeattrs+0x2d0>
1f79: 0f 1f 80 00 00 00 00 nop DWORD PTR [rax+0x0]
off += cattr->attlen;
1f80: 41 0f bf 75 02 movsx esi,WORD PTR [r13+0x2]
if (attnum == reqnatts)
1f85: 49 63 ec movsxd rbp,r12d
off += cattr->attlen;
1f88: 48 8b 54 24 f0 mov rdx,QWORD PTR [rsp-0x10]
1f8d: 01 f3 add ebx,esi
if (attnum == reqnatts)
1f8f: 48 39 e8 cmp rax,rbp
1f92: 0f 85 2f fd ff ff jne 1cc7 <tts_heap_getsomeattrs+0x117>
*offp = off;
1f98: 41 89 5b 48 mov DWORD PTR [r11+0x48],ebx
}
1f9c: 5b pop rbx
1f9d: 5d pop rbp
1f9e: 41 5c pop r12
1fa0: 41 5d pop r13
1fa2: 41 5e pop r14
1fa4: 41 5f pop r15
1fa6: c3 ret
1fa7: 66 0f 1f 84 00 00 00 nop WORD PTR [rax+rax*1+0x0]
1fae: 00 00
tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits));
1fb0: 4c 8d 43 18 lea r8,[rbx+0x18]
if (reqnatts > firstNonGuaranteedAttr)
1fb4: 41 39 c4 cmp r12d,eax
1fb7: 0f 8e cb 00 00 00 jle 2088 <tts_heap_getsomeattrs+0x4d8>
natts = Min(HeapTupleHeaderGetNatts(tup), reqnatts);
1fbd: 0f b7 43 12 movzx eax,WORD PTR [rbx+0x12]
1fc1: 25 ff 07 00 00 and eax,0x7ff
1fc6: 44 39 e0 cmp eax,r12d
1fc9: 41 0f 4f c4 cmovg eax,r12d
1fcd: 41 89 c6 mov r14d,eax
1fd0: 4c 63 c8 movsxd r9,eax
1fd3: e9 56 fc ff ff jmp 1c2e <tts_heap_getsomeattrs+0x7e>
1fd8: 0f 1f 84 00 00 00 00 nop DWORD PTR [rax+rax*1+0x0]
1fdf: 00
1fe0: 31 ed xor ebp,ebp
firstNonGuaranteedAttr = 0;
1fe2: 31 c0 xor eax,eax
1fe4: e9 02 fc ff ff jmp 1beb <tts_heap_getsomeattrs+0x3b>
1fe9: 0f 1f 80 00 00 00 00 nop DWORD PTR [rax+0x0]
return (Datum) X;
1ff0: 49 0f be 45 00 movsx rax,BYTE PTR [r13+0x0]
values[attnum] = (Datum) 0;
1ff5: 4a 89 04 d7 mov QWORD PTR [rdi+r10*8],rax
for (; attnum < natts; attnum++)
1ff9: 49 83 c2 01 add r10,0x1
1ffd: 4d 39 ca cmp r10,r9
2000: 0f 85 ba fd ff ff jne 1dc0 <tts_heap_getsomeattrs+0x210>
2006: e9 22 fe ff ff jmp 1e2d <tts_heap_getsomeattrs+0x27d>
200b: 0f 1f 44 00 00 nop DWORD PTR [rax+rax*1+0x0]
2010: 41 d0 ee shr r14b,1
*off += VARSIZE_ANY(DatumGetPointer(res));
2013: 45 0f b6 f6 movzx r14d,r14b
2017: e9 75 fd ff ff jmp 1d91 <tts_heap_getsomeattrs+0x1e1>
201c: 0f 1f 40 00 nop DWORD PTR [rax+0x0]
*off = TYPEALIGN(attalignby, *off);
2020: 8d 5c 18 ff lea ebx,[rax+rbx*1-0x1]
2024: f7 d8 neg eax
2026: 21 c3 and ebx,eax
res = PointerGetDatum(tupptr + *off);
2028: 89 de mov esi,ebx
202a: 4c 01 c6 add rsi,r8
if (VARATT_IS_1B_E(PTR))
202d: 44 0f b6 2e movzx r13d,BYTE PTR [rsi]
return (Datum) (uintptr_t) X;
2031: 48 89 f0 mov rax,rsi
2034: 41 80 fd 01 cmp r13b,0x1
2038: 0f 85 f1 fe ff ff jne 1f2f <tts_heap_getsomeattrs+0x37f>
return VARTAG_1B_E(PTR);
203e: 0f b6 76 01 movzx esi,BYTE PTR [rsi+0x1]
if (tag == VARTAG_INDIRECT)
2042: 83 fe 01 cmp esi,0x1
2045: 0f 84 12 02 00 00 je 225d <tts_heap_getsomeattrs+0x6ad>
return ((tag & ~1) == VARTAG_EXPANDED_RO);
204b: 41 89 f5 mov r13d,esi
204e: 41 83 e5 fe and r13d,0xfffffffe
else if (VARTAG_IS_EXPANDED(tag))
2052: 41 83 fd 02 cmp r13d,0x2
2056: 0f 84 01 02 00 00 je 225d <tts_heap_getsomeattrs+0x6ad>
else if (tag == VARTAG_ONDISK)
205c: 83 fe 12 cmp esi,0x12
205f: 40 0f 94 c6 sete sil
2063: 40 0f b6 f6 movzx esi,sil
2067: 48 c1 e6 04 shl rsi,0x4
*off += VARSIZE_ANY(DatumGetPointer(res));
206b: 44 8d 6e 02 lea r13d,[rsi+0x2]
206f: e9 cc fe ff ff jmp 1f40 <tts_heap_getsomeattrs+0x390>
2074: 0f 1f 40 00 nop DWORD PTR [rax+0x0]
return VARSIZE_4B(PTR);
2078: 44 8b 2e mov r13d,DWORD PTR [rsi]
207b: 41 c1 ed 02 shr r13d,0x2
207f: e9 bc fe ff ff jmp 1f40 <tts_heap_getsomeattrs+0x390>
2084: 0f 1f 40 00 nop DWORD PTR [rax+0x0]
natts = reqnatts;
2088: 4d 63 cc movsxd r9,r12d
208b: 45 89 e6 mov r14d,r12d
208e: e9 9b fb ff ff jmp 1c2e <tts_heap_getsomeattrs+0x7e>
2093: 0f 1f 44 00 00 nop DWORD PTR [rax+rax*1+0x0]
return (Datum) X;
2098: 49 63 45 00 movsxd rax,DWORD PTR [r13+0x0]
values[attnum] = (Datum) 0;
209c: 4a 89 04 d7 mov QWORD PTR [rdi+r10*8],rax
for (; attnum < natts; attnum++)
20a0: 49 83 c2 01 add r10,0x1
20a4: 4d 39 ca cmp r10,r9
20a7: 0f 85 13 fd ff ff jne 1dc0 <tts_heap_getsomeattrs+0x210>
20ad: e9 7b fd ff ff jmp 1e2d <tts_heap_getsomeattrs+0x27d>
20b2: 66 0f 1f 44 00 00 nop WORD PTR [rax+rax*1+0x0]
return (Datum) X;
20b8: 49 0f bf 45 00 movsx rax,WORD PTR [r13+0x0]
values[attnum] = (Datum) 0;
20bd: 4a 89 04 d7 mov QWORD PTR [rdi+r10*8],rax
for (; attnum < natts; attnum++)
20c1: 49 83 c2 01 add r10,0x1
20c5: 4d 39 ca cmp r10,r9
20c8: 0f 85 f2 fc ff ff jne 1dc0 <tts_heap_getsomeattrs+0x210>
20ce: e9 5a fd ff ff jmp 1e2d <tts_heap_getsomeattrs+0x27d>
20d3: 0f 1f 44 00 00 nop DWORD PTR [rax+rax*1+0x0]
return VARTAG_1B_E(PTR);
20d8: 45 0f b6 6d 01 movzx r13d,BYTE PTR [r13+0x1]
if (tag == VARTAG_INDIRECT)
20dd: 41 83 fd 01 cmp r13d,0x1
20e1: 0f 84 6b 01 00 00 je 2252 <tts_heap_getsomeattrs+0x6a2>
return ((tag & ~1) == VARTAG_EXPANDED_RO);
20e7: 45 89 ee mov r14d,r13d
20ea: 41 83 e6 fe and r14d,0xfffffffe
else if (VARTAG_IS_EXPANDED(tag))
20ee: 41 83 fe 02 cmp r14d,0x2
20f2: 0f 84 5a 01 00 00 je 2252 <tts_heap_getsomeattrs+0x6a2>
else if (tag == VARTAG_ONDISK)
20f8: 41 83 fd 12 cmp r13d,0x12
20fc: 41 0f 94 c5 sete r13b
2100: 45 0f b6 ed movzx r13d,r13b
2104: 49 c1 e5 04 shl r13,0x4
2108: 45 8d 75 02 lea r14d,[r13+0x2]
210c: e9 80 fc ff ff jmp 1d91 <tts_heap_getsomeattrs+0x1e1>
2111: 0f 1f 80 00 00 00 00 nop DWORD PTR [rax+0x0]
* case.
*/
static inline int
first_null_attr(const bits8 *bits, int natts)
{
int nattByte = natts >> 3;
2118: 45 89 cd mov r13d,r9d
211b: 41 c1 fd 03 sar r13d,0x3
}
}
#endif
/* Process all bytes up to just before the byte for the natts attribute */
for (bytenum = 0; bytenum < nattByte; bytenum++)
211f: 45 85 ed test r13d,r13d
2122: 0f 8e 40 01 00 00 jle 2268 <tts_heap_getsomeattrs+0x6b8>
2128: 48 8d 73 17 lea rsi,[rbx+0x17]
212c: 31 ff xor edi,edi
212e: eb 20 jmp 2150 <tts_heap_getsomeattrs+0x5a0>
2130: 0f 1f 44 00 00 nop DWORD PTR [rax+rax*1+0x0]
2135: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
213c: 00 00 00 00
2140: 83 c7 01 add edi,0x1
2143: 48 83 c6 01 add rsi,0x1
2147: 41 39 fd cmp r13d,edi
214a: 0f 84 ec 00 00 00 je 223c <tts_heap_getsomeattrs+0x68c>
{
/* break if there's any NULL attrs (a 0 bit) */
if (bits[bytenum] != 0xFF)
2150: 0f b6 06 movzx eax,BYTE PTR [rsi]
2153: 3c ff cmp al,0xff
2155: 74 e9 je 2140 <tts_heap_getsomeattrs+0x590>
* looking for the first 1-bit. This works even when the byte is 0xFF, as
* the bitwise NOT of 0xFF in 32 bits is 0xFFFFFF00, in which case
* pg_rightmost_one_pos32() will return 8. We may end up with a value
* higher than natts here, but we'll fix that with the Min() below.
*/
res = bytenum << 3;
2157: c1 e7 03 shl edi,0x3
res += pg_rightmost_one_pos32(~((uint32) bits[bytenum]));
215a: f7 d0 not eax
int nbytes = (natts + 7) >> 3;
215c: 45 8d 69 07 lea r13d,[r9+0x7]
pg_rightmost_one_pos32(uint32 word)
{
#ifdef HAVE__BUILTIN_CTZ
Assert(word != 0);
return __builtin_ctz(word);
2160: f3 0f bc c0 tzcnt eax,eax
res += pg_rightmost_one_pos32(~((uint32) bits[bytenum]));
2164: 01 f8 add eax,edi
/*
* Since we did no masking to mask out bits beyond the natt'th bit, we may
* have found a bit higher than natts, so we must cap res to natts
*/
res = Min(res, natts);
2166: 41 39 c1 cmp r9d,eax
2169: 41 0f 4e c1 cmovle eax,r9d
int nbytes = (natts + 7) >> 3;
216d: 41 c1 fd 03 sar r13d,0x3
res = Min(res, natts);
2171: 4c 63 f0 movsxd r14,eax
isnull_8 &= UINT64CONST(0x0101010101010101);
2174: 49 bf 01 01 01 01 01 movabs r15,0x101010101010101
217b: 01 01 01
217e: 4d 63 ed movsxd r13,r13d
for (bytenum = 0; bytenum < nattByte; bytenum++)
2181: 31 ff xor edi,edi
2183: 66 0f 1f 44 00 00 nop WORD PTR [rax+rax*1+0x0]
2189: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
2190: 00 00 00 00
2194: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
219b: 00 00 00 00
219f: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
21a6: 00 00 00 00
21aa: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
21b1: 00 00 00 00
21b5: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
21bc: 00 00 00 00
bits8 nullbyte = ~bits[i];
21c0: 0f b6 74 3b 17 movzx esi,BYTE PTR [rbx+rdi*1+0x17]
21c5: f7 d6 not esi
isnull_8 |= ((uint64) ((nullbyte >> 4) * SPREAD_BITS_MULTIPLIER_32)) << 32;
21c7: 89 f0 mov eax,esi
isnull_8 = (nullbyte & 0xf) * SPREAD_BITS_MULTIPLIER_32;
21c9: 83 e6 0f and esi,0xf
isnull_8 |= ((uint64) ((nullbyte >> 4) * SPREAD_BITS_MULTIPLIER_32)) << 32;
21cc: c0 e8 04 shr al,0x4
isnull_8 = (nullbyte & 0xf) * SPREAD_BITS_MULTIPLIER_32;
21cf: 48 69 f6 81 40 20 00 imul rsi,rsi,0x204081
isnull_8 |= ((uint64) ((nullbyte >> 4) * SPREAD_BITS_MULTIPLIER_32)) << 32;
21d6: 83 e0 0f and eax,0xf
21d9: 48 69 c0 81 40 20 00 imul rax,rax,0x204081
21e0: 48 c1 e0 20 shl rax,0x20
21e4: 48 09 f0 or rax,rsi
isnull_8 &= UINT64CONST(0x0101010101010101);
21e7: 4c 21 f8 and rax,r15
21ea: 48 89 04 f9 mov QWORD PTR [rcx+rdi*8],rax
for (int i = 0; i < nbytes; i++, isnull += 8)
21ee: 48 83 c7 01 add rdi,0x1
21f2: 4c 39 ef cmp rdi,r13
21f5: 75 c9 jne 21c0 <tts_heap_getsomeattrs+0x610>
firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, firstNullAttr);
21f7: 45 39 f2 cmp r10d,r14d
21fa: 4d 0f 4f d6 cmovg r10,r14
21fe: e9 2b fa ff ff jmp 1c2e <tts_heap_getsomeattrs+0x7e>
2203: 0f 1f 44 00 00 nop DWORD PTR [rax+rax*1+0x0]
for (; attnum < firstNullAttr; attnum++)
2208: 49 89 c2 mov r10,rax
220b: e9 a0 fb ff ff jmp 1db0 <tts_heap_getsomeattrs+0x200>
for (; attnum < natts; attnum++)
2210: 4d 89 d1 mov r9,r10
2213: e9 15 fc ff ff jmp 1e2d <tts_heap_getsomeattrs+0x27d>
2218: 0f 1f 84 00 00 00 00 nop DWORD PTR [rax+rax*1+0x0]
221f: 00
*offp = off;
2220: 41 89 5b 48 mov DWORD PTR [r11+0x48],ebx
slot_getmissingattrs(slot, attnum, reqnatts);
2224: 44 89 e2 mov edx,r12d
}
2227: 5b pop rbx
slot_getmissingattrs(slot, attnum, reqnatts);
2228: 44 89 ce mov esi,r9d
}
222b: 5d pop rbp
slot_getmissingattrs(slot, attnum, reqnatts);
222c: 4c 89 df mov rdi,r11
}
222f: 41 5c pop r12
2231: 41 5d pop r13
2233: 41 5e pop r14
2235: 41 5f pop r15
slot_getmissingattrs(slot, attnum, reqnatts);
2237: e9 b4 f8 ff ff jmp 1af0 <slot_getmissingattrs>
res = bytenum << 3;
223c: 42 8d 3c ed 00 00 00 lea edi,[r13*8+0x0]
2243: 00
res += pg_rightmost_one_pos32(~((uint32) bits[bytenum]));
2244: 4d 63 ed movsxd r13,r13d
2247: 42 0f b6 44 2b 17 movzx eax,BYTE PTR [rbx+r13*1+0x17]
224d: e9 08 ff ff ff jmp 215a <tts_heap_getsomeattrs+0x5aa>
2252: 41 be 0a 00 00 00 mov r14d,0xa
2258: e9 34 fb ff ff jmp 1d91 <tts_heap_getsomeattrs+0x1e1>
225d: 41 bd 0a 00 00 00 mov r13d,0xa
2263: e9 d8 fc ff ff jmp 1f40 <tts_heap_getsomeattrs+0x390>
2268: 0f b6 43 17 movzx eax,BYTE PTR [rbx+0x17]
int nbytes = (natts + 7) >> 3;
226c: 45 8d 69 07 lea r13d,[r9+0x7]
res += pg_rightmost_one_pos32(~((uint32) bits[bytenum]));
2270: f7 d0 not eax
2272: f3 0f bc c0 tzcnt eax,eax
res = Min(res, natts);
2276: 41 39 c1 cmp r9d,eax
2279: 41 0f 4e c1 cmovle eax,r9d
int nbytes = (natts + 7) >> 3;
227d: 41 c1 fd 03 sar r13d,0x3
res = Min(res, natts);
2281: 4c 63 f0 movsxd r14,eax
for (int i = 0; i < nbytes; i++, isnull += 8)
2284: 41 83 fd 01 cmp r13d,0x1
2288: 0f 85 69 ff ff ff jne 21f7 <tts_heap_getsomeattrs+0x647>
228e: e9 e1 fe ff ff jmp 2174 <tts_heap_getsomeattrs+0x5c4>
2293: 66 90 xchg ax,ax
2295: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
229c: 00 00 00 00
00000000000022a0 <tts_minimal_getsomeattrs>:
{
22a0: f3 0f 1e fa endbr64
22a4: 41 57 push r15
22a6: 49 89 fb mov r11,rdi
22a9: 41 56 push r14
22ab: 41 55 push r13
22ad: 41 54 push r12
22af: 4c 63 e6 movsxd r12,esi
22b2: 55 push rbp
22b3: 53 push rbx
HeapTupleHeader tup = tuple->t_data;
22b4: 48 8b 47 40 mov rax,QWORD PTR [rdi+0x40]
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
22b8: 48 8b 57 10 mov rdx,QWORD PTR [rdi+0x10]
isnull = slot->tts_isnull;
22bc: 48 8b 4f 20 mov rcx,QWORD PTR [rdi+0x20]
HeapTupleHeader tup = tuple->t_data;
22c0: 48 8b 58 10 mov rbx,QWORD PTR [rax+0x10]
if (TTS_OBEYS_NOT_NULL_CONSTRAINTS(slot))
22c4: f6 47 04 08 test BYTE PTR [rdi+0x4],0x8
22c8: 0f 84 22 04 00 00 je 26f0 <tts_minimal_getsomeattrs+0x450>
firstNonGuaranteedAttr = Min(reqnatts, tupleDesc->firstNonGuaranteedAttr);
22ce: 8b 42 14 mov eax,DWORD PTR [rdx+0x14]
22d1: 41 39 c4 cmp r12d,eax
22d4: 41 0f 4e c4 cmovle eax,r12d
if (attnum < firstNonGuaranteedAttr)
22d8: 48 63 e8 movsxd rbp,eax
firstNonCacheOffsetAttr = tupleDesc->firstNonCachedOffsetAttr;
22db: 4c 63 52 10 movsxd r10,DWORD PTR [rdx+0x10]
if (HeapTupleHasNulls(tuple))
22df: f6 43 14 01 test BYTE PTR [rbx+0x14],0x1
22e3: 0f 84 d7 03 00 00 je 26c0 <tts_minimal_getsomeattrs+0x420>
natts = HeapTupleHeaderGetNatts(tup);
22e9: 44 0f b7 4b 12 movzx r9d,WORD PTR [rbx+0x12]
22ee: 41 81 e1 ff 07 00 00 and r9d,0x7ff
22f5: 45 8d 41 07 lea r8d,[r9+0x7]
22f9: 41 c1 f8 03 sar r8d,0x3
tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits) +
22fd: 41 83 c0 1e add r8d,0x1e
2301: 41 81 e0 f8 03 00 00 and r8d,0x3f8
2308: 49 01 d8 add r8,rbx
natts = Min(natts, reqnatts);
230b: 45 39 cc cmp r12d,r9d
230e: 4d 0f 4e cc cmovle r9,r12
firstNullAttr = natts;
2312: 45 89 ce mov r14d,r9d
if (natts > firstNonGuaranteedAttr)
2315: 41 39 c1 cmp r9d,eax
2318: 0f 8f 0a 05 00 00 jg 2828 <tts_minimal_getsomeattrs+0x588>
attnum = slot->tts_nvalid;
231e: 49 0f bf 43 06 movsx rax,WORD PTR [r11+0x6]
values = slot->tts_values;
2323: 49 8b 7b 18 mov rdi,QWORD PTR [r11+0x18]
slot->tts_nvalid = reqnatts;
2327: 66 45 89 63 06 mov WORD PTR [r11+0x6],r12w
if (attnum < firstNonGuaranteedAttr)
232c: 48 39 e8 cmp rax,rbp
232f: 73 7f jae 23b0 <tts_minimal_getsomeattrs+0x110>
2331: 48 89 54 24 f0 mov QWORD PTR [rsp-0x10],rdx
2336: 48 8d 74 c2 20 lea rsi,[rdx+rax*8+0x20]
233b: eb 22 jmp 235f <tts_minimal_getsomeattrs+0xbf>
233d: 0f 1f 00 nop DWORD PTR [rax]
switch (attlen)
2340: 66 41 83 ff 01 cmp r15w,0x1
2345: 74 59 je 23a0 <tts_minimal_getsomeattrs+0x100>
return (Datum) X;
2347: 48 8b 12 mov rdx,QWORD PTR [rdx]
values[attnum] = fetch_att_noerr(tp + off, true, attlen);
234a: 48 89 14 c7 mov QWORD PTR [rdi+rax*8],rdx
attnum++;
234e: 48 83 c0 01 add rax,0x1
} while (attnum < firstNonGuaranteedAttr);
2352: 48 83 c6 08 add rsi,0x8
2356: 48 39 e8 cmp rax,rbp
2359: 0f 83 31 03 00 00 jae 2690 <tts_minimal_getsomeattrs+0x3f0>
isnull[attnum] = false;
235f: c6 04 01 00 mov BYTE PTR [rcx+rax*1],0x0
off = cattr->attcacheoff;
2363: 0f bf 16 movsx edx,WORD PTR [rsi]
cattr = &cattrs[attnum];
2366: 49 89 f5 mov r13,rsi
attlen = cattr->attlen;
2369: 44 0f b7 7e 02 movzx r15d,WORD PTR [rsi+0x2]
off = cattr->attcacheoff;
236e: 48 89 d3 mov rbx,rdx
values[attnum] = fetch_att_noerr(tp + off, true, attlen);
2371: 4c 01 c2 add rdx,r8
2374: 66 41 83 ff 02 cmp r15w,0x2
2379: 74 15 je 2390 <tts_minimal_getsomeattrs+0xf0>
237b: 66 41 83 ff 04 cmp r15w,0x4
2380: 75 be jne 2340 <tts_minimal_getsomeattrs+0xa0>
return (Datum) X;
2382: 48 63 12 movsxd rdx,DWORD PTR [rdx]
return Int32GetDatum(*((const int32 *) T));
2385: eb c3 jmp 234a <tts_minimal_getsomeattrs+0xaa>
2387: 66 0f 1f 84 00 00 00 nop WORD PTR [rax+rax*1+0x0]
238e: 00 00
return (Datum) X;
2390: 48 0f bf 12 movsx rdx,WORD PTR [rdx]
return Int16GetDatum(*((const int16 *) T));
2394: eb b4 jmp 234a <tts_minimal_getsomeattrs+0xaa>
2396: 66 2e 0f 1f 84 00 00 cs nop WORD PTR [rax+rax*1+0x0]
239d: 00 00 00
return (Datum) X;
23a0: 48 0f be 12 movsx rdx,BYTE PTR [rdx]
return CharGetDatum(*((const char *) T));
23a4: eb a4 jmp 234a <tts_minimal_getsomeattrs+0xaa>
23a6: 66 2e 0f 1f 84 00 00 cs nop WORD PTR [rax+rax*1+0x0]
23ad: 00 00 00
off = *offp;
23b0: 41 8b 5b 68 mov ebx,DWORD PTR [r11+0x68]
if (unlikely(attnum < reqnatts))
23b4: 49 63 ec movsxd rbp,r12d
firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, natts);
23b7: 45 39 ca cmp r10d,r9d
23ba: 4d 0f 4f d1 cmovg r10,r9
if (attnum < firstNonCacheOffsetAttr)
23be: 4c 39 d0 cmp rax,r10
23c1: 0f 82 c9 01 00 00 jb 2590 <tts_minimal_getsomeattrs+0x2f0>
for (; attnum < firstNullAttr; attnum++)
23c7: 4d 63 d6 movsxd r10,r14d
23ca: 4c 39 d0 cmp rax,r10
23cd: 72 5e jb 242d <tts_minimal_getsomeattrs+0x18d>
23cf: e9 34 05 00 00 jmp 2908 <tts_minimal_getsomeattrs+0x668>
23d4: 0f 1f 40 00 nop DWORD PTR [rax+0x0]
*off = TYPEALIGN(attalignby, *off);
23d8: 8d 5c 1e ff lea ebx,[rsi+rbx*1-0x1]
23dc: f7 de neg esi
23de: 21 de and esi,ebx
*off += attlen;
23e0: 41 0f bf dd movsx ebx,r13w
offset_ptr = tupptr + *off;
23e4: 41 89 f6 mov r14d,esi
*off += attlen;
23e7: 01 f3 add ebx,esi
offset_ptr = tupptr + *off;
23e9: 4d 01 c6 add r14,r8
return (Datum) (uintptr_t) X;
23ec: 4c 89 f6 mov rsi,r14
if (attbyval)
23ef: 80 7c c2 24 00 cmp BYTE PTR [rdx+rax*8+0x24],0x0
23f4: 74 2a je 2420 <tts_minimal_getsomeattrs+0x180>
switch (attlen)
23f6: 66 41 83 fd 02 cmp r13w,0x2
23fb: 0f 84 0f 02 00 00 je 2610 <tts_minimal_getsomeattrs+0x370>
2401: 66 41 83 fd 04 cmp r13w,0x4
2406: 0f 84 f4 01 00 00 je 2600 <tts_minimal_getsomeattrs+0x360>
240c: 66 41 83 fd 01 cmp r13w,0x1
2411: 0f 85 d9 01 00 00 jne 25f0 <tts_minimal_getsomeattrs+0x350>
return (Datum) X;
2417: 49 0f be 36 movsx rsi,BYTE PTR [r14]
241b: 0f 1f 44 00 00 nop DWORD PTR [rax+rax*1+0x0]
values[attnum] = align_fetch_then_add(tp,
2420: 48 89 34 c7 mov QWORD PTR [rdi+rax*8],rsi
for (; attnum < firstNullAttr; attnum++)
2424: 48 83 c0 01 add rax,0x1
2428: 4c 39 d0 cmp rax,r10
242b: 74 73 je 24a0 <tts_minimal_getsomeattrs+0x200>
isnull[attnum] = false;
242d: c6 04 01 00 mov BYTE PTR [rcx+rax*1],0x0
attlen = cattr->attlen;
2431: 44 0f b7 6c c2 22 movzx r13d,WORD PTR [rdx+rax*8+0x22]
cattr->attalignby);
2437: 0f b6 74 c2 25 movzx esi,BYTE PTR [rdx+rax*8+0x25]
if (attlen > 0)
243c: 66 45 85 ed test r13w,r13w
2440: 7f 96 jg 23d8 <tts_minimal_getsomeattrs+0x138>
if (!VARATT_IS_SHORT(tupptr + *off))
2442: 41 89 dd mov r13d,ebx
2445: 4d 01 c5 add r13,r8
2448: 41 f6 45 00 01 test BYTE PTR [r13+0x0],0x1
244d: 75 0e jne 245d <tts_minimal_getsomeattrs+0x1bd>
*off = TYPEALIGN(attalignby, *off);
244f: 8d 5c 1e ff lea ebx,[rsi+rbx*1-0x1]
2453: f7 de neg esi
2455: 21 f3 and ebx,esi
res = PointerGetDatum(tupptr + *off);
2457: 41 89 dd mov r13d,ebx
245a: 4d 01 c5 add r13,r8
if (VARATT_IS_1B_E(PTR))
245d: 45 0f b6 75 00 movzx r14d,BYTE PTR [r13+0x0]
return (Datum) (uintptr_t) X;
2462: 4c 89 ee mov rsi,r13
2465: 41 80 fe 01 cmp r14b,0x1
2469: 0f 84 79 03 00 00 je 27e8 <tts_minimal_getsomeattrs+0x548>
else if (VARATT_IS_1B(PTR))
246f: 41 f6 c6 01 test r14b,0x1
2473: 0f 85 a7 02 00 00 jne 2720 <tts_minimal_getsomeattrs+0x480>
return VARSIZE_4B(PTR);
2479: 45 8b 75 00 mov r14d,DWORD PTR [r13+0x0]
247d: 41 c1 ee 02 shr r14d,0x2
values[attnum] = align_fetch_then_add(tp,
2481: 48 89 34 c7 mov QWORD PTR [rdi+rax*8],rsi
for (; attnum < firstNullAttr; attnum++)
2485: 48 83 c0 01 add rax,0x1
*off += VARSIZE_ANY(DatumGetPointer(res));
2489: 44 01 f3 add ebx,r14d
248c: 4c 39 d0 cmp rax,r10
248f: 75 9c jne 242d <tts_minimal_getsomeattrs+0x18d>
2491: 0f 1f 40 00 nop DWORD PTR [rax+0x0]
2495: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
249c: 00 00 00 00
for (; attnum < natts; attnum++)
24a0: 4d 39 ca cmp r10,r9
24a3: 0f 83 67 04 00 00 jae 2910 <tts_minimal_getsomeattrs+0x670>
24a9: 0f 1f 80 00 00 00 00 nop DWORD PTR [rax+0x0]
if (isnull[attnum])
24b0: 31 c0 xor eax,eax
24b2: 42 80 3c 11 00 cmp BYTE PTR [rcx+r10*1],0x0
24b7: 75 57 jne 2510 <tts_minimal_getsomeattrs+0x270>
attlen = cattr->attlen;
24b9: 42 0f b7 74 d2 22 movzx esi,WORD PTR [rdx+r10*8+0x22]
cattr->attalignby);
24bf: 42 0f b6 44 d2 25 movzx eax,BYTE PTR [rdx+r10*8+0x25]
if (attlen > 0)
24c5: 66 85 f6 test si,si
24c8: 0f 8e 52 01 00 00 jle 2620 <tts_minimal_getsomeattrs+0x380>
*off = TYPEALIGN(attalignby, *off);
24ce: 8d 5c 18 ff lea ebx,[rax+rbx*1-0x1]
24d2: f7 d8 neg eax
24d4: 21 d8 and eax,ebx
*off += attlen;
24d6: 0f bf de movsx ebx,si
offset_ptr = tupptr + *off;
24d9: 41 89 c5 mov r13d,eax
*off += attlen;
24dc: 01 c3 add ebx,eax
offset_ptr = tupptr + *off;
24de: 4d 01 c5 add r13,r8
24e1: 4c 89 e8 mov rax,r13
if (attbyval)
24e4: 42 80 7c d2 24 00 cmp BYTE PTR [rdx+r10*8+0x24],0x0
24ea: 74 24 je 2510 <tts_minimal_getsomeattrs+0x270>
switch (attlen)
24ec: 66 83 fe 02 cmp si,0x2
24f0: 0f 84 d2 02 00 00 je 27c8 <tts_minimal_getsomeattrs+0x528>
24f6: 66 83 fe 04 cmp si,0x4
24fa: 0f 84 a8 02 00 00 je 27a8 <tts_minimal_getsomeattrs+0x508>
2500: 66 83 fe 01 cmp si,0x1
2504: 0f 84 f6 01 00 00 je 2700 <tts_minimal_getsomeattrs+0x460>
return (Datum) X;
250a: 49 8b 45 00 mov rax,QWORD PTR [r13+0x0]
250e: 66 90 xchg ax,ax
values[attnum] = (Datum) 0;
2510: 4a 89 04 d7 mov QWORD PTR [rdi+r10*8],rax
for (; attnum < natts; attnum++)
2514: 49 83 c2 01 add r10,0x1
2518: 4d 39 ca cmp r10,r9
251b: 75 93 jne 24b0 <tts_minimal_getsomeattrs+0x210>
if (unlikely(attnum < reqnatts))
251d: 49 39 e9 cmp r9,rbp
2520: 0f 82 fa 03 00 00 jb 2920 <tts_minimal_getsomeattrs+0x680>
*offp = off;
2526: 41 89 5b 68 mov DWORD PTR [r11+0x68],ebx
}
252a: 5b pop rbx
252b: 5d pop rbp
252c: 41 5c pop r12
252e: 41 5d pop r13
2530: 41 5e pop r14
2532: 41 5f pop r15
2534: c3 ret
2535: 0f 1f 00 nop DWORD PTR [rax]
switch (attlen)
2538: 66 83 fb 01 cmp bx,0x1
253c: 0f 84 2e 01 00 00 je 2670 <tts_minimal_getsomeattrs+0x3d0>
2542: 48 8b 1e mov rbx,QWORD PTR [rsi]
2545: 0f 1f 40 00 nop DWORD PTR [rax+0x0]
2549: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
2550: 00 00 00 00
2554: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
255b: 00 00 00 00
255f: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
2566: 00 00 00 00
256a: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
2571: 00 00 00 00
2575: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
257c: 00 00 00 00
} while (++attnum < firstNonCacheOffsetAttr);
2580: 48 8d 70 01 lea rsi,[rax+0x1]
values[attnum] = fetch_att_noerr(tp + off,
2584: 48 89 1c c7 mov QWORD PTR [rdi+rax*8],rbx
} while (++attnum < firstNonCacheOffsetAttr);
2588: 49 39 f2 cmp r10,rsi
258b: 74 53 je 25e0 <tts_minimal_getsomeattrs+0x340>
258d: 48 89 f0 mov rax,rsi
isnull[attnum] = false;
2590: c6 04 01 00 mov BYTE PTR [rcx+rax*1],0x0
off = cattr->attcacheoff;
2594: 0f bf 74 c2 20 movsx esi,WORD PTR [rdx+rax*8+0x20]
2599: 49 89 f5 mov r13,rsi
values[attnum] = fetch_att_noerr(tp + off,
259c: 4c 01 c6 add rsi,r8
return (Datum) (uintptr_t) X;
259f: 48 89 f3 mov rbx,rsi
if (attbyval)
25a2: 80 7c c2 24 00 cmp BYTE PTR [rdx+rax*8+0x24],0x0
25a7: 74 d7 je 2580 <tts_minimal_getsomeattrs+0x2e0>
cattr->attlen);
25a9: 0f b7 5c c2 22 movzx ebx,WORD PTR [rdx+rax*8+0x22]
switch (attlen)
25ae: 66 83 fb 02 cmp bx,0x2
25b2: 0f 84 c8 00 00 00 je 2680 <tts_minimal_getsomeattrs+0x3e0>
25b8: 66 83 fb 04 cmp bx,0x4
25bc: 0f 85 76 ff ff ff jne 2538 <tts_minimal_getsomeattrs+0x298>
return (Datum) X;
25c2: 48 63 1e movsxd rbx,DWORD PTR [rsi]
} while (++attnum < firstNonCacheOffsetAttr);
25c5: 48 8d 70 01 lea rsi,[rax+0x1]
values[attnum] = fetch_att_noerr(tp + off,
25c9: 48 89 1c c7 mov QWORD PTR [rdi+rax*8],rbx
} while (++attnum < firstNonCacheOffsetAttr);
25cd: 49 39 f2 cmp r10,rsi
25d0: 75 bb jne 258d <tts_minimal_getsomeattrs+0x2ed>
25d2: 0f 1f 00 nop DWORD PTR [rax]
25d5: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
25dc: 00 00 00 00
off += cattr->attlen;
25e0: 0f bf 5c c2 22 movsx ebx,WORD PTR [rdx+rax*8+0x22]
} while (++attnum < firstNonCacheOffsetAttr);
25e5: 4c 89 d0 mov rax,r10
off += cattr->attlen;
25e8: 44 01 eb add ebx,r13d
25eb: e9 d7 fd ff ff jmp 23c7 <tts_minimal_getsomeattrs+0x127>
return (Datum) X;
25f0: 49 8b 36 mov rsi,QWORD PTR [r14]
return Int64GetDatum(*((const int64 *) offset_ptr));
25f3: e9 28 fe ff ff jmp 2420 <tts_minimal_getsomeattrs+0x180>
25f8: 0f 1f 84 00 00 00 00 nop DWORD PTR [rax+rax*1+0x0]
25ff: 00
return (Datum) X;
2600: 49 63 36 movsxd rsi,DWORD PTR [r14]
return Int32GetDatum(*((const int32 *) offset_ptr));
2603: e9 18 fe ff ff jmp 2420 <tts_minimal_getsomeattrs+0x180>
2608: 0f 1f 84 00 00 00 00 nop DWORD PTR [rax+rax*1+0x0]
260f: 00
return (Datum) X;
2610: 49 0f bf 36 movsx rsi,WORD PTR [r14]
return Int16GetDatum(*((const int16 *) offset_ptr));
2614: e9 07 fe ff ff jmp 2420 <tts_minimal_getsomeattrs+0x180>
2619: 0f 1f 80 00 00 00 00 nop DWORD PTR [rax+0x0]
if (!VARATT_IS_SHORT(tupptr + *off))
2620: 89 de mov esi,ebx
2622: 4c 01 c6 add rsi,r8
2625: f6 06 01 test BYTE PTR [rsi],0x1
2628: 0f 84 02 01 00 00 je 2730 <tts_minimal_getsomeattrs+0x490>
if (VARATT_IS_1B_E(PTR))
262e: 44 0f b6 2e movzx r13d,BYTE PTR [rsi]
return (Datum) (uintptr_t) X;
2632: 48 89 f0 mov rax,rsi
2635: 41 80 fd 01 cmp r13b,0x1
2639: 0f 84 0f 01 00 00 je 274e <tts_minimal_getsomeattrs+0x4ae>
else if (VARATT_IS_1B(PTR))
263f: 41 f6 c5 01 test r13b,0x1
2643: 0f 84 3f 01 00 00 je 2788 <tts_minimal_getsomeattrs+0x4e8>
return VARSIZE_1B(PTR);
2649: 41 d0 ed shr r13b,1
*off += VARSIZE_ANY(DatumGetPointer(res));
264c: 45 0f b6 ed movzx r13d,r13b
values[attnum] = (Datum) 0;
2650: 4a 89 04 d7 mov QWORD PTR [rdi+r10*8],rax
for (; attnum < natts; attnum++)
2654: 49 83 c2 01 add r10,0x1
2658: 44 01 eb add ebx,r13d
265b: 4d 39 ca cmp r10,r9
265e: 0f 85 4c fe ff ff jne 24b0 <tts_minimal_getsomeattrs+0x210>
2664: e9 b4 fe ff ff jmp 251d <tts_minimal_getsomeattrs+0x27d>
2669: 0f 1f 80 00 00 00 00 nop DWORD PTR [rax+0x0]
return (Datum) X;
2670: 48 0f be 1e movsx rbx,BYTE PTR [rsi]
return CharGetDatum(*((const char *) T));
2674: e9 07 ff ff ff jmp 2580 <tts_minimal_getsomeattrs+0x2e0>
2679: 0f 1f 80 00 00 00 00 nop DWORD PTR [rax+0x0]
return (Datum) X;
2680: 48 0f bf 1e movsx rbx,WORD PTR [rsi]
return Int16GetDatum(*((const int16 *) T));
2684: e9 f7 fe ff ff jmp 2580 <tts_minimal_getsomeattrs+0x2e0>
2689: 0f 1f 80 00 00 00 00 nop DWORD PTR [rax+0x0]
off += cattr->attlen;
2690: 41 0f bf 75 02 movsx esi,WORD PTR [r13+0x2]
if (attnum == reqnatts)
2695: 49 63 ec movsxd rbp,r12d
off += cattr->attlen;
2698: 48 8b 54 24 f0 mov rdx,QWORD PTR [rsp-0x10]
269d: 01 f3 add ebx,esi
if (attnum == reqnatts)
269f: 48 39 e8 cmp rax,rbp
26a2: 0f 85 0f fd ff ff jne 23b7 <tts_minimal_getsomeattrs+0x117>
*offp = off;
26a8: 41 89 5b 68 mov DWORD PTR [r11+0x68],ebx
}
26ac: 5b pop rbx
26ad: 5d pop rbp
26ae: 41 5c pop r12
26b0: 41 5d pop r13
26b2: 41 5e pop r14
26b4: 41 5f pop r15
26b6: c3 ret
26b7: 66 0f 1f 84 00 00 00 nop WORD PTR [rax+rax*1+0x0]
26be: 00 00
tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits));
26c0: 4c 8d 43 18 lea r8,[rbx+0x18]
if (reqnatts > firstNonGuaranteedAttr)
26c4: 41 39 c4 cmp r12d,eax
26c7: 0f 8e cb 00 00 00 jle 2798 <tts_minimal_getsomeattrs+0x4f8>
natts = Min(HeapTupleHeaderGetNatts(tup), reqnatts);
26cd: 0f b7 43 12 movzx eax,WORD PTR [rbx+0x12]
26d1: 25 ff 07 00 00 and eax,0x7ff
26d6: 44 39 e0 cmp eax,r12d
26d9: 41 0f 4f c4 cmovg eax,r12d
26dd: 41 89 c6 mov r14d,eax
26e0: 4c 63 c8 movsxd r9,eax
26e3: e9 36 fc ff ff jmp 231e <tts_minimal_getsomeattrs+0x7e>
26e8: 0f 1f 84 00 00 00 00 nop DWORD PTR [rax+rax*1+0x0]
26ef: 00
26f0: 31 ed xor ebp,ebp
firstNonGuaranteedAttr = 0;
26f2: 31 c0 xor eax,eax
26f4: e9 e2 fb ff ff jmp 22db <tts_minimal_getsomeattrs+0x3b>
26f9: 0f 1f 80 00 00 00 00 nop DWORD PTR [rax+0x0]
return (Datum) X;
2700: 49 0f be 45 00 movsx rax,BYTE PTR [r13+0x0]
values[attnum] = (Datum) 0;
2705: 4a 89 04 d7 mov QWORD PTR [rdi+r10*8],rax
for (; attnum < natts; attnum++)
2709: 49 83 c2 01 add r10,0x1
270d: 4d 39 ca cmp r10,r9
2710: 0f 85 9a fd ff ff jne 24b0 <tts_minimal_getsomeattrs+0x210>
2716: e9 02 fe ff ff jmp 251d <tts_minimal_getsomeattrs+0x27d>
271b: 0f 1f 44 00 00 nop DWORD PTR [rax+rax*1+0x0]
2720: 41 d0 ee shr r14b,1
*off += VARSIZE_ANY(DatumGetPointer(res));
2723: 45 0f b6 f6 movzx r14d,r14b
2727: e9 55 fd ff ff jmp 2481 <tts_minimal_getsomeattrs+0x1e1>
272c: 0f 1f 40 00 nop DWORD PTR [rax+0x0]
*off = TYPEALIGN(attalignby, *off);
2730: 8d 5c 18 ff lea ebx,[rax+rbx*1-0x1]
2734: f7 d8 neg eax
2736: 21 c3 and ebx,eax
res = PointerGetDatum(tupptr + *off);
2738: 89 de mov esi,ebx
273a: 4c 01 c6 add rsi,r8
if (VARATT_IS_1B_E(PTR))
273d: 44 0f b6 2e movzx r13d,BYTE PTR [rsi]
return (Datum) (uintptr_t) X;
2741: 48 89 f0 mov rax,rsi
2744: 41 80 fd 01 cmp r13b,0x1
2748: 0f 85 f1 fe ff ff jne 263f <tts_minimal_getsomeattrs+0x39f>
return VARTAG_1B_E(PTR);
274e: 0f b6 76 01 movzx esi,BYTE PTR [rsi+0x1]
if (tag == VARTAG_INDIRECT)
2752: 83 fe 01 cmp esi,0x1
2755: 0f 84 02 02 00 00 je 295d <tts_minimal_getsomeattrs+0x6bd>
return ((tag & ~1) == VARTAG_EXPANDED_RO);
275b: 41 89 f5 mov r13d,esi
275e: 41 83 e5 fe and r13d,0xfffffffe
else if (VARTAG_IS_EXPANDED(tag))
2762: 41 83 fd 02 cmp r13d,0x2
2766: 0f 84 f1 01 00 00 je 295d <tts_minimal_getsomeattrs+0x6bd>
else if (tag == VARTAG_ONDISK)
276c: 83 fe 12 cmp esi,0x12
276f: 40 0f 94 c6 sete sil
2773: 40 0f b6 f6 movzx esi,sil
2777: 48 c1 e6 04 shl rsi,0x4
*off += VARSIZE_ANY(DatumGetPointer(res));
277b: 44 8d 6e 02 lea r13d,[rsi+0x2]
277f: e9 cc fe ff ff jmp 2650 <tts_minimal_getsomeattrs+0x3b0>
2784: 0f 1f 40 00 nop DWORD PTR [rax+0x0]
return VARSIZE_4B(PTR);
2788: 44 8b 2e mov r13d,DWORD PTR [rsi]
278b: 41 c1 ed 02 shr r13d,0x2
278f: e9 bc fe ff ff jmp 2650 <tts_minimal_getsomeattrs+0x3b0>
2794: 0f 1f 40 00 nop DWORD PTR [rax+0x0]
natts = reqnatts;
2798: 4d 63 cc movsxd r9,r12d
279b: 45 89 e6 mov r14d,r12d
279e: e9 7b fb ff ff jmp 231e <tts_minimal_getsomeattrs+0x7e>
27a3: 0f 1f 44 00 00 nop DWORD PTR [rax+rax*1+0x0]
return (Datum) X;
27a8: 49 63 45 00 movsxd rax,DWORD PTR [r13+0x0]
values[attnum] = (Datum) 0;
27ac: 4a 89 04 d7 mov QWORD PTR [rdi+r10*8],rax
for (; attnum < natts; attnum++)
27b0: 49 83 c2 01 add r10,0x1
27b4: 4d 39 ca cmp r10,r9
27b7: 0f 85 f3 fc ff ff jne 24b0 <tts_minimal_getsomeattrs+0x210>
27bd: e9 5b fd ff ff jmp 251d <tts_minimal_getsomeattrs+0x27d>
27c2: 66 0f 1f 44 00 00 nop WORD PTR [rax+rax*1+0x0]
return (Datum) X;
27c8: 49 0f bf 45 00 movsx rax,WORD PTR [r13+0x0]
values[attnum] = (Datum) 0;
27cd: 4a 89 04 d7 mov QWORD PTR [rdi+r10*8],rax
for (; attnum < natts; attnum++)
27d1: 49 83 c2 01 add r10,0x1
27d5: 4d 39 ca cmp r10,r9
27d8: 0f 85 d2 fc ff ff jne 24b0 <tts_minimal_getsomeattrs+0x210>
27de: e9 3a fd ff ff jmp 251d <tts_minimal_getsomeattrs+0x27d>
27e3: 0f 1f 44 00 00 nop DWORD PTR [rax+rax*1+0x0]
return VARTAG_1B_E(PTR);
27e8: 45 0f b6 6d 01 movzx r13d,BYTE PTR [r13+0x1]
if (tag == VARTAG_INDIRECT)
27ed: 41 83 fd 01 cmp r13d,0x1
27f1: 0f 84 5b 01 00 00 je 2952 <tts_minimal_getsomeattrs+0x6b2>
return ((tag & ~1) == VARTAG_EXPANDED_RO);
27f7: 45 89 ee mov r14d,r13d
27fa: 41 83 e6 fe and r14d,0xfffffffe
else if (VARTAG_IS_EXPANDED(tag))
27fe: 41 83 fe 02 cmp r14d,0x2
2802: 0f 84 4a 01 00 00 je 2952 <tts_minimal_getsomeattrs+0x6b2>
else if (tag == VARTAG_ONDISK)
2808: 41 83 fd 12 cmp r13d,0x12
280c: 41 0f 94 c5 sete r13b
2810: 45 0f b6 ed movzx r13d,r13b
2814: 49 c1 e5 04 shl r13,0x4
2818: 45 8d 75 02 lea r14d,[r13+0x2]
281c: e9 60 fc ff ff jmp 2481 <tts_minimal_getsomeattrs+0x1e1>
2821: 0f 1f 80 00 00 00 00 nop DWORD PTR [rax+0x0]
int nattByte = natts >> 3;
2828: 45 89 cd mov r13d,r9d
282b: 41 c1 fd 03 sar r13d,0x3
for (bytenum = 0; bytenum < nattByte; bytenum++)
282f: 45 85 ed test r13d,r13d
2832: 0f 8e 30 01 00 00 jle 2968 <tts_minimal_getsomeattrs+0x6c8>
2838: 48 8d 73 17 lea rsi,[rbx+0x17]
283c: 31 ff xor edi,edi
283e: eb 10 jmp 2850 <tts_minimal_getsomeattrs+0x5b0>
2840: 83 c7 01 add edi,0x1
2843: 48 83 c6 01 add rsi,0x1
2847: 41 39 fd cmp r13d,edi
284a: 0f 84 ec 00 00 00 je 293c <tts_minimal_getsomeattrs+0x69c>
if (bits[bytenum] != 0xFF)
2850: 0f b6 06 movzx eax,BYTE PTR [rsi]
2853: 3c ff cmp al,0xff
2855: 74 e9 je 2840 <tts_minimal_getsomeattrs+0x5a0>
res = bytenum << 3;
2857: c1 e7 03 shl edi,0x3
res += pg_rightmost_one_pos32(~((uint32) bits[bytenum]));
285a: f7 d0 not eax
int nbytes = (natts + 7) >> 3;
285c: 45 8d 69 07 lea r13d,[r9+0x7]
2860: f3 0f bc c0 tzcnt eax,eax
res += pg_rightmost_one_pos32(~((uint32) bits[bytenum]));
2864: 01 f8 add eax,edi
res = Min(res, natts);
2866: 41 39 c1 cmp r9d,eax
2869: 41 0f 4e c1 cmovle eax,r9d
int nbytes = (natts + 7) >> 3;
286d: 41 c1 fd 03 sar r13d,0x3
res = Min(res, natts);
2871: 4c 63 f0 movsxd r14,eax
isnull_8 &= UINT64CONST(0x0101010101010101);
2874: 49 bf 01 01 01 01 01 movabs r15,0x101010101010101
287b: 01 01 01
287e: 4d 63 ed movsxd r13,r13d
for (bytenum = 0; bytenum < nattByte; bytenum++)
2881: 31 ff xor edi,edi
2883: 66 0f 1f 44 00 00 nop WORD PTR [rax+rax*1+0x0]
2889: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
2890: 00 00 00 00
2894: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
289b: 00 00 00 00
289f: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
28a6: 00 00 00 00
28aa: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
28b1: 00 00 00 00
28b5: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
28bc: 00 00 00 00
bits8 nullbyte = ~bits[i];
28c0: 0f b6 74 3b 17 movzx esi,BYTE PTR [rbx+rdi*1+0x17]
28c5: f7 d6 not esi
isnull_8 |= ((uint64) ((nullbyte >> 4) * SPREAD_BITS_MULTIPLIER_32)) << 32;
28c7: 89 f0 mov eax,esi
isnull_8 = (nullbyte & 0xf) * SPREAD_BITS_MULTIPLIER_32;
28c9: 83 e6 0f and esi,0xf
isnull_8 |= ((uint64) ((nullbyte >> 4) * SPREAD_BITS_MULTIPLIER_32)) << 32;
28cc: c0 e8 04 shr al,0x4
isnull_8 = (nullbyte & 0xf) * SPREAD_BITS_MULTIPLIER_32;
28cf: 48 69 f6 81 40 20 00 imul rsi,rsi,0x204081
isnull_8 |= ((uint64) ((nullbyte >> 4) * SPREAD_BITS_MULTIPLIER_32)) << 32;
28d6: 83 e0 0f and eax,0xf
28d9: 48 69 c0 81 40 20 00 imul rax,rax,0x204081
28e0: 48 c1 e0 20 shl rax,0x20
28e4: 48 09 f0 or rax,rsi
isnull_8 &= UINT64CONST(0x0101010101010101);
28e7: 4c 21 f8 and rax,r15
28ea: 48 89 04 f9 mov QWORD PTR [rcx+rdi*8],rax
for (int i = 0; i < nbytes; i++, isnull += 8)
28ee: 48 83 c7 01 add rdi,0x1
28f2: 4c 39 ef cmp rdi,r13
28f5: 75 c9 jne 28c0 <tts_minimal_getsomeattrs+0x620>
firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, firstNullAttr);
28f7: 45 39 f2 cmp r10d,r14d
28fa: 4d 0f 4f d6 cmovg r10,r14
28fe: e9 1b fa ff ff jmp 231e <tts_minimal_getsomeattrs+0x7e>
2903: 0f 1f 44 00 00 nop DWORD PTR [rax+rax*1+0x0]
for (; attnum < firstNullAttr; attnum++)
2908: 49 89 c2 mov r10,rax
290b: e9 90 fb ff ff jmp 24a0 <tts_minimal_getsomeattrs+0x200>
for (; attnum < natts; attnum++)
2910: 4d 89 d1 mov r9,r10
2913: e9 05 fc ff ff jmp 251d <tts_minimal_getsomeattrs+0x27d>
2918: 0f 1f 84 00 00 00 00 nop DWORD PTR [rax+rax*1+0x0]
291f: 00
*offp = off;
2920: 41 89 5b 68 mov DWORD PTR [r11+0x68],ebx
slot_getmissingattrs(slot, attnum, reqnatts);
2924: 44 89 e2 mov edx,r12d
}
2927: 5b pop rbx
slot_getmissingattrs(slot, attnum, reqnatts);
2928: 44 89 ce mov esi,r9d
}
292b: 5d pop rbp
slot_getmissingattrs(slot, attnum, reqnatts);
292c: 4c 89 df mov rdi,r11
}
292f: 41 5c pop r12
2931: 41 5d pop r13
2933: 41 5e pop r14
2935: 41 5f pop r15
slot_getmissingattrs(slot, attnum, reqnatts);
2937: e9 b4 f1 ff ff jmp 1af0 <slot_getmissingattrs>
res = bytenum << 3;
293c: 42 8d 3c ed 00 00 00 lea edi,[r13*8+0x0]
2943: 00
res += pg_rightmost_one_pos32(~((uint32) bits[bytenum]));
2944: 4d 63 ed movsxd r13,r13d
2947: 42 0f b6 44 2b 17 movzx eax,BYTE PTR [rbx+r13*1+0x17]
294d: e9 08 ff ff ff jmp 285a <tts_minimal_getsomeattrs+0x5ba>
2952: 41 be 0a 00 00 00 mov r14d,0xa
2958: e9 24 fb ff ff jmp 2481 <tts_minimal_getsomeattrs+0x1e1>
295d: 41 bd 0a 00 00 00 mov r13d,0xa
2963: e9 e8 fc ff ff jmp 2650 <tts_minimal_getsomeattrs+0x3b0>
2968: 0f b6 43 17 movzx eax,BYTE PTR [rbx+0x17]
int nbytes = (natts + 7) >> 3;
296c: 45 8d 69 07 lea r13d,[r9+0x7]
res += pg_rightmost_one_pos32(~((uint32) bits[bytenum]));
2970: f7 d0 not eax
2972: f3 0f bc c0 tzcnt eax,eax
res = Min(res, natts);
2976: 41 39 c1 cmp r9d,eax
2979: 41 0f 4e c1 cmovle eax,r9d
int nbytes = (natts + 7) >> 3;
297d: 41 c1 fd 03 sar r13d,0x3
res = Min(res, natts);
2981: 4c 63 f0 movsxd r14,eax
for (int i = 0; i < nbytes; i++, isnull += 8)
2984: 41 83 fd 01 cmp r13d,0x1
2988: 0f 85 69 ff ff ff jne 28f7 <tts_minimal_getsomeattrs+0x657>
298e: e9 e1 fe ff ff jmp 2874 <tts_minimal_getsomeattrs+0x5d4>
2993: 66 90 xchg ax,ax
2995: 66 66 2e 0f 1f 84 00 data16 cs nop WORD PTR [rax+rax*1+0x0]
299c: 00 00 00 00
[text/plain] v11-0001-Introduce-deform_bench-test-module.patch (7.3K, 3-v11-0001-Introduce-deform_bench-test-module.patch)
download | inline diff:
From 46c83290a6ed1256cbefd9fa62de808424601d70 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Tue, 27 Jan 2026 15:08:09 +1300
Subject: [PATCH v11 1/5] Introduce deform_bench test module
For benchmarking tuple deformation.
---
src/test/modules/deform_bench/.gitignore | 4 +
src/test/modules/deform_bench/Makefile | 21 ++++
.../deform_bench/deform_bench--1.0.sql | 8 ++
src/test/modules/deform_bench/deform_bench.c | 107 ++++++++++++++++++
.../modules/deform_bench/deform_bench.control | 4 +
src/test/modules/deform_bench/meson.build | 22 ++++
src/test/modules/meson.build | 1 +
7 files changed, 167 insertions(+)
create mode 100644 src/test/modules/deform_bench/.gitignore
create mode 100644 src/test/modules/deform_bench/Makefile
create mode 100644 src/test/modules/deform_bench/deform_bench--1.0.sql
create mode 100644 src/test/modules/deform_bench/deform_bench.c
create mode 100644 src/test/modules/deform_bench/deform_bench.control
create mode 100644 src/test/modules/deform_bench/meson.build
diff --git a/src/test/modules/deform_bench/.gitignore b/src/test/modules/deform_bench/.gitignore
new file mode 100644
index 00000000000..5dcb3ff9723
--- /dev/null
+++ b/src/test/modules/deform_bench/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/src/test/modules/deform_bench/Makefile b/src/test/modules/deform_bench/Makefile
new file mode 100644
index 00000000000..b5fc0f7a583
--- /dev/null
+++ b/src/test/modules/deform_bench/Makefile
@@ -0,0 +1,21 @@
+# src/test/modules/deform_bench/Makefile
+
+MODULE_big = deform_bench
+OBJS = deform_bench.o
+
+EXTENSION = deform_bench
+DATA = deform_bench--1.0.sql
+PGFILEDESC = "deform_bench - tuple deform benchmarking"
+
+REGRESS = deform_bench
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = src/test/modules/deform_bench
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/src/test/modules/deform_bench/deform_bench--1.0.sql b/src/test/modules/deform_bench/deform_bench--1.0.sql
new file mode 100644
index 00000000000..492b71dba3b
--- /dev/null
+++ b/src/test/modules/deform_bench/deform_bench--1.0.sql
@@ -0,0 +1,8 @@
+/* deform_bench--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION deform_bench" to load this file. \quit
+
+CREATE FUNCTION deform_bench(tableoid Oid, attnum int[]) RETURNS FLOAT
+AS 'MODULE_PATHNAME', 'deform_bench'
+LANGUAGE C VOLATILE STRICT;
diff --git a/src/test/modules/deform_bench/deform_bench.c b/src/test/modules/deform_bench/deform_bench.c
new file mode 100644
index 00000000000..7838f639bef
--- /dev/null
+++ b/src/test/modules/deform_bench/deform_bench.c
@@ -0,0 +1,107 @@
+/*-------------------------------------------------------------------------
+ *
+ * deform_bench.c
+ *
+ * for benchmarking tuple deformation routines
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include <time.h>
+#include <sys/time.h>
+
+#include "access/heapam.h"
+#include "access/relscan.h"
+#include "catalog/pg_am_d.h"
+#include "catalog/pg_type_d.h"
+#include "funcapi.h"
+#include "miscadmin.h"
+#include "utils/array.h"
+#include "utils/arrayaccess.h"
+#include "utils/builtins.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(deform_bench);
+
+Datum
+deform_bench(PG_FUNCTION_ARGS)
+{
+ Oid tableoid = PG_GETARG_OID(0);
+ ArrayType *array = PG_GETARG_ARRAYTYPE_P(1);
+ TableScanDesc scan;
+ Relation rel;
+ TupleDesc tupdesc;
+ TupleTableSlot *slot;
+ Datum *elem_datums = NULL;
+ bool *elem_nulls = NULL;
+ int elem_count;
+ int *attnums;
+ clock_t start,
+ end;
+
+ rel = relation_open(tableoid, AccessShareLock);
+
+ if (rel->rd_rel->relam != HEAP_TABLE_AM_OID)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("only heap AM is supported")));
+
+ tupdesc = RelationGetDescr(rel);
+ slot = MakeTupleTableSlot(tupdesc, &TTSOpsBufferHeapTuple);
+ scan = table_beginscan_strat(rel, GetActiveSnapshot(), 0, NULL, true, false);
+
+ /*
+ * The array is used to allow callers to define how many atts to deform.
+ * e.g: '{1,10}'::int[] would deform attnum=1, then in a 2nd pass deform
+ * the remainder up to attnum=10. Passing an element as NULL means all
+ * attnums. This allows simulation of incremental deformation. Generally
+ * if you're passing an array with more than 1 element, then the array
+ * should be in ascending order. Doing something like '{10,1}' would mean
+ * we've already deformed 10 attributes and on the 2nd pass there's
+ * nothing to do since attnum=1 was already deformed in the first pass.
+ *
+ * You'll get an ERROR if you pass a number higher than the number of
+ * attributes in the table.
+ */
+ deconstruct_array(array,
+ INT4OID,
+ sizeof(int32),
+ true,
+ 'i',
+ &elem_datums,
+ &elem_nulls,
+ &elem_count);
+
+ attnums = palloc_array(int, elem_count);
+
+ for (int i = 0; i < elem_count; i++)
+ {
+ /* Make a NULL element mean all attributes */
+ if (elem_nulls[i])
+ attnums[i] = tupdesc->natts;
+ else
+ attnums[i] = DatumGetInt32(elem_datums[i]);
+ }
+
+ start = clock();
+
+ while (heap_getnextslot(scan, ForwardScanDirection, slot))
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ /* Deform in stages according to the attnums array */
+ for (int i = 0; i < elem_count; i++)
+ slot_getsomeattrs(slot, attnums[i]);
+ }
+
+ end = clock();
+
+ ExecDropSingleTupleTableSlot(slot);
+ table_endscan(scan);
+ relation_close(rel, AccessShareLock);
+
+
+ /* Returns the number of milliseconds to run the test */
+ PG_RETURN_FLOAT8((double) (end - start) / (CLOCKS_PER_SEC / 1000));
+}
diff --git a/src/test/modules/deform_bench/deform_bench.control b/src/test/modules/deform_bench/deform_bench.control
new file mode 100644
index 00000000000..a2023f9d738
--- /dev/null
+++ b/src/test/modules/deform_bench/deform_bench.control
@@ -0,0 +1,4 @@
+# deform_bench extension
+comment = 'functions for benchmarking tuple deformation'
+default_version = '1.0'
+module_pathname = '$libdir/deform_bench'
diff --git a/src/test/modules/deform_bench/meson.build b/src/test/modules/deform_bench/meson.build
new file mode 100644
index 00000000000..82049585244
--- /dev/null
+++ b/src/test/modules/deform_bench/meson.build
@@ -0,0 +1,22 @@
+# Copyright (c) 2026, PostgreSQL Global Development Group
+
+deform_bench_sources = files(
+ 'deform_bench.c',
+)
+
+if host_system == 'windows'
+ deform_bench_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+ '--NAME', 'deform_bench',
+ '--FILEDESC', 'deform_bench - benchmarking tuple deformation',])
+endif
+
+deform_bench = shared_module('deform_bench',
+ deform_bench_sources,
+ kwargs: pg_test_mod_args,
+)
+test_install_libs += deform_bench
+
+test_install_data += files(
+ 'deform_bench--1.0.sql',
+ 'deform_bench.control',
+)
diff --git a/src/test/modules/meson.build b/src/test/modules/meson.build
index 2634a519935..ef2b0af4581 100644
--- a/src/test/modules/meson.build
+++ b/src/test/modules/meson.build
@@ -2,6 +2,7 @@
subdir('brin')
subdir('commit_ts')
+subdir('deform_bench')
subdir('delay_execution')
subdir('dummy_index_am')
subdir('dummy_seclabel')
--
2.51.0
[text/plain] v11-0002-Allow-sibling-call-optimization-in-slot_getsomea.patch (7.3K, 4-v11-0002-Allow-sibling-call-optimization-in-slot_getsomea.patch)
download | inline diff:
From 5d372c316557406e319b26dcf381d896aecea226 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Mon, 16 Feb 2026 14:20:19 +1300
Subject: [PATCH v11 2/5] Allow sibling call optimization in
slot_getsomeattrs_int()
This changes the TupleTableSlotOps contract to make it so the
getsomeattrs() function is in charge of calling
slot_getmissingattrs().
Since this removes all code from slot_getsomeattrs_int() aside from the
getsomeattrs() call itself, we may as well adjust slot_getsomeattrs() so
that it calls getsomeattrs() directly. We leave slot_getsomeattrs_int()
intact as this is still called from the JIT code.
---
src/backend/executor/execTuples.c | 57 ++++++++++++++++---------------
src/include/executor/tuptable.h | 13 ++++---
2 files changed, 37 insertions(+), 33 deletions(-)
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index b768eae9e53..5b9bb21fa7b 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -73,7 +73,7 @@
static TupleDesc ExecTypeFromTLInternal(List *targetList,
bool skipjunk);
static pg_attribute_always_inline void slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
- int natts);
+ int reqnatts);
static inline void tts_buffer_heap_store_tuple(TupleTableSlot *slot,
HeapTuple tuple,
Buffer buffer,
@@ -1108,7 +1108,10 @@ slot_deform_heap_tuple_internal(TupleTableSlot *slot, HeapTuple tuple,
* slot_deform_heap_tuple
* Given a TupleTableSlot, extract data from the slot's physical tuple
* into its Datum/isnull arrays. Data is extracted up through the
- * natts'th column (caller must ensure this is a legal column number).
+ * reqnatts'th column. If there are insufficient attributes in the given
+ * tuple, then slot_getmissingattrs() is called to populate the
+ * remainder. If reqnatts is above the number of attributes in the
+ * slot's TupleDesc, an error is raised.
*
* This is essentially an incremental version of heap_deform_tuple:
* on each call we extract attributes up to the one needed, without
@@ -1120,7 +1123,7 @@ slot_deform_heap_tuple_internal(TupleTableSlot *slot, HeapTuple tuple,
*/
static pg_attribute_always_inline void
slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
- int natts)
+ int reqnatts)
{
bool hasnulls = HeapTupleHasNulls(tuple);
int attnum;
@@ -1128,13 +1131,14 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
bool slow; /* can we use/set attcacheoff? */
/* We can only fetch as many attributes as the tuple has. */
- natts = Min(HeapTupleHeaderGetNatts(tuple->t_data), natts);
+ natts = Min(HeapTupleHeaderGetNatts(tuple->t_data), reqnatts);
/*
* Check whether the first call for this tuple, and initialize or restore
* loop state.
*/
attnum = slot->tts_nvalid;
+ slot->tts_nvalid = reqnatts;
if (attnum == 0)
{
/* Start from the first attribute */
@@ -1199,12 +1203,15 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
/*
* Save state for next execution
*/
- slot->tts_nvalid = attnum;
*offp = off;
if (slow)
slot->tts_flags |= TTS_FLAG_SLOW;
else
slot->tts_flags &= ~TTS_FLAG_SLOW;
+
+ /* Fetch any missing attrs and raise an error if reqnatts is invalid. */
+ if (unlikely(attnum < reqnatts))
+ slot_getmissingattrs(slot, attnum, reqnatts);
}
const TupleTableSlotOps TTSOpsVirtual = {
@@ -2058,34 +2065,36 @@ slot_getmissingattrs(TupleTableSlot *slot, int startAttNum, int lastAttNum)
{
AttrMissing *attrmiss = NULL;
+ /* Check for invalid attnums */
+ if (unlikely(lastAttNum > slot->tts_tupleDescriptor->natts))
+ elog(ERROR, "invalid attribute number %d", lastAttNum);
+
if (slot->tts_tupleDescriptor->constr)
attrmiss = slot->tts_tupleDescriptor->constr->missing;
if (!attrmiss)
{
/* no missing values array at all, so just fill everything in as NULL */
- memset(slot->tts_values + startAttNum, 0,
- (lastAttNum - startAttNum) * sizeof(Datum));
- memset(slot->tts_isnull + startAttNum, 1,
- (lastAttNum - startAttNum) * sizeof(bool));
+ for (int attnum = startAttNum; attnum < lastAttNum; attnum++)
+ {
+ slot->tts_values[attnum] = (Datum) 0;
+ slot->tts_isnull[attnum] = true;
+ }
}
else
{
- int missattnum;
-
- /* if there is a missing values array we must process them one by one */
- for (missattnum = startAttNum;
- missattnum < lastAttNum;
- missattnum++)
+ /* use attrmiss to set the missing values */
+ for (int attnum = startAttNum; attnum < lastAttNum; attnum++)
{
- slot->tts_values[missattnum] = attrmiss[missattnum].am_value;
- slot->tts_isnull[missattnum] = !attrmiss[missattnum].am_present;
+ slot->tts_values[attnum] = attrmiss[attnum].am_value;
+ slot->tts_isnull[attnum] = !attrmiss[attnum].am_present;
}
}
}
/*
- * slot_getsomeattrs_int - workhorse for slot_getsomeattrs()
+ * slot_getsomeattrs_int
+ * external function to call getsomeattrs() for use in JIT
*/
void
slot_getsomeattrs_int(TupleTableSlot *slot, int attnum)
@@ -2094,21 +2103,13 @@ slot_getsomeattrs_int(TupleTableSlot *slot, int attnum)
Assert(slot->tts_nvalid < attnum); /* checked in slot_getsomeattrs */
Assert(attnum > 0);
- if (unlikely(attnum > slot->tts_tupleDescriptor->natts))
- elog(ERROR, "invalid attribute number %d", attnum);
-
/* Fetch as many attributes as possible from the underlying tuple. */
slot->tts_ops->getsomeattrs(slot, attnum);
/*
- * If the underlying tuple doesn't have enough attributes, tuple
- * descriptor must have the missing attributes.
+ * Avoid putting new code here as that would prevent the compiler from
+ * using the sibling call optimization for the above function.
*/
- if (unlikely(slot->tts_nvalid < attnum))
- {
- slot_getmissingattrs(slot, slot->tts_nvalid, attnum);
- slot->tts_nvalid = attnum;
- }
}
/* ----------------------------------------------------------------
diff --git a/src/include/executor/tuptable.h b/src/include/executor/tuptable.h
index a2dfd707e78..3b09abbf99f 100644
--- a/src/include/executor/tuptable.h
+++ b/src/include/executor/tuptable.h
@@ -151,10 +151,12 @@ struct TupleTableSlotOps
/*
* Fill up first natts entries of tts_values and tts_isnull arrays with
- * values from the tuple contained in the slot. The function may be called
- * with natts more than the number of attributes available in the tuple,
- * in which case it should set tts_nvalid to the number of returned
- * columns.
+ * values from the tuple contained in the slot and set the slot's
+ * tts_nvalid to natts. The function may be called with an natts value
+ * more than the number of attributes available in the tuple, in which
+ * case the function must call slot_getmissingattrs() to populate the
+ * remaining attributes. The function must raise an ERROR if 'natts' is
+ * higher than the number of attributes in the slot's TupleDesc.
*/
void (*getsomeattrs) (TupleTableSlot *slot, int natts);
@@ -357,8 +359,9 @@ extern void slot_getsomeattrs_int(TupleTableSlot *slot, int attnum);
static inline void
slot_getsomeattrs(TupleTableSlot *slot, int attnum)
{
+ /* Populate slot with attributes up to 'attnum', if it's not already */
if (slot->tts_nvalid < attnum)
- slot_getsomeattrs_int(slot, attnum);
+ slot->tts_ops->getsomeattrs(slot, attnum);
}
/*
--
2.51.0
[text/plain] v11-0005-Reduce-size-of-CompactAttribute-struct-to-8-byte.patch (5.5K, 5-v11-0005-Reduce-size-of-CompactAttribute-struct-to-8-byte.patch)
download | inline diff:
From 099a6186e1886432ed24653178ab1ce9113900c9 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Mon, 23 Feb 2026 09:39:37 +1300
Subject: [PATCH v11 5/5] Reduce size of CompactAttribute struct to 8 bytes
Previously, this was 16 bytes. With the use of some bitflags and by
reducing the attcacheoff field size to a 16-bit type, we can halve the
size of the struct.
It's unlikely that caching the offsets for offsets larger than what will
fit in a 16-bit int will help much as the tuple is very likely to have
some non-fixed-width types anyway, the offsets of which we cannot cache.
---
src/backend/access/common/tupdesc.c | 10 ++++++++++
src/backend/executor/execTuples.c | 16 ++++++++++++----
src/include/access/tupdesc.h | 16 ++++++++--------
3 files changed, 30 insertions(+), 12 deletions(-)
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index c68561337d7..71461ba6096 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -530,6 +530,16 @@ TupleDescFinalize(TupleDesc tupdesc)
off = att_nominal_alignby(off, cattr->attalignby);
+ /*
+ * attcacheoff is an int16, so don't try to cache any offsets larger
+ * than will fit in that type. Any attributes which are offset more
+ * than 2^15 are likely due to variable-length attributes. Since we
+ * don't cache offsets for or beyond variable-length attributes, using
+ * an int16 rather than an int32 here is unlikely to cost us anything.
+ */
+ if (off > PG_INT16_MAX)
+ break;
+
cattr->attcacheoff = off;
off += cattr->attlen;
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 83a8c02894d..345b22ca932 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -1013,6 +1013,7 @@ static pg_attribute_always_inline void
slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int reqnatts)
{
+ CompactAttribute *cattrs;
CompactAttribute *cattr;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
HeapTupleHeader tup = tuple->t_data;
@@ -1101,6 +1102,13 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
values = slot->tts_values;
slot->tts_nvalid = reqnatts;
+ /*
+ * We store the tupleDesc's CompactAttribute array in 'cattrs' as gcc
+ * seems to be unwilling to optimize accessing the CompactAttribute
+ * element efficiently when accessing it via TupleDescCompactAttr().
+ */
+ cattrs = tupleDesc->compact_attrs;
+
/* Ensure we calculated tp correctly */
Assert(tp == (char *) tup + tup->t_hoff);
@@ -1111,7 +1119,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int attlen;
isnull[attnum] = false;
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
attlen = cattr->attlen;
/* We don't expect any non-byval types */
@@ -1156,7 +1164,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
do
{
isnull[attnum] = false;
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
off = cattr->attcacheoff;
values[attnum] = fetch_att_noerr(tp + off,
@@ -1183,7 +1191,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int attlen;
isnull[attnum] = false;
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
attlen = cattr->attlen;
/*
@@ -1216,7 +1224,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
continue;
}
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
attlen = cattr->attlen;
/* As above, we don't expect cstrings */
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index ad7bc013812..e98036b58bf 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -55,7 +55,7 @@ typedef struct TupleConstr
* directly after the FormData_pg_attribute struct is populated or
* altered in any way.
*
- * Currently, this struct is 16 bytes. Any code changes which enlarge this
+ * Currently, this struct is 8 bytes. Any code changes which enlarge this
* struct should be considered very carefully.
*
* Code which must access a TupleDesc's attribute data should always make use
@@ -67,17 +67,17 @@ typedef struct TupleConstr
*/
typedef struct CompactAttribute
{
- int32 attcacheoff; /* fixed offset into tuple, if known, or -1 */
+ int16 attcacheoff; /* fixed offset into tuple, if known, or -1 */
int16 attlen; /* attr len in bytes or -1 = varlen, -2 =
* cstring */
bool attbyval; /* as FormData_pg_attribute.attbyval */
- bool attispackable; /* FormData_pg_attribute.attstorage !=
- * TYPSTORAGE_PLAIN */
- bool atthasmissing; /* as FormData_pg_attribute.atthasmissing */
- bool attisdropped; /* as FormData_pg_attribute.attisdropped */
- bool attgenerated; /* FormData_pg_attribute.attgenerated != '\0' */
- char attnullability; /* status of not-null constraint, see below */
uint8 attalignby; /* alignment requirement in bytes */
+ bool attispackable:1; /* FormData_pg_attribute.attstorage !=
+ * TYPSTORAGE_PLAIN */
+ bool atthasmissing:1; /* as FormData_pg_attribute.atthasmissing */
+ bool attisdropped:1; /* as FormData_pg_attribute.attisdropped */
+ bool attgenerated:1; /* FormData_pg_attribute.attgenerated != '\0' */
+ char attnullability; /* status of not-null constraint, see below */
} CompactAttribute;
/* Valid values for CompactAttribute->attnullability */
--
2.51.0
[text/plain] v11-0003-Add-empty-TupleDescFinalize-function.patch (29.0K, 6-v11-0003-Add-empty-TupleDescFinalize-function.patch)
download | inline diff:
From 3fa14f2411303b5433dd2e3434c840a77395e213 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Wed, 21 Jan 2026 15:41:37 +1300
Subject: [PATCH v11 3/5] Add empty TupleDescFinalize() function
Currently does nothing, but will in a future commit.
---
contrib/dblink/dblink.c | 4 ++++
contrib/pg_buffercache/pg_buffercache_pages.c | 2 ++
contrib/pg_visibility/pg_visibility.c | 2 ++
src/backend/access/brin/brin_tuple.c | 1 +
src/backend/access/common/tupdesc.c | 13 +++++++++++++
src/backend/access/gin/ginutil.c | 1 +
src/backend/access/gist/gistscan.c | 1 +
src/backend/access/spgist/spgutils.c | 1 +
src/backend/access/transam/twophase.c | 1 +
src/backend/access/transam/xlogfuncs.c | 1 +
src/backend/backup/basebackup_copy.c | 3 +++
src/backend/catalog/index.c | 2 ++
src/backend/catalog/pg_publication.c | 1 +
src/backend/catalog/toasting.c | 6 ++++++
src/backend/commands/explain.c | 1 +
src/backend/commands/functioncmds.c | 1 +
src/backend/commands/sequence.c | 1 +
src/backend/commands/tablecmds.c | 4 ++++
src/backend/commands/wait.c | 1 +
src/backend/executor/execSRF.c | 2 ++
src/backend/executor/execTuples.c | 4 ++++
src/backend/executor/nodeFunctionscan.c | 2 ++
src/backend/parser/parse_relation.c | 4 +++-
src/backend/parser/parse_target.c | 2 ++
.../replication/libpqwalreceiver/libpqwalreceiver.c | 1 +
src/backend/replication/walsender.c | 5 +++++
src/backend/utils/adt/acl.c | 1 +
src/backend/utils/adt/genfile.c | 1 +
src/backend/utils/adt/lockfuncs.c | 1 +
src/backend/utils/adt/orderedsetaggs.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 5 +++++
src/backend/utils/adt/tsvector_op.c | 1 +
src/backend/utils/cache/relcache.c | 8 ++++++++
src/backend/utils/fmgr/funcapi.c | 6 ++++++
src/backend/utils/misc/guc_funcs.c | 5 +++++
src/include/access/tupdesc.h | 1 +
src/pl/plpgsql/src/pl_comp.c | 2 ++
.../test_custom_stats/test_custom_fixed_stats.c | 1 +
src/test/modules/test_predtest/test_predtest.c | 1 +
39 files changed, 100 insertions(+), 1 deletion(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 2498d80c8e7..4038950a6ef 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -881,6 +881,7 @@ materializeResult(FunctionCallInfo fcinfo, PGconn *conn, PGresult *res)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "status",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
ntuples = 1;
nfields = 1;
}
@@ -1044,6 +1045,7 @@ materializeQueryResult(FunctionCallInfo fcinfo,
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "status",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
attinmeta = TupleDescGetAttInMetadata(tupdesc);
oldcontext = MemoryContextSwitchTo(rsinfo->econtext->ecxt_per_query_memory);
@@ -1529,6 +1531,8 @@ dblink_get_pkey(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 2, "colname",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+
/*
* Generate attribute metadata needed later to produce tuples from raw
* C strings
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 89b86855243..a6b4fb5252b 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -174,6 +174,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
INT4OID, -1, 0);
+ TupleDescFinalize(tupledesc);
fctx->tupdesc = BlessTupleDesc(tupledesc);
/* Allocate NBuffers worth of BufferCachePagesRec records. */
@@ -442,6 +443,7 @@ pg_buffercache_os_pages_internal(FunctionCallInfo fcinfo, bool include_numa)
TupleDescInitEntry(tupledesc, (AttrNumber) 3, "numa_node",
INT4OID, -1, 0);
+ TupleDescFinalize(tupledesc);
fctx->tupdesc = BlessTupleDesc(tupledesc);
fctx->include_numa = include_numa;
diff --git a/contrib/pg_visibility/pg_visibility.c b/contrib/pg_visibility/pg_visibility.c
index 9bc3a784bf7..dfab0b64cf5 100644
--- a/contrib/pg_visibility/pg_visibility.c
+++ b/contrib/pg_visibility/pg_visibility.c
@@ -469,6 +469,8 @@ pg_visibility_tupdesc(bool include_blkno, bool include_pd)
TupleDescInitEntry(tupdesc, ++a, "pd_all_visible", BOOLOID, -1, 0);
Assert(a == maxattr);
+ TupleDescFinalize(tupdesc);
+
return BlessTupleDesc(tupdesc);
}
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 69c233c62eb..742ac089a28 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -84,6 +84,7 @@ brtuple_disk_tupdesc(BrinDesc *brdesc)
MemoryContextSwitchTo(oldcxt);
+ TupleDescFinalize(tupdesc);
brdesc->bd_disktdesc = tupdesc;
}
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index b69d10f0a45..2137385a833 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -221,6 +221,9 @@ CreateTupleDesc(int natts, Form_pg_attribute *attrs)
memcpy(TupleDescAttr(desc, i), attrs[i], ATTRIBUTE_FIXED_PART_SIZE);
populate_compact_attribute(desc, i);
}
+
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -265,6 +268,8 @@ CreateTupleDescCopy(TupleDesc tupdesc)
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -311,6 +316,8 @@ CreateTupleDescTruncatedCopy(TupleDesc tupdesc, int natts)
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -396,6 +403,8 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -438,6 +447,8 @@ TupleDescCopy(TupleDesc dst, TupleDesc src)
* source's refcount would be wrong in any case.)
*/
dst->tdrefcount = -1;
+
+ TupleDescFinalize(dst);
}
/*
@@ -1065,6 +1076,8 @@ BuildDescFromLists(const List *names, const List *types, const List *typmods, co
TupleDescInitEntryCollation(desc, attnum, attcollation);
}
+ TupleDescFinalize(desc);
+
return desc;
}
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index ff927279cc3..fe7b984ff32 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -129,6 +129,7 @@ initGinState(GinState *state, Relation index)
attr->attndims);
TupleDescInitEntryCollation(state->tupdesc[i], (AttrNumber) 2,
attr->attcollation);
+ TupleDescFinalize(state->tupdesc[i]);
}
/*
diff --git a/src/backend/access/gist/gistscan.c b/src/backend/access/gist/gistscan.c
index f23bc4a6757..c65f93abdae 100644
--- a/src/backend/access/gist/gistscan.c
+++ b/src/backend/access/gist/gistscan.c
@@ -201,6 +201,7 @@ gistrescan(IndexScanDesc scan, ScanKey key, int nkeys,
attno - 1)->atttypid,
-1, 0);
}
+ TupleDescFinalize(so->giststate->fetchTupdesc);
scan->xs_hitupdesc = so->giststate->fetchTupdesc;
/* Also create a memory context that will hold the returned tuples */
diff --git a/src/backend/access/spgist/spgutils.c b/src/backend/access/spgist/spgutils.c
index 9f5379b87ac..b246e8127db 100644
--- a/src/backend/access/spgist/spgutils.c
+++ b/src/backend/access/spgist/spgutils.c
@@ -340,6 +340,7 @@ getSpGistTupleDesc(Relation index, SpGistTypeDesc *keyType)
TupleDescCompactAttr(outTupDesc, i)->attcacheoff = -1;
populate_compact_attribute(outTupDesc, spgKeyColumn);
+ TupleDescFinalize(outTupDesc);
}
return outTupDesc;
}
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index e4340b59640..7f4ed02a6b9 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -744,6 +744,7 @@ pg_prepared_xact(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 5, "dbid",
OIDOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
/*
diff --git a/src/backend/access/transam/xlogfuncs.c b/src/backend/access/transam/xlogfuncs.c
index 2efe4105efb..b6bc616c74c 100644
--- a/src/backend/access/transam/xlogfuncs.c
+++ b/src/backend/access/transam/xlogfuncs.c
@@ -400,6 +400,7 @@ pg_walfile_name_offset(PG_FUNCTION_ARGS)
TupleDescInitEntry(resultTupleDesc, (AttrNumber) 2, "file_offset",
INT4OID, -1, 0);
+ TupleDescFinalize(resultTupleDesc);
resultTupleDesc = BlessTupleDesc(resultTupleDesc);
/*
diff --git a/src/backend/backup/basebackup_copy.c b/src/backend/backup/basebackup_copy.c
index 07f58b39d8c..6c3453efd80 100644
--- a/src/backend/backup/basebackup_copy.c
+++ b/src/backend/backup/basebackup_copy.c
@@ -357,6 +357,8 @@ SendXlogRecPtrResult(XLogRecPtr ptr, TimeLineID tli)
*/
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "tli", INT8OID, -1, 0);
+ TupleDescFinalize(tupdesc);
+
/* send RowDescription */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -388,6 +390,7 @@ SendTablespaceList(List *tablespaces)
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 1, "spcoid", OIDOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "spclocation", TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 3, "size", INT8OID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* send RowDescription */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 43de42ce39e..75e97fb394a 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -481,6 +481,8 @@ ConstructTupleDescriptor(Relation heapRelation,
populate_compact_attribute(indexTupDesc, i);
}
+ TupleDescFinalize(indexTupDesc);
+
return indexTupDesc;
}
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index 9a4791c573e..fa353a0dd37 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -1230,6 +1230,7 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 4, "qual",
PG_NODE_TREEOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
funcctx->user_fctx = table_infos;
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index c78dcea98c1..078a1cf5127 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -229,6 +229,12 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attcompression = InvalidCompressionMethod;
TupleDescAttr(tupdesc, 2)->attcompression = InvalidCompressionMethod;
+ populate_compact_attribute(tupdesc, 0);
+ populate_compact_attribute(tupdesc, 1);
+ populate_compact_attribute(tupdesc, 2);
+
+ TupleDescFinalize(tupdesc);
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 93918a223b8..5f922c3f5c2 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -281,6 +281,7 @@ ExplainResultDesc(ExplainStmt *stmt)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "QUERY PLAN",
result_type, -1, 0);
+ TupleDescFinalize(tupdesc);
return tupdesc;
}
diff --git a/src/backend/commands/functioncmds.c b/src/backend/commands/functioncmds.c
index 242372b1e68..3afd762e9dc 100644
--- a/src/backend/commands/functioncmds.c
+++ b/src/backend/commands/functioncmds.c
@@ -2424,6 +2424,7 @@ CallStmtResultDesc(CallStmt *stmt)
-1,
0);
}
+ TupleDescFinalize(tupdesc);
}
return tupdesc;
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index e1b808bbb60..551667650ba 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -1808,6 +1808,7 @@ pg_get_sequence_data(PG_FUNCTION_ARGS)
BOOLOID, -1, 0);
TupleDescInitEntry(resultTupleDesc, (AttrNumber) 3, "page_lsn",
LSNOID, -1, 0);
+ TupleDescFinalize(resultTupleDesc);
resultTupleDesc = BlessTupleDesc(resultTupleDesc);
seqrel = try_relation_open(relid, AccessShareLock);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index b04b0dbd2a0..8678cecd53f 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -1030,6 +1030,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
}
}
+ TupleDescFinalize(descriptor);
+
/*
* For relations with table AM and partitioned tables, select access
* method to use: an explicitly indicated one, or (in the case of a
@@ -1458,6 +1460,8 @@ BuildDescForRelation(const List *columns)
populate_compact_attribute(desc, attnum - 1);
}
+ TupleDescFinalize(desc);
+
return desc;
}
diff --git a/src/backend/commands/wait.c b/src/backend/commands/wait.c
index 1290df10c6f..8e920a72372 100644
--- a/src/backend/commands/wait.c
+++ b/src/backend/commands/wait.c
@@ -338,5 +338,6 @@ WaitStmtResultDesc(WaitStmt *stmt)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "status",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
return tupdesc;
}
diff --git a/src/backend/executor/execSRF.c b/src/backend/executor/execSRF.c
index a0b111dc0e4..b481e50acfb 100644
--- a/src/backend/executor/execSRF.c
+++ b/src/backend/executor/execSRF.c
@@ -272,6 +272,7 @@ ExecMakeTableFunctionResult(SetExprState *setexpr,
funcrettype,
-1,
0);
+ TupleDescFinalize(tupdesc);
rsinfo.setDesc = tupdesc;
}
MemoryContextSwitchTo(oldcontext);
@@ -776,6 +777,7 @@ init_sexpr(Oid foid, Oid input_collation, Expr *node,
funcrettype,
-1,
0);
+ TupleDescFinalize(tupdesc);
sexpr->funcResultDesc = tupdesc;
sexpr->funcReturnsTuple = false;
}
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 5b9bb21fa7b..bb997182481 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -2174,6 +2174,8 @@ ExecTypeFromTLInternal(List *targetList, bool skipjunk)
cur_resno++;
}
+ TupleDescFinalize(typeInfo);
+
return typeInfo;
}
@@ -2208,6 +2210,8 @@ ExecTypeFromExprList(List *exprList)
cur_resno++;
}
+ TupleDescFinalize(typeInfo);
+
return typeInfo;
}
diff --git a/src/backend/executor/nodeFunctionscan.c b/src/backend/executor/nodeFunctionscan.c
index 63e605e1f81..feb82d64967 100644
--- a/src/backend/executor/nodeFunctionscan.c
+++ b/src/backend/executor/nodeFunctionscan.c
@@ -414,6 +414,7 @@ ExecInitFunctionScan(FunctionScan *node, EState *estate, int eflags)
TupleDescInitEntryCollation(tupdesc,
(AttrNumber) 1,
exprCollation(funcexpr));
+ TupleDescFinalize(tupdesc);
}
else
{
@@ -485,6 +486,7 @@ ExecInitFunctionScan(FunctionScan *node, EState *estate, int eflags)
0);
}
+ TupleDescFinalize(scan_tupdesc);
Assert(attno == natts);
}
diff --git a/src/backend/parser/parse_relation.c b/src/backend/parser/parse_relation.c
index e003db520de..9c415e166ee 100644
--- a/src/backend/parser/parse_relation.c
+++ b/src/backend/parser/parse_relation.c
@@ -1883,6 +1883,7 @@ addRangeTableEntryForFunction(ParseState *pstate,
TupleDescInitEntryCollation(tupdesc,
(AttrNumber) 1,
exprCollation(funcexpr));
+ TupleDescFinalize(tupdesc);
}
else if (functypclass == TYPEFUNC_RECORD)
{
@@ -1940,6 +1941,7 @@ addRangeTableEntryForFunction(ParseState *pstate,
i++;
}
+ TupleDescFinalize(tupdesc);
/*
* Ensure that the coldeflist defines a legal set of names (no
@@ -2008,7 +2010,7 @@ addRangeTableEntryForFunction(ParseState *pstate,
0);
/* no need to set collation */
}
-
+ TupleDescFinalize(tupdesc);
Assert(natts == totalatts);
}
else
diff --git a/src/backend/parser/parse_target.c b/src/backend/parser/parse_target.c
index dbf5b2b5c01..a03d82c0540 100644
--- a/src/backend/parser/parse_target.c
+++ b/src/backend/parser/parse_target.c
@@ -1572,6 +1572,8 @@ expandRecordVariable(ParseState *pstate, Var *var, int levelsup)
}
Assert(lname == NULL && lvar == NULL); /* lists same length? */
+ TupleDescFinalize(tupleDesc);
+
return tupleDesc;
}
diff --git a/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c b/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
index 7c8639b32e9..9f04c9ed25d 100644
--- a/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
+++ b/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
@@ -1073,6 +1073,7 @@ libpqrcv_processTuples(PGresult *pgres, WalRcvExecResult *walres,
for (coln = 0; coln < nRetTypes; coln++)
TupleDescInitEntry(walres->tupledesc, (AttrNumber) coln + 1,
PQfname(pgres, coln), retTypes[coln], -1, 0);
+ TupleDescFinalize(walres->tupledesc);
attinmeta = TupleDescGetAttInMetadata(walres->tupledesc);
/* No point in doing more here if there were no tuples returned. */
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c
index 2cde8ebc729..33a9e8d7f21 100644
--- a/src/backend/replication/walsender.c
+++ b/src/backend/replication/walsender.c
@@ -451,6 +451,7 @@ IdentifySystem(void)
TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 4, "dbname",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -496,6 +497,7 @@ ReadReplicationSlot(ReadReplicationSlotCmd *cmd)
/* TimeLineID is unsigned, so int4 is not wide enough. */
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 3, "restart_tli",
INT8OID, -1, 0);
+ TupleDescFinalize(tupdesc);
memset(nulls, true, READ_REPLICATION_SLOT_COLS * sizeof(bool));
@@ -598,6 +600,7 @@ SendTimeLineHistory(TimeLineHistoryCmd *cmd)
tupdesc = CreateTemplateTupleDesc(2);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 1, "filename", TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "content", TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
TLHistoryFileName(histfname, cmd->timeline);
TLHistoryFilePath(path, cmd->timeline);
@@ -1015,6 +1018,7 @@ StartReplication(StartReplicationCmd *cmd)
INT8OID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "next_tli_startpos",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuple */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -1369,6 +1373,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd)
TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 4, "output_plugin",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
diff --git a/src/backend/utils/adt/acl.c b/src/backend/utils/adt/acl.c
index 641673f0b0e..ce07f2bc046 100644
--- a/src/backend/utils/adt/acl.c
+++ b/src/backend/utils/adt/acl.c
@@ -1819,6 +1819,7 @@ aclexplode(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 4, "is_grantable",
BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
/* allocate memory for user context */
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index c083608b1d5..bfb949401d0 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -454,6 +454,7 @@ pg_stat_file(PG_FUNCTION_ARGS)
"creation", TIMESTAMPTZOID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 6,
"isdir", BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
memset(isnull, false, sizeof(isnull));
diff --git a/src/backend/utils/adt/lockfuncs.c b/src/backend/utils/adt/lockfuncs.c
index 9dadd6da672..4481c354fd6 100644
--- a/src/backend/utils/adt/lockfuncs.c
+++ b/src/backend/utils/adt/lockfuncs.c
@@ -146,6 +146,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 16, "waitstart",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
/*
diff --git a/src/backend/utils/adt/orderedsetaggs.c b/src/backend/utils/adt/orderedsetaggs.c
index 3b6da8e36ac..fd8b8676470 100644
--- a/src/backend/utils/adt/orderedsetaggs.c
+++ b/src/backend/utils/adt/orderedsetaggs.c
@@ -233,6 +233,7 @@ ordered_set_startup(FunctionCallInfo fcinfo, bool use_tuples)
-1,
0);
+ TupleDescFinalize(newdesc);
FreeTupleDesc(qstate->tupdesc);
qstate->tupdesc = newdesc;
}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index b1df96e7b0b..0b10da3b180 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -769,6 +769,7 @@ pg_stat_get_backend_subxact(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 2, "subxact_overflow",
BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
if ((local_beentry = pgstat_get_local_beentry_by_proc_number(procNumber)) != NULL)
@@ -1670,6 +1671,7 @@ pg_stat_wal_build_tuple(PgStat_WalCounters wal_counters,
TupleDescInitEntry(tupdesc, (AttrNumber) 6, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
/* Fill values and NULLs */
@@ -2097,6 +2099,7 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 7, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
/* Get statistics about the archiver process */
@@ -2178,6 +2181,7 @@ pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
TIMESTAMPTZOID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 13, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
namestrcpy(&slotname, text_to_cstring(slotname_text));
@@ -2265,6 +2269,7 @@ pg_stat_get_subscription_stats(PG_FUNCTION_ARGS)
INT8OID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 13, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
if (!subentry)
diff --git a/src/backend/utils/adt/tsvector_op.c b/src/backend/utils/adt/tsvector_op.c
index 71c7c7d3b3c..d8dece42b9b 100644
--- a/src/backend/utils/adt/tsvector_op.c
+++ b/src/backend/utils/adt/tsvector_op.c
@@ -651,6 +651,7 @@ tsvector_unnest(PG_FUNCTION_ARGS)
TEXTARRAYOID, -1, 0);
if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
elog(ERROR, "return type must be a row type");
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = tupdesc;
funcctx->user_fctx = PG_GETARG_TSVECTOR_COPY(0);
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 6b634c9fff1..770edb34e08 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -729,6 +729,8 @@ RelationBuildTupleDesc(Relation relation)
pfree(constr);
relation->rd_att->constr = NULL;
}
+
+ TupleDescFinalize(relation->rd_att);
}
/*
@@ -1985,6 +1987,7 @@ formrdesc(const char *relationName, Oid relationReltype,
/* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
TupleDescCompactAttr(relation->rd_att, 0)->attcacheoff = 0;
+ TupleDescFinalize(relation->rd_att);
/* mark not-null status */
if (has_not_null)
@@ -3688,6 +3691,8 @@ RelationBuildLocalRelation(const char *relname,
for (i = 0; i < natts; i++)
TupleDescAttr(rel->rd_att, i)->attrelid = relid;
+ TupleDescFinalize(rel->rd_att);
+
rel->rd_rel->reltablespace = reltablespace;
if (mapped_relation)
@@ -4443,6 +4448,7 @@ BuildHardcodedDescriptor(int natts, const FormData_pg_attribute *attrs)
/* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
TupleDescCompactAttr(result, 0)->attcacheoff = 0;
+ TupleDescFinalize(result);
/* Note: we don't bother to set up a TupleConstr entry */
@@ -6268,6 +6274,8 @@ load_relcache_init_file(bool shared)
populate_compact_attribute(rel->rd_att, i);
}
+ TupleDescFinalize(rel->rd_att);
+
/* next read the access method specific field */
if (fread(&len, 1, sizeof(len), fp) != sizeof(len))
goto read_failed;
diff --git a/src/backend/utils/fmgr/funcapi.c b/src/backend/utils/fmgr/funcapi.c
index 8a934ea8dca..516d02cfb82 100644
--- a/src/backend/utils/fmgr/funcapi.c
+++ b/src/backend/utils/fmgr/funcapi.c
@@ -340,6 +340,8 @@ get_expr_result_type(Node *expr,
exprCollation(col));
i++;
}
+ TupleDescFinalize(tupdesc);
+
if (resultTypeId)
*resultTypeId = rexpr->row_typeid;
if (resultTupleDesc)
@@ -1044,6 +1046,7 @@ resolve_polymorphic_tupdesc(TupleDesc tupdesc, oidvector *declared_args,
}
}
+ TupleDescFinalize(tupdesc);
return true;
}
@@ -1853,6 +1856,8 @@ build_function_result_tupdesc_d(char prokind,
0);
}
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -1970,6 +1975,7 @@ TypeGetTupleDesc(Oid typeoid, List *colaliases)
typeoid,
-1,
0);
+ TupleDescFinalize(tupdesc);
}
else if (functypclass == TYPEFUNC_RECORD)
{
diff --git a/src/backend/utils/misc/guc_funcs.c b/src/backend/utils/misc/guc_funcs.c
index 8524dd3a981..472cb5393ce 100644
--- a/src/backend/utils/misc/guc_funcs.c
+++ b/src/backend/utils/misc/guc_funcs.c
@@ -444,6 +444,7 @@ GetPGVariableResultDesc(const char *name)
TupleDescInitEntry(tupdesc, (AttrNumber) 1, varname,
TEXTOID, -1, 0);
}
+ TupleDescFinalize(tupdesc);
return tupdesc;
}
@@ -465,6 +466,7 @@ ShowGUCConfigOption(const char *name, DestReceiver *dest)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 1, varname,
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -499,6 +501,7 @@ ShowAllGUCConfig(DestReceiver *dest)
TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 3, "description",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -934,6 +937,8 @@ show_all_settings(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 17, "pending_restart",
BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+
/*
* Generate attribute metadata needed later to produce tuples from raw
* C strings
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index d46cdbf7a3c..595413dbbc5 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -195,6 +195,7 @@ extern TupleDesc CreateTupleDescTruncatedCopy(TupleDesc tupdesc, int natts);
extern TupleDesc CreateTupleDescCopyConstr(TupleDesc tupdesc);
+#define TupleDescFinalize(d) ((void) 0)
#define TupleDescSize(src) \
(offsetof(struct TupleDescData, compact_attrs) + \
(src)->natts * sizeof(CompactAttribute) + \
diff --git a/src/pl/plpgsql/src/pl_comp.c b/src/pl/plpgsql/src/pl_comp.c
index 5ecc7766757..b72c963b3be 100644
--- a/src/pl/plpgsql/src/pl_comp.c
+++ b/src/pl/plpgsql/src/pl_comp.c
@@ -1912,6 +1912,8 @@ build_row_from_vars(PLpgSQL_variable **vars, int numvars)
TupleDescInitEntryCollation(row->rowtupdesc, i + 1, typcoll);
}
+ TupleDescFinalize(row->rowtupdesc);
+
return row;
}
diff --git a/src/test/modules/test_custom_stats/test_custom_fixed_stats.c b/src/test/modules/test_custom_stats/test_custom_fixed_stats.c
index 485e08e5c19..f9e7c717280 100644
--- a/src/test/modules/test_custom_stats/test_custom_fixed_stats.c
+++ b/src/test/modules/test_custom_stats/test_custom_fixed_stats.c
@@ -206,6 +206,7 @@ test_custom_stats_fixed_report(PG_FUNCTION_ARGS)
INT8OID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 2, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
values[0] = Int64GetDatum(stats->numcalls);
diff --git a/src/test/modules/test_predtest/test_predtest.c b/src/test/modules/test_predtest/test_predtest.c
index 679a5de456d..48ca2a4ea70 100644
--- a/src/test/modules/test_predtest/test_predtest.c
+++ b/src/test/modules/test_predtest/test_predtest.c
@@ -230,6 +230,7 @@ test_predtest(PG_FUNCTION_ARGS)
"s_r_holds", BOOLOID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 8,
"w_r_holds", BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
tupdesc = BlessTupleDesc(tupdesc);
values[0] = BoolGetDatum(strong_implied_by);
--
2.51.0
[text/plain] v11-0004-Optimize-tuple-deformation.patch (66.9K, 7-v11-0004-Optimize-tuple-deformation.patch)
download | inline diff:
From 0c4bc383f1deae72103063a7e912f276dfd4a1c5 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Tue, 31 Dec 2024 09:19:24 +1300
Subject: [PATCH v11 4/5] Optimize tuple deformation
This commit includes various optimizations to improve the performance of
tuple deformation.
We now precalculate CompactAttribute's attcacheoff, which allows us to
remove the code from the deform routines which was setting the
attcacheoff. Setting the attcacheoff is handled by TupleDescFinalize(),
which must be called before the TupleDesc is used for anything. Having
this TupleDescFinalize() function means we can store the first
attribute in the TupleDesc which does not have an offset cached. That
allows us to add a dedicated deforming loop to deform all attributes up
to the final one with an attcacheoff set, or up to the first NULL
attribute, whichever comes first.
We also record the maximum attribute number which is guaranteed to exist
in the tuple, that is, has a NOT NULL constraint and isn't an
atthasmissing attribute. When deforming only attributes prior to the
guaranteed attnum, we've no need to access the tuple's natt count. As an
additional optimization, we only count fixed-width columns when
calculating the maximum guaranteed column as this eliminates the need to
emit code to fetch byref types in the deformation loop for guaranteed
attributes.
Some locations in the code deform tuples that have yet to go through NOT
NULL constraint validation. We're unable to perform the guaranteed
attribute optimization when that's the case. The optimization is opt-in
via the TupleTableSlot using the TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS
flag.
This commit also adds a more efficient way of populating the isnull
array by using a bit-wise trick which performs multiplication on the
inverse of the tuple's bitmap byte and masking out all but the lower bit
of each of the boolean's byte. This results in much more optimal code
when compared to determining the NULLness via att_isnull(). 8 isnull
elements are processed at once using this method, which means we need to
round the tts_isnull array size up to the next 8 bytes. The palloc code
does this anyway, but the round-up needed to be formalized so as not to
overwrite the sentinel byte in debug builds.
---
src/backend/access/common/heaptuple.c | 360 ++++++++---------
src/backend/access/common/indextuple.c | 363 +++++++----------
src/backend/access/common/tupdesc.c | 51 +++
src/backend/access/spgist/spgutils.c | 3 -
src/backend/executor/execTuples.c | 392 +++++++++++--------
src/backend/executor/nodeBitmapHeapscan.c | 3 +
src/backend/executor/nodeIndexonlyscan.c | 3 +
src/backend/executor/nodeIndexscan.c | 3 +
src/backend/executor/nodeSamplescan.c | 3 +
src/backend/executor/nodeSeqscan.c | 3 +
src/backend/executor/nodeTidrangescan.c | 3 +
src/backend/executor/nodeTidscan.c | 3 +
src/backend/jit/llvm/llvmjit_deform.c | 6 -
src/backend/utils/cache/relcache.c | 12 -
src/include/access/tupdesc.h | 20 +-
src/include/access/tupmacs.h | 224 ++++++++++-
src/include/executor/tuptable.h | 17 +-
src/test/modules/deform_bench/deform_bench.c | 1 +
18 files changed, 846 insertions(+), 624 deletions(-)
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 11bec20e82e..b2ac7fef35b 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -498,19 +498,7 @@ heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc)
* nocachegetattr
*
* This only gets called from fastgetattr(), in cases where we
- * can't use a cacheoffset and the value is not null.
- *
- * This caches attribute offsets in the attribute descriptor.
- *
- * An alternative way to speed things up would be to cache offsets
- * with the tuple, but that seems more difficult unless you take
- * the storage hit of actually putting those offsets into the
- * tuple you send to disk. Yuck.
- *
- * This scheme will be slightly slower than that, but should
- * perform well for queries which hit large #'s of tuples. After
- * you cache the offsets once, examining all the other tuples using
- * the same attribute descriptor will go much quicker. -cim 5/4/91
+ * can't use the attcacheoff and the value is not null.
*
* NOTE: if you need to change this code, see also heap_deform_tuple.
* Also see nocache_index_getattr, which is the same code for index
@@ -522,194 +510,125 @@ nocachegetattr(HeapTuple tup,
int attnum,
TupleDesc tupleDesc)
{
+ CompactAttribute *cattr;
HeapTupleHeader td = tup->t_data;
char *tp; /* ptr to data part of tuple */
bits8 *bp = td->t_bits; /* ptr to null bitmap in tuple */
- bool slow = false; /* do we have to walk attrs? */
int off; /* current offset within data */
+ int startAttr;
+ int firstNullAttr;
+ int i;
+ bool hasnulls = HeapTupleHasNulls(tup);
- /* ----------------
- * Three cases:
- *
- * 1: No nulls and no variable-width attributes.
- * 2: Has a null or a var-width AFTER att.
- * 3: Has nulls or var-widths BEFORE att.
- * ----------------
- */
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
attnum--;
- if (!HeapTupleNoNulls(tup))
+ /*
+ * To minimize the number of attributes we need to look at, start walking
+ * the tuple at the attribute with the highest attcacheoff prior to attnum
+ * or the first NULL attribute prior to attnum, whichever comes first.
+ */
+ if (hasnulls)
+ firstNullAttr = first_null_attr(bp, attnum);
+ else
+ firstNullAttr = attnum;
+
+ if (tupleDesc->firstNonCachedOffsetAttr > 0)
{
/*
- * there's a null somewhere in the tuple
- *
- * check to see if any preceding bits are null...
+ * Start at the highest attcacheoff attribute with no NULLs in prior
+ * attributes.
*/
- int byte = attnum >> 3;
- int finalbit = attnum & 0x07;
-
- /* check for nulls "before" final bit of last byte */
- if ((~bp[byte]) & ((1 << finalbit) - 1))
- slow = true;
- else
- {
- /* check for nulls in any "earlier" bytes */
- int i;
-
- for (i = 0; i < byte; i++)
- {
- if (bp[i] != 0xFF)
- {
- slow = true;
- break;
- }
- }
- }
+ startAttr = Min(tupleDesc->firstNonCachedOffsetAttr - 1, firstNullAttr);
+ off = TupleDescCompactAttr(tupleDesc, startAttr)->attcacheoff;
+ }
+ else
+ {
+ /* Otherwise, start at the beginning... */
+ startAttr = 0;
+ off = 0;
}
tp = (char *) td + td->t_hoff;
- if (!slow)
+ /*
+ * Calculate 'off' up to the first NULL attr. We use two cheaper loops
+ * when the tuple has no variable-width columns. When variable-width
+ * columns exists, we use att_addlength_pointer() to move the offset
+ * beyond the current attribute.
+ */
+ if (!HeapTupleHasVarWidth(tup))
{
- CompactAttribute *att;
-
- /*
- * If we get here, there are no nulls up to and including the target
- * attribute. If we have a cached offset, we can use it.
- */
- att = TupleDescCompactAttr(tupleDesc, attnum);
- if (att->attcacheoff >= 0)
- return fetchatt(att, tp + att->attcacheoff);
-
- /*
- * Otherwise, check for non-fixed-length attrs up to and including
- * target. If there aren't any, it's safe to cheaply initialize the
- * cached offsets for these attrs.
- */
- if (HeapTupleHasVarWidth(tup))
+ for (i = startAttr; i < firstNullAttr; i++)
{
- int j;
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- for (j = 0; j <= attnum; j++)
- {
- if (TupleDescCompactAttr(tupleDesc, j)->attlen <= 0)
- {
- slow = true;
- break;
- }
- }
+ off = att_nominal_alignby(off, cattr->attalignby);
+ off += cattr->attlen;
}
- }
-
- if (!slow)
- {
- int natts = tupleDesc->natts;
- int j = 1;
-
- /*
- * If we get here, we have a tuple with no nulls or var-widths up to
- * and including the target attribute, so we can use the cached offset
- * ... only we don't have it yet, or we'd not have got here. Since
- * it's cheap to compute offsets for fixed-width columns, we take the
- * opportunity to initialize the cached offsets for *all* the leading
- * fixed-width columns, in hope of avoiding future visits to this
- * routine.
- */
- TupleDescCompactAttr(tupleDesc, 0)->attcacheoff = 0;
-
- /* we might have set some offsets in the slow path previously */
- while (j < natts && TupleDescCompactAttr(tupleDesc, j)->attcacheoff > 0)
- j++;
-
- off = TupleDescCompactAttr(tupleDesc, j - 1)->attcacheoff +
- TupleDescCompactAttr(tupleDesc, j - 1)->attlen;
- for (; j < natts; j++)
+ for (; i < attnum; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, j);
+ if (att_isnull(i, bp))
+ continue;
- if (att->attlen <= 0)
- break;
-
- off = att_nominal_alignby(off, att->attalignby);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- att->attcacheoff = off;
-
- off += att->attlen;
+ off = att_nominal_alignby(off, cattr->attalignby);
+ off += cattr->attlen;
}
-
- Assert(j > attnum);
-
- off = TupleDescCompactAttr(tupleDesc, attnum)->attcacheoff;
}
else
{
- bool usecache = true;
- int i;
-
- /*
- * Now we know that we have to walk the tuple CAREFULLY. But we still
- * might be able to cache some offsets for next time.
- *
- * Note - This loop is a little tricky. For each non-null attribute,
- * we have to first account for alignment padding before the attr,
- * then advance over the attr based on its length. Nulls have no
- * storage and no alignment padding either. We can use/set
- * attcacheoff until we reach either a null or a var-width attribute.
- */
- off = 0;
- for (i = 0;; i++) /* loop exit is at "break" */
+ for (i = startAttr; i < firstNullAttr; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, i);
+ int attlen;
- if (HeapTupleHasNulls(tup) && att_isnull(i, bp))
- {
- usecache = false;
- continue; /* this cannot be the target att */
- }
+ cattr = TupleDescCompactAttr(tupleDesc, i);
+ attlen = cattr->attlen;
- /* If we know the next offset, we can skip the rest */
- if (usecache && att->attcacheoff >= 0)
- off = att->attcacheoff;
- else if (att->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be
- * no pad bytes in any case: then the offset will be valid for
- * either an aligned or unaligned value.
- */
- if (usecache &&
- off == att_nominal_alignby(off, att->attalignby))
- att->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, att->attalignby, -1,
- tp + off);
- usecache = false;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, att->attalignby);
+ /*
+ * cstrings don't exist in heap tuples. Use pg_assume to instruct
+ * the compiler not to emit the cstring-related code in
+ * att_addlength_pointer().
+ */
+ pg_assume(attlen > 0 || attlen == -1);
- if (usecache)
- att->attcacheoff = off;
- }
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ attlen,
+ tp + off);
+ off = att_addlength_pointer(off, attlen, tp + off);
+ }
- if (i == attnum)
- break;
+ for (; i < attnum; i++)
+ {
+ int attlen;
- off = att_addlength_pointer(off, att->attlen, tp + off);
+ if (att_isnull(i, bp))
+ continue;
- if (usecache && att->attlen <= 0)
- usecache = false;
+ cattr = TupleDescCompactAttr(tupleDesc, i);
+ attlen = cattr->attlen;
+
+ /* As above, heaptuples have no cstrings */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ off = att_pointer_alignby(off, cattr->attalignby, attlen,
+ tp + off);
+ off = att_addlength_pointer(off, attlen, tp + off);
}
}
- return fetchatt(TupleDescCompactAttr(tupleDesc, attnum), tp + off);
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+
+ return fetchatt(cattr, tp + off);
}
/* ----------------
@@ -1347,6 +1266,7 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,
Datum *values, bool *isnull)
{
HeapTupleHeader tup = tuple->t_data;
+ CompactAttribute *cattr;
bool hasnulls = HeapTupleHasNulls(tuple);
int tdesc_natts = tupleDesc->natts;
int natts; /* number of atts to extract */
@@ -1354,70 +1274,98 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,
char *tp; /* ptr to tuple data */
uint32 off; /* offset in tuple data */
bits8 *bp = tup->t_bits; /* ptr to null bitmap in tuple */
- bool slow = false; /* can we use/set attcacheoff? */
+ int firstNonCacheOffsetAttr;
+ int firstNullAttr;
natts = HeapTupleHeaderGetNatts(tup);
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
+
/*
* In inheritance situations, it is possible that the given tuple actually
* has more fields than the caller is expecting. Don't run off the end of
* the caller's arrays.
*/
natts = Min(natts, tdesc_natts);
+ firstNonCacheOffsetAttr = Min(tupleDesc->firstNonCachedOffsetAttr, natts);
+
+ if (hasnulls)
+ {
+ firstNullAttr = first_null_attr(bp, natts);
+
+ /*
+ * XXX: it'd be nice to use populate_isnull_array() here, but that
+ * requires that the isnull array's size is rounded up to the next
+ * multiple of 8. Doing that would require adjusting many locations
+ * that allocate the array.
+ */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, firstNullAttr);
+ }
+ else
+ firstNullAttr = natts;
tp = (char *) tup + tup->t_hoff;
+ attnum = 0;
- off = 0;
+ if (firstNonCacheOffsetAttr > 0)
+ {
+#ifdef USE_ASSERT_CHECKING
+ /* In Assert enabled builds, verify attcacheoff is correct */
+ int offcheck = 0;
+#endif
+ do
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ off = cattr->attcacheoff;
- for (attnum = 0; attnum < natts; attnum++)
+#ifdef USE_ASSERT_CHECKING
+ offcheck = att_nominal_alignby(offcheck, cattr->attalignby);
+ Assert(offcheck == cattr->attcacheoff);
+ offcheck += cattr->attlen;
+#endif
+
+ values[attnum] = fetch_att_noerr(tp + off,
+ cattr->attbyval,
+ cattr->attlen);
+ } while (++attnum < firstNonCacheOffsetAttr);
+ off += cattr->attlen;
+ }
+ else
+ off = 0;
+
+ for (; attnum < firstNullAttr; attnum++)
{
- CompactAttribute *thisatt = TupleDescCompactAttr(tupleDesc, attnum);
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
+ }
+
+ for (; attnum < natts; attnum++)
+ {
+ Assert(hasnulls);
- if (hasnulls && att_isnull(attnum, bp))
+ if (att_isnull(attnum, bp))
{
values[attnum] = (Datum) 0;
isnull[attnum] = true;
- slow = true; /* can't use attcacheoff anymore */
continue;
}
isnull[attnum] = false;
-
- if (!slow && thisatt->attcacheoff >= 0)
- off = thisatt->attcacheoff;
- else if (thisatt->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be no
- * pad bytes in any case: then the offset will be valid for either
- * an aligned or unaligned value.
- */
- if (!slow &&
- off == att_nominal_alignby(off, thisatt->attalignby))
- thisatt->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, thisatt->attalignby, -1,
- tp + off);
- slow = true;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, thisatt->attalignby);
-
- if (!slow)
- thisatt->attcacheoff = off;
- }
-
- values[attnum] = fetchatt(thisatt, tp + off);
-
- off = att_addlength_pointer(off, thisatt->attlen, tp + off);
-
- if (thisatt->attlen <= 0)
- slow = true; /* can't use attcacheoff anymore */
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
}
/*
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index d6350201e01..8c410853191 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -223,18 +223,6 @@ index_form_tuple_context(TupleDesc tupleDescriptor,
*
* This gets called from index_getattr() macro, and only in cases
* where we can't use cacheoffset and the value is not null.
- *
- * This caches attribute offsets in the attribute descriptor.
- *
- * An alternative way to speed things up would be to cache offsets
- * with the tuple, but that seems more difficult unless you take
- * the storage hit of actually putting those offsets into the
- * tuple you send to disk. Yuck.
- *
- * This scheme will be slightly slower than that, but should
- * perform well for queries which hit large #'s of tuples. After
- * you cache the offsets once, examining all the other tuples using
- * the same attribute descriptor will go much quicker. -cim 5/4/91
* ----------------
*/
Datum
@@ -242,205 +230,124 @@ nocache_index_getattr(IndexTuple tup,
int attnum,
TupleDesc tupleDesc)
{
+ CompactAttribute *cattr;
char *tp; /* ptr to data part of tuple */
bits8 *bp = NULL; /* ptr to null bitmap in tuple */
- bool slow = false; /* do we have to walk attrs? */
int data_off; /* tuple data offset */
int off; /* current offset within data */
+ int startAttr;
+ int firstNullAttr;
+ bool hasnulls = IndexTupleHasNulls(tup);
+ int i;
- /* ----------------
- * Three cases:
- *
- * 1: No nulls and no variable-width attributes.
- * 2: Has a null or a var-width AFTER att.
- * 3: Has nulls or var-widths BEFORE att.
- * ----------------
- */
-
- data_off = IndexInfoFindDataOffset(tup->t_info);
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
attnum--;
- if (IndexTupleHasNulls(tup))
- {
- /*
- * there's a null somewhere in the tuple
- *
- * check to see if desired att is null
- */
+ data_off = IndexInfoFindDataOffset(tup->t_info);
+ tp = (char *) tup + data_off;
- /* XXX "knows" t_bits are just after fixed tuple header! */
+ /*
+ * To minimize the number of attributes we need to look at, start walking
+ * the tuple at the attribute with the highest attcacheoff prior to attnum
+ * or the first NULL attribute prior to attnum, whichever comes first.
+ */
+ if (hasnulls)
+ {
bp = (bits8 *) ((char *) tup + sizeof(IndexTupleData));
-
- /*
- * Now check to see if any preceding bits are null...
- */
- {
- int byte = attnum >> 3;
- int finalbit = attnum & 0x07;
-
- /* check for nulls "before" final bit of last byte */
- if ((~bp[byte]) & ((1 << finalbit) - 1))
- slow = true;
- else
- {
- /* check for nulls in any "earlier" bytes */
- int i;
-
- for (i = 0; i < byte; i++)
- {
- if (bp[i] != 0xFF)
- {
- slow = true;
- break;
- }
- }
- }
- }
+ firstNullAttr = first_null_attr(bp, attnum);
}
+ else
+ firstNullAttr = attnum;
- tp = (char *) tup + data_off;
-
- if (!slow)
+ if (tupleDesc->firstNonCachedOffsetAttr > 0)
{
- CompactAttribute *att;
-
- /*
- * If we get here, there are no nulls up to and including the target
- * attribute. If we have a cached offset, we can use it.
- */
- att = TupleDescCompactAttr(tupleDesc, attnum);
- if (att->attcacheoff >= 0)
- return fetchatt(att, tp + att->attcacheoff);
-
/*
- * Otherwise, check for non-fixed-length attrs up to and including
- * target. If there aren't any, it's safe to cheaply initialize the
- * cached offsets for these attrs.
+ * Start at the highest attcacheoff attribute with no NULLs in prior
+ * attributes.
*/
- if (IndexTupleHasVarwidths(tup))
- {
- int j;
-
- for (j = 0; j <= attnum; j++)
- {
- if (TupleDescCompactAttr(tupleDesc, j)->attlen <= 0)
- {
- slow = true;
- break;
- }
- }
- }
+ startAttr = Min(tupleDesc->firstNonCachedOffsetAttr - 1, firstNullAttr);
+ off = TupleDescCompactAttr(tupleDesc, startAttr)->attcacheoff;
}
-
- if (!slow)
+ else
{
- int natts = tupleDesc->natts;
- int j = 1;
-
- /*
- * If we get here, we have a tuple with no nulls or var-widths up to
- * and including the target attribute, so we can use the cached offset
- * ... only we don't have it yet, or we'd not have got here. Since
- * it's cheap to compute offsets for fixed-width columns, we take the
- * opportunity to initialize the cached offsets for *all* the leading
- * fixed-width columns, in hope of avoiding future visits to this
- * routine.
- */
- TupleDescCompactAttr(tupleDesc, 0)->attcacheoff = 0;
+ /* Otherwise, start at the beginning... */
+ startAttr = 0;
+ off = 0;
+ }
- /* we might have set some offsets in the slow path previously */
- while (j < natts && TupleDescCompactAttr(tupleDesc, j)->attcacheoff > 0)
- j++;
+ /*
+ * Calculate 'off' up to the first NULL attr. We use two cheaper loops
+ * when the tuple has no variable-width columns. When variable-width
+ * columns exists, we use att_addlength_pointer() to move the offset
+ * beyond the current attribute.
+ */
+ if (IndexTupleHasVarwidths(tup))
+ {
+ /* Calculate the offset up until the first NULL */
+ for (i = startAttr; i < firstNullAttr; i++)
+ {
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- off = TupleDescCompactAttr(tupleDesc, j - 1)->attcacheoff +
- TupleDescCompactAttr(tupleDesc, j - 1)->attlen;
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+ off = att_addlength_pointer(off, cattr->attlen, tp + off);
+ }
- for (; j < natts; j++)
+ /* Calculate the offset for any remaining columns. */
+ for (; i < attnum; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, j);
+ Assert(hasnulls);
- if (att->attlen <= 0)
- break;
+ if (att_isnull(i, bp))
+ continue;
- off = att_nominal_alignby(off, att->attalignby);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- att->attcacheoff = off;
-
- off += att->attlen;
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+ off = att_addlength_pointer(off, cattr->attlen, tp + off);
}
-
- Assert(j > attnum);
-
- off = TupleDescCompactAttr(tupleDesc, attnum)->attcacheoff;
}
else
{
- bool usecache = true;
- int i;
+ /* Handle tuples with only fixed-width attributes */
- /*
- * Now we know that we have to walk the tuple CAREFULLY. But we still
- * might be able to cache some offsets for next time.
- *
- * Note - This loop is a little tricky. For each non-null attribute,
- * we have to first account for alignment padding before the attr,
- * then advance over the attr based on its length. Nulls have no
- * storage and no alignment padding either. We can use/set
- * attcacheoff until we reach either a null or a var-width attribute.
- */
- off = 0;
- for (i = 0;; i++) /* loop exit is at "break" */
+ /* Calculate the offset up until the first NULL */
+ for (i = startAttr; i < firstNullAttr; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, i);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- if (IndexTupleHasNulls(tup) && att_isnull(i, bp))
- {
- usecache = false;
- continue; /* this cannot be the target att */
- }
-
- /* If we know the next offset, we can skip the rest */
- if (usecache && att->attcacheoff >= 0)
- off = att->attcacheoff;
- else if (att->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be
- * no pad bytes in any case: then the offset will be valid for
- * either an aligned or unaligned value.
- */
- if (usecache &&
- off == att_nominal_alignby(off, att->attalignby))
- att->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, att->attalignby, -1,
- tp + off);
- usecache = false;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, att->attalignby);
+ Assert(cattr->attlen > 0);
+ off = att_nominal_alignby(off, cattr->attalignby);
+ off += cattr->attlen;
+ }
- if (usecache)
- att->attcacheoff = off;
- }
+ /* Calculate the offset for any remaining columns. */
+ for (; i < attnum; i++)
+ {
+ Assert(hasnulls);
- if (i == attnum)
- break;
+ if (att_isnull(i, bp))
+ continue;
- off = att_addlength_pointer(off, att->attlen, tp + off);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- if (usecache && att->attlen <= 0)
- usecache = false;
+ Assert(cattr->attlen > 0);
+ off = att_nominal_alignby(off, cattr->attalignby);
+ off += cattr->attlen;
}
}
- return fetchatt(TupleDescCompactAttr(tupleDesc, attnum), tp + off);
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ off = att_pointer_alignby(off, cattr->attalignby,
+ cattr->attlen, tp + off);
+ return fetchatt(cattr, tp + off);
}
/*
@@ -480,63 +387,87 @@ index_deform_tuple_internal(TupleDesc tupleDescriptor,
Datum *values, bool *isnull,
char *tp, bits8 *bp, int hasnulls)
{
+ CompactAttribute *cattr;
int natts = tupleDescriptor->natts; /* number of atts to extract */
- int attnum;
- int off = 0; /* offset in tuple data */
- bool slow = false; /* can we use/set attcacheoff? */
+ int attnum = 0;
+ uint32 off = 0; /* offset in tuple data */
+ int firstNonCacheOffsetAttr;
+ int firstNullAttr;
/* Assert to protect callers who allocate fixed-size arrays */
Assert(natts <= INDEX_MAX_KEYS);
- for (attnum = 0; attnum < natts; attnum++)
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDescriptor->firstNonCachedOffsetAttr >= 0);
+
+ firstNonCacheOffsetAttr = Min(tupleDescriptor->firstNonCachedOffsetAttr, natts);
+
+ if (hasnulls)
+ {
+ firstNullAttr = first_null_attr(bp, natts);
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, firstNullAttr);
+ }
+ else
+ firstNullAttr = natts;
+
+ if (firstNonCacheOffsetAttr > 0)
{
- CompactAttribute *thisatt = TupleDescCompactAttr(tupleDescriptor, attnum);
+#ifdef USE_ASSERT_CHECKING
+ /* In Assert enabled builds, verify attcacheoff is correct */
+ off = 0;
+#endif
- if (hasnulls && att_isnull(attnum, bp))
+ do
{
- values[attnum] = (Datum) 0;
- isnull[attnum] = true;
- slow = true; /* can't use attcacheoff anymore */
- continue;
- }
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDescriptor, attnum);
- isnull[attnum] = false;
+#ifdef USE_ASSERT_CHECKING
+ off = att_nominal_alignby(off, cattr->attalignby);
+ Assert(off == cattr->attcacheoff);
+ off += cattr->attlen;
+#endif
- if (!slow && thisatt->attcacheoff >= 0)
- off = thisatt->attcacheoff;
- else if (thisatt->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be no
- * pad bytes in any case: then the offset will be valid for either
- * an aligned or unaligned value.
- */
- if (!slow &&
- off == att_nominal_alignby(off, thisatt->attalignby))
- thisatt->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, thisatt->attalignby, -1,
- tp + off);
- slow = true;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, thisatt->attalignby);
+ values[attnum] = fetch_att_noerr(tp + cattr->attcacheoff, cattr->attbyval,
+ cattr->attlen);
+ } while (++attnum < firstNonCacheOffsetAttr);
- if (!slow)
- thisatt->attcacheoff = off;
- }
+ off = cattr->attcacheoff + cattr->attlen;
+ }
- values[attnum] = fetchatt(thisatt, tp + off);
+ for (; attnum < firstNullAttr; attnum++)
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDescriptor, attnum);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
+ }
- off = att_addlength_pointer(off, thisatt->attlen, tp + off);
+ for (; attnum < natts; attnum++)
+ {
+ Assert(hasnulls);
- if (thisatt->attlen <= 0)
- slow = true; /* can't use attcacheoff anymore */
+ if (att_isnull(attnum, bp))
+ {
+ values[attnum] = (Datum) 0;
+ isnull[attnum] = true;
+ continue;
+ }
+
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDescriptor, attnum);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
}
}
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 2137385a833..c68561337d7 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -197,6 +197,10 @@ CreateTemplateTupleDesc(int natts)
desc->tdtypmod = -1;
desc->tdrefcount = -1; /* assume not reference-counted */
+ /* This will be set to the correct value by TupleDescFinalize() */
+ desc->firstNonCachedOffsetAttr = -1;
+ desc->firstNonGuaranteedAttr = -1;
+
return desc;
}
@@ -457,6 +461,9 @@ TupleDescCopy(TupleDesc dst, TupleDesc src)
* descriptor to another.
*
* !!! Constraints and defaults are not copied !!!
+ *
+ * The caller must take care of calling TupleDescFinalize() on 'dst' once all
+ * TupleDesc changes have been made.
*/
void
TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
@@ -489,6 +496,50 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
populate_compact_attribute(dst, dstAttno - 1);
}
+/*
+ * TupleDescFinalize
+ * Finalize the given TupleDesc. This must be called after the
+ * attributes arrays have been populated or adjusted by any code.
+ *
+ * Must be called after populate_compact_attribute() and before
+ * BlessTupleDesc().
+ */
+void
+TupleDescFinalize(TupleDesc tupdesc)
+{
+ int firstNonCachedOffsetAttr = 0;
+ int firstNonGuaranteedAttr = tupdesc->natts;
+ int off = 0;
+
+ for (int i = 0; i < tupdesc->natts; i++)
+ {
+ CompactAttribute *cattr = TupleDescCompactAttr(tupdesc, i);
+
+ /*
+ * Find the highest attnum which is guaranteed to exist in all tuples
+ * in the table. We currently only pay attention to byval attributes
+ * to allow additional optimizations during tuple deformation.
+ */
+ if (firstNonGuaranteedAttr == tupdesc->natts &&
+ (cattr->attnullability != ATTNULLABLE_VALID || !cattr->attbyval ||
+ cattr->atthasmissing || cattr->attisdropped || cattr->attlen <= 0))
+ firstNonGuaranteedAttr = i;
+
+ if (cattr->attlen <= 0)
+ break;
+
+ off = att_nominal_alignby(off, cattr->attalignby);
+
+ cattr->attcacheoff = off;
+
+ off += cattr->attlen;
+ firstNonCachedOffsetAttr = i + 1;
+ }
+
+ tupdesc->firstNonCachedOffsetAttr = firstNonCachedOffsetAttr;
+ tupdesc->firstNonGuaranteedAttr = firstNonGuaranteedAttr;
+}
+
/*
* Free a TupleDesc including all substructure
*/
diff --git a/src/backend/access/spgist/spgutils.c b/src/backend/access/spgist/spgutils.c
index b246e8127db..a4694bd8065 100644
--- a/src/backend/access/spgist/spgutils.c
+++ b/src/backend/access/spgist/spgutils.c
@@ -335,9 +335,6 @@ getSpGistTupleDesc(Relation index, SpGistTypeDesc *keyType)
/* We shouldn't need to bother with making these valid: */
att->attcompression = InvalidCompressionMethod;
att->attcollation = InvalidOid;
- /* In case we changed typlen, we'd better reset following offsets */
- for (int i = spgFirstIncludeColumn; i < outTupDesc->natts; i++)
- TupleDescCompactAttr(outTupDesc, i)->attcacheoff = -1;
populate_compact_attribute(outTupDesc, spgKeyColumn);
TupleDescFinalize(outTupDesc);
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index bb997182481..83a8c02894d 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -993,225 +993,254 @@ tts_buffer_heap_store_tuple(TupleTableSlot *slot, HeapTuple tuple,
}
/*
- * slot_deform_heap_tuple_internal
- * An always inline helper function for use in slot_deform_heap_tuple to
- * allow the compiler to emit specialized versions of this function for
- * various combinations of "slow" and "hasnulls". For example, if a
- * given tuple has no nulls, then we needn't check "hasnulls" for every
- * attribute that we're deforming. The caller can just call this
- * function with hasnulls set to constant-false and have the compiler
- * remove the constant-false branches and emit more optimal code.
- *
- * Returns the next attnum to deform, which can be equal to natts when the
- * function manages to deform all requested attributes. *offp is an input and
- * output parameter which is the byte offset within the tuple to start deforming
- * from which, on return, gets set to the offset where the next attribute
- * should be deformed from. *slowp is set to true when subsequent deforming
- * of this tuple must use a version of this function with "slow" passed as
- * true.
- *
- * Callers cannot assume when we return "attnum" (i.e. all requested
- * attributes have been deformed) that slow mode isn't required for any
- * additional deforming as the final attribute may have caused a switch to
- * slow mode.
+ * slot_deform_heap_tuple
+ * Given a TupleTableSlot, extract data from the slot's physical tuple
+ * into its Datum/isnull arrays. Data is extracted up through the
+ * reqnatts'th column. If there are insufficient attributes in the given
+ * tuple, then slot_getmissingattrs() is called to populate the
+ * remainder. If reqnatts is above the number of attributes in the
+ * slot's TupleDesc, an error is raised.
+ *
+ * This is essentially an incremental version of heap_deform_tuple:
+ * on each call we extract attributes up to the one needed, without
+ * re-computing information about previously extracted attributes.
+ * slot->tts_nvalid is the number of attributes already extracted.
+ *
+ * This is marked as always inline, so the different offp for different types
+ * of slots gets optimized away.
*/
-static pg_attribute_always_inline int
-slot_deform_heap_tuple_internal(TupleTableSlot *slot, HeapTuple tuple,
- int attnum, int natts, bool slow,
- bool hasnulls, uint32 *offp, bool *slowp)
+static pg_attribute_always_inline void
+slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
+ int reqnatts)
{
+ CompactAttribute *cattr;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
- Datum *values = slot->tts_values;
- bool *isnull = slot->tts_isnull;
HeapTupleHeader tup = tuple->t_data;
+ size_t attnum;
+ int firstNonCacheOffsetAttr;
+ int firstNonGuaranteedAttr;
+ int firstNullAttr;
+ int natts;
+ Datum *values;
+ bool *isnull;
char *tp; /* ptr to tuple data */
- bits8 *bp = tup->t_bits; /* ptr to null bitmap in tuple */
- bool slownext = false;
+ uint32 off; /* offset in tuple data */
- tp = (char *) tup + tup->t_hoff;
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
- for (; attnum < natts; attnum++)
+ isnull = slot->tts_isnull;
+
+ /*
+ * Some callers may form and deform tuples prior to NOT NULL constraints
+ * being checked. Here we'd like to optimize the case where we only need
+ * to fetch attributes before or up to the point where the attribute is
+ * guaranteed to exist in the tuple. We rely on the slot flag being set
+ * correctly to only enable this optimization when it's valid to do so.
+ * This optimization allows us to save fetching the number of attributes
+ * from the tuple and saves the additional cost of handling non-byval
+ * attrs.
+ */
+ if (TTS_OBEYS_NOT_NULL_CONSTRAINTS(slot))
+ firstNonGuaranteedAttr = Min(reqnatts, tupleDesc->firstNonGuaranteedAttr);
+ else
+ firstNonGuaranteedAttr = 0;
+
+ firstNonCacheOffsetAttr = tupleDesc->firstNonCachedOffsetAttr;
+
+ if (HeapTupleHasNulls(tuple))
{
- CompactAttribute *thisatt = TupleDescCompactAttr(tupleDesc, attnum);
+ natts = HeapTupleHeaderGetNatts(tup);
+ tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits) +
+ BITMAPLEN(natts));
- if (hasnulls && att_isnull(attnum, bp))
+ natts = Min(natts, reqnatts);
+ if (natts > firstNonGuaranteedAttr)
{
- values[attnum] = (Datum) 0;
- isnull[attnum] = true;
- if (!slow)
- {
- *slowp = true;
- return attnum + 1;
- }
- else
- continue;
- }
+ bits8 *bp = tup->t_bits;
- isnull[attnum] = false;
+ /* Find the first NULL attr */
+ firstNullAttr = first_null_attr(bp, natts);
- /* calculate the offset of this attribute */
- if (!slow && thisatt->attcacheoff >= 0)
- *offp = thisatt->attcacheoff;
- else if (thisatt->attlen == -1)
- {
/*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be no
- * pad bytes in any case: then the offset will be valid for either
- * an aligned or unaligned value.
+ * And populate the isnull array for all attributes being fetched
+ * from the tuple.
*/
- if (!slow && *offp == att_nominal_alignby(*offp, thisatt->attalignby))
- thisatt->attcacheoff = *offp;
- else
- {
- *offp = att_pointer_alignby(*offp,
- thisatt->attalignby,
- -1,
- tp + *offp);
+ populate_isnull_array(bp, natts, isnull);
- if (!slow)
- slownext = true;
- }
+ /* We can only use any cached offsets until the first NULL attr */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, firstNullAttr);
}
else
{
- /* not varlena, so safe to use att_nominal_alignby */
- *offp = att_nominal_alignby(*offp, thisatt->attalignby);
+ /* Otherwise all required columns are guaranteed to exist */
+ firstNullAttr = natts;
+ }
+ }
+ else
+ {
+ tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits));
- if (!slow)
- thisatt->attcacheoff = *offp;
+ /*
+ * We only need to look at the tuple's natts if we need more than the
+ * guaranteed number of columns
+ */
+ if (reqnatts > firstNonGuaranteedAttr)
+ natts = Min(HeapTupleHeaderGetNatts(tup), reqnatts);
+ else
+ {
+ /* No need to access the number of attributes in the tuple */
+ natts = reqnatts;
}
- values[attnum] = fetchatt(thisatt, tp + *offp);
+ /* All attrs can be fetched without checking for NULLs */
+ firstNullAttr = natts;
+ }
+
+ attnum = slot->tts_nvalid;
+ values = slot->tts_values;
+ slot->tts_nvalid = reqnatts;
- *offp = att_addlength_pointer(*offp, thisatt->attlen, tp + *offp);
+ /* Ensure we calculated tp correctly */
+ Assert(tp == (char *) tup + tup->t_hoff);
- /* check if we need to switch to slow mode */
- if (!slow)
+ if (attnum < firstNonGuaranteedAttr)
+ {
+ do
{
+ int attlen;
+
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ attlen = cattr->attlen;
+
+ /* We don't expect any non-byval types */
+ pg_assume(attlen > 0);
+
/*
- * We're unable to deform any further if the above code set
- * 'slownext', or if this isn't a fixed-width attribute.
+ * Technically we could support non-byval fixed-width types, but
+ * not doing so allows us to pass true to fetch_att_noerr() which
+ * eliminates the !attbyval branch.
*/
- if (slownext || thisatt->attlen <= 0)
- {
- *slowp = true;
- return attnum + 1;
- }
- }
- }
+ Assert(cattr->attbyval == true);
- return natts;
-}
-
-/*
- * slot_deform_heap_tuple
- * Given a TupleTableSlot, extract data from the slot's physical tuple
- * into its Datum/isnull arrays. Data is extracted up through the
- * reqnatts'th column. If there are insufficient attributes in the given
- * tuple, then slot_getmissingattrs() is called to populate the
- * remainder. If reqnatts is above the number of attributes in the
- * slot's TupleDesc, an error is raised.
- *
- * This is essentially an incremental version of heap_deform_tuple:
- * on each call we extract attributes up to the one needed, without
- * re-computing information about previously extracted attributes.
- * slot->tts_nvalid is the number of attributes already extracted.
- *
- * This is marked as always inline, so the different offp for different types
- * of slots gets optimized away.
- */
-static pg_attribute_always_inline void
-slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
- int reqnatts)
-{
- bool hasnulls = HeapTupleHasNulls(tuple);
- int attnum;
- uint32 off; /* offset in tuple data */
- bool slow; /* can we use/set attcacheoff? */
+ off = cattr->attcacheoff;
+ values[attnum] = fetch_att_noerr(tp + off, true, attlen);
+ attnum++;
+ } while (attnum < firstNonGuaranteedAttr);
- /* We can only fetch as many attributes as the tuple has. */
- natts = Min(HeapTupleHeaderGetNatts(tuple->t_data), reqnatts);
+ off += cattr->attlen;
- /*
- * Check whether the first call for this tuple, and initialize or restore
- * loop state.
- */
- attnum = slot->tts_nvalid;
- slot->tts_nvalid = reqnatts;
- if (attnum == 0)
- {
- /* Start from the first attribute */
- off = 0;
- slow = false;
+ if (attnum == reqnatts)
+ goto done;
}
else
{
/* Restore state from previous execution */
off = *offp;
- slow = TTS_SLOW(slot);
+
+ /* We expect *offp to be set to 0 when attnum == 0 */
+ Assert(off == 0 || attnum > 0);
}
+ /* We can only fetch as many attributes as the tuple has. */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, natts);
+
/*
- * If 'slow' isn't set, try deforming using deforming code that does not
- * contain any of the extra checks required for non-fixed offset
- * deforming. During deforming, if or when we find a NULL or a variable
- * length attribute, we'll switch to a deforming method which includes the
- * extra code required for non-fixed offset deforming, a.k.a slow mode.
- * Because this is performance critical, we inline
- * slot_deform_heap_tuple_internal passing the 'slow' and 'hasnull'
- * parameters as constants to allow the compiler to emit specialized code
- * with the known-const false comparisons and subsequent branches removed.
+ * Handle the portion of the tuple that we have cached the offset for up
+ * to the first NULL attribute. The offset is effectively fixed for
+ * these, so we can use the CompactAttribute's attcacheoff.
*/
- if (!slow)
+ if (attnum < firstNonCacheOffsetAttr)
{
- /* Tuple without any NULLs? We can skip doing any NULL checking */
- if (!hasnulls)
- attnum = slot_deform_heap_tuple_internal(slot,
- tuple,
- attnum,
- natts,
- false, /* slow */
- false, /* hasnulls */
- &off,
- &slow);
- else
- attnum = slot_deform_heap_tuple_internal(slot,
- tuple,
- attnum,
- natts,
- false, /* slow */
- true, /* hasnulls */
- &off,
- &slow);
+ do
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+
+ off = cattr->attcacheoff;
+ values[attnum] = fetch_att_noerr(tp + off,
+ cattr->attbyval,
+ cattr->attlen);
+ } while (++attnum < firstNonCacheOffsetAttr);
+
+ /*
+ * Point the offset after the end of the last attribute with a cached
+ * offset. We expect the final cached offset attribute to have a
+ * fixed width, so just add the attlen to the attcacheoff
+ */
+ Assert(cattr->attlen > 0);
+ off += cattr->attlen;
}
- /* If there's still work to do then we must be in slow mode */
- if (attnum < natts)
+ /*
+ * Handle any portion of the tuple that doesn't have a fixed offset up
+ * until the first NULL attribute. This loop only differs from the one
+ * after it by the NULL checks.
+ */
+ for (; attnum < firstNullAttr; attnum++)
{
- /* XXX is it worth adding a separate call when hasnulls is false? */
- attnum = slot_deform_heap_tuple_internal(slot,
- tuple,
- attnum,
- natts,
- true, /* slow */
- hasnulls,
- &off,
- &slow);
+ int attlen;
+
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ attlen = cattr->attlen;
+
+ /*
+ * cstrings don't exist in heap tuples. Use pg_assume to instruct the
+ * compiler not to emit the cstring-related code in
+ * align_fetch_then_add().
+ */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ attlen,
+ cattr->attalignby);
}
/*
- * Save state for next execution
+ * Now handle any remaining attributes in the tuple up to the requested
+ * attnum. This time, include NULL checks as we're now at the first NULL
+ * attribute.
*/
- *offp = off;
- if (slow)
- slot->tts_flags |= TTS_FLAG_SLOW;
- else
- slot->tts_flags &= ~TTS_FLAG_SLOW;
+ for (; attnum < natts; attnum++)
+ {
+ int attlen;
+
+ if (isnull[attnum])
+ {
+ values[attnum] = (Datum) 0;
+ continue;
+ }
+
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ attlen = cattr->attlen;
- /* Fetch any missing attrs and raise an error if reqnatts is invalid. */
+ /* As above, we don't expect cstrings */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ attlen,
+ cattr->attalignby);
+ }
+
+ /* Fetch any missing attrs and raise an error if reqnatts is invalid */
if (unlikely(attnum < reqnatts))
+ {
+ *offp = off;
slot_getmissingattrs(slot, attnum, reqnatts);
+ return;
+ }
+done:
+
+ /* Save current offset for next execution */
+ *offp = off;
}
const TupleTableSlotOps TTSOpsVirtual = {
@@ -1341,10 +1370,17 @@ MakeTupleTableSlot(TupleDesc tupleDesc,
slot->tts_values = (Datum *)
(((char *) slot)
+ MAXALIGN(basesz));
+
+ /*
+ * We round the size of tts_isnull up to the next highest multiple of
+ * 8. This is needed as populate_isnull_array() operates on 8
+ * elements at a time when converting a tuple's NULL bitmap into a
+ * boolean array.
+ */
slot->tts_isnull = (bool *)
(((char *) slot)
+ MAXALIGN(basesz)
- + MAXALIGN(tupleDesc->natts * sizeof(Datum)));
+ + TYPEALIGN(8, tupleDesc->natts * sizeof(Datum)));
PinTupleDesc(tupleDesc);
}
@@ -1514,8 +1550,14 @@ ExecSetSlotDescriptor(TupleTableSlot *slot, /* slot to change */
*/
slot->tts_values = (Datum *)
MemoryContextAlloc(slot->tts_mcxt, tupdesc->natts * sizeof(Datum));
+
+ /*
+ * We round the size of tts_isnull up to the next highest multiple of 8.
+ * This is needed as populate_isnull_array() operates on 8 elements at a
+ * time when converting a tuple's NULL bitmap into a boolean array.
+ */
slot->tts_isnull = (bool *)
- MemoryContextAlloc(slot->tts_mcxt, tupdesc->natts * sizeof(bool));
+ MemoryContextAlloc(slot->tts_mcxt, TYPEALIGN(8, tupdesc->natts * sizeof(bool)));
}
/* --------------------------------
@@ -2260,10 +2302,16 @@ ExecTypeSetColNames(TupleDesc typeInfo, List *namesList)
* This happens "for free" if the tupdesc came from a relcache entry, but
* not if we have manufactured a tupdesc for a transient RECORD datatype.
* In that case we have to notify typcache.c of the existence of the type.
+ *
+ * TupleDescFinalize() must be called on the TupleDesc before calling this
+ * function.
*/
TupleDesc
BlessTupleDesc(TupleDesc tupdesc)
{
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupdesc->firstNonCachedOffsetAttr >= 0);
+
if (tupdesc->tdtypeid == RECORDOID &&
tupdesc->tdtypmod < 0)
assign_record_type_typmod(tupdesc);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index c68c26cbf38..b17c4e721b3 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -383,6 +383,9 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
RelationGetDescr(currentRelation),
table_slot_callbacks(currentRelation));
+ scanstate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
+
/*
* Initialize result type and projection.
*/
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index c2d09374517..506fdf446d2 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -569,6 +569,9 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
ExecInitScanTupleSlot(estate, &indexstate->ss, tupDesc,
&TTSOpsVirtual);
+ indexstate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
+
/*
* We need another slot, in a format that's suitable for the table AM, for
* when we need to fetch a tuple from the table for rechecking visibility.
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index a616abff04c..c77746ab9f5 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -940,6 +940,9 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
RelationGetDescr(currentRelation),
table_slot_callbacks(currentRelation));
+ indexstate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
+
/*
* Initialize result type and projection.
*/
diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c
index 1b0af70fd7a..d29ef2872f7 100644
--- a/src/backend/executor/nodeSamplescan.c
+++ b/src/backend/executor/nodeSamplescan.c
@@ -130,6 +130,9 @@ ExecInitSampleScan(SampleScan *node, EState *estate, int eflags)
RelationGetDescr(scanstate->ss.ss_currentRelation),
table_slot_callbacks(scanstate->ss.ss_currentRelation));
+ scanstate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
+
/*
* Initialize result type and projection.
*/
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index af3c788ce8b..3ff2a2843eb 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -246,6 +246,9 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags)
RelationGetDescr(scanstate->ss.ss_currentRelation),
table_slot_callbacks(scanstate->ss.ss_currentRelation));
+ scanstate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
+
/*
* Initialize result type and projection.
*/
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index 503817da65b..2ece0255e7d 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -396,6 +396,9 @@ ExecInitTidRangeScan(TidRangeScan *node, EState *estate, int eflags)
RelationGetDescr(currentRelation),
table_slot_callbacks(currentRelation));
+ tidrangestate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
+
/*
* Initialize result type and projection.
*/
diff --git a/src/backend/executor/nodeTidscan.c b/src/backend/executor/nodeTidscan.c
index 4eddb0828b5..484e3306e0b 100644
--- a/src/backend/executor/nodeTidscan.c
+++ b/src/backend/executor/nodeTidscan.c
@@ -538,6 +538,9 @@ ExecInitTidScan(TidScan *node, EState *estate, int eflags)
RelationGetDescr(currentRelation),
table_slot_callbacks(currentRelation));
+ tidstate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
+
/*
* Initialize result type and projection.
*/
diff --git a/src/backend/jit/llvm/llvmjit_deform.c b/src/backend/jit/llvm/llvmjit_deform.c
index 3eb087eb56b..12521e3e46a 100644
--- a/src/backend/jit/llvm/llvmjit_deform.c
+++ b/src/backend/jit/llvm/llvmjit_deform.c
@@ -62,7 +62,6 @@ slot_compile_deform(LLVMJitContext *context, TupleDesc desc,
LLVMValueRef v_tts_values;
LLVMValueRef v_tts_nulls;
LLVMValueRef v_slotoffp;
- LLVMValueRef v_flagsp;
LLVMValueRef v_nvalidp;
LLVMValueRef v_nvalid;
LLVMValueRef v_maxatt;
@@ -178,7 +177,6 @@ slot_compile_deform(LLVMJitContext *context, TupleDesc desc,
v_tts_nulls =
l_load_struct_gep(b, StructTupleTableSlot, v_slot, FIELDNO_TUPLETABLESLOT_ISNULL,
"tts_ISNULL");
- v_flagsp = l_struct_gep(b, StructTupleTableSlot, v_slot, FIELDNO_TUPLETABLESLOT_FLAGS, "");
v_nvalidp = l_struct_gep(b, StructTupleTableSlot, v_slot, FIELDNO_TUPLETABLESLOT_NVALID, "");
if (ops == &TTSOpsHeapTuple || ops == &TTSOpsBufferHeapTuple)
@@ -747,14 +745,10 @@ slot_compile_deform(LLVMJitContext *context, TupleDesc desc,
{
LLVMValueRef v_off = l_load(b, TypeSizeT, v_offp, "");
- LLVMValueRef v_flags;
LLVMBuildStore(b, l_int16_const(lc, natts), v_nvalidp);
v_off = LLVMBuildTrunc(b, v_off, LLVMInt32TypeInContext(lc), "");
LLVMBuildStore(b, v_off, v_slotoffp);
- v_flags = l_load(b, LLVMInt16TypeInContext(lc), v_flagsp, "tts_flags");
- v_flags = LLVMBuildOr(b, v_flags, l_int16_const(lc, TTS_FLAG_SLOW), "");
- LLVMBuildStore(b, v_flags, v_flagsp);
LLVMBuildRetVoid(b);
}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 770edb34e08..998be24ac41 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -666,14 +666,6 @@ RelationBuildTupleDesc(Relation relation)
elog(ERROR, "pg_attribute catalog is missing %d attribute(s) for relation OID %u",
need, RelationGetRelid(relation));
- /*
- * We can easily set the attcacheoff value for the first attribute: it
- * must be zero. This eliminates the need for special cases for attnum=1
- * that used to exist in fastgetattr() and index_getattr().
- */
- if (RelationGetNumberOfAttributes(relation) > 0)
- TupleDescCompactAttr(relation->rd_att, 0)->attcacheoff = 0;
-
/*
* Set up constraint/default info
*/
@@ -1985,8 +1977,6 @@ formrdesc(const char *relationName, Oid relationReltype,
populate_compact_attribute(relation->rd_att, i);
}
- /* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
- TupleDescCompactAttr(relation->rd_att, 0)->attcacheoff = 0;
TupleDescFinalize(relation->rd_att);
/* mark not-null status */
@@ -4446,8 +4436,6 @@ BuildHardcodedDescriptor(int natts, const FormData_pg_attribute *attrs)
populate_compact_attribute(result, i);
}
- /* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
- TupleDescCompactAttr(result, 0)->attcacheoff = 0;
TupleDescFinalize(result);
/* Note: we don't bother to set up a TupleConstr entry */
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 595413dbbc5..ad7bc013812 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -131,6 +131,19 @@ typedef struct CompactAttribute
* Any code making changes manually to and fields in the FormData_pg_attribute
* array must subsequently call populate_compact_attribute() to flush the
* changes out to the corresponding 'compact_attrs' element.
+ *
+ * firstNonCachedOffsetAttr stores the index into the compact_attrs array for
+ * the first attribute that we don't have a known attcacheoff for.
+ *
+ * firstNonGuaranteedAttr stores the index to into the compact_attrs array for
+ * the first attribute that is either NULLable, missing, or !attbyval. This
+ * can be used in locations as a guarantee that attributes before this will
+ * always exist in tuples. The !attbyval part isn't required for this, but
+ * including this allows various tuple deforming routines to forego any checks
+ * for !attbyval.
+ *
+ * Once a TupleDesc has been populated, before it is used for any purpose,
+ * TupleDescFinalize() must be called on it.
*/
typedef struct TupleDescData
{
@@ -138,6 +151,11 @@ typedef struct TupleDescData
Oid tdtypeid; /* composite type ID for tuple type */
int32 tdtypmod; /* typmod for tuple type */
int tdrefcount; /* reference count, or -1 if not counting */
+ int firstNonCachedOffsetAttr; /* index of the first att without
+ * an attcacheoff */
+ int firstNonGuaranteedAttr; /* index of the first nullable,
+ * missing, dropped, or !attbyval
+ * attribute. */
TupleConstr *constr; /* constraints, or NULL if none */
/* compact_attrs[N] is the compact metadata of Attribute Number N+1 */
CompactAttribute compact_attrs[FLEXIBLE_ARRAY_MEMBER];
@@ -195,7 +213,6 @@ extern TupleDesc CreateTupleDescTruncatedCopy(TupleDesc tupdesc, int natts);
extern TupleDesc CreateTupleDescCopyConstr(TupleDesc tupdesc);
-#define TupleDescFinalize(d) ((void) 0)
#define TupleDescSize(src) \
(offsetof(struct TupleDescData, compact_attrs) + \
(src)->natts * sizeof(CompactAttribute) + \
@@ -206,6 +223,7 @@ extern void TupleDescCopy(TupleDesc dst, TupleDesc src);
extern void TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
TupleDesc src, AttrNumber srcAttno);
+extern void TupleDescFinalize(TupleDesc tupdesc);
extern void FreeTupleDesc(TupleDesc tupdesc);
extern void IncrTupleDescRefCount(TupleDesc tupdesc);
diff --git a/src/include/access/tupmacs.h b/src/include/access/tupmacs.h
index d64c18b950b..87dbeb76618 100644
--- a/src/include/access/tupmacs.h
+++ b/src/include/access/tupmacs.h
@@ -15,7 +15,9 @@
#define TUPMACS_H
#include "catalog/pg_type_d.h" /* for TYPALIGN macros */
-
+#include "port/pg_bitutils.h"
+#include "port/pg_bswap.h"
+#include "varatt.h"
/*
* Check a tuple's null bitmap to determine whether the attribute is null.
@@ -28,6 +30,62 @@ att_isnull(int ATT, const bits8 *BITS)
return !(BITS[ATT >> 3] & (1 << (ATT & 0x07)));
}
+/*
+ * populate_isnull_array
+ * Transform a tuple's null bitmap into a boolean array.
+ *
+ * Caller must ensure that the isnull array is sized so it contains
+ * at least as many elements as there are bits in the 'bits' array.
+ * Callers should be aware that isnull is populated 8 elements at a time,
+ * effectively as if natts is rounded up to the next multiple of 8.
+ */
+static inline void
+populate_isnull_array(const bits8 *bits, int natts, bool *isnull)
+{
+ int nbytes = (natts + 7) >> 3;
+
+ /*
+ * Multiplying the inverted NULL bitmap byte by this value results in the
+ * lowest bit in each byte being set the same as each bit of the inverted
+ * byte. We perform this as 2 32-bit operations rather than a single
+ * 64-bit operation as multiplying by the required value to do this in
+ * 64-bits would result in overflowing a uint64 in some cases.
+ *
+ * XXX if we ever require BMI2 (-march=x86-64-v3), then this could be done
+ * more efficiently on most X86-64 CPUs with the PDEP instruction. Beware
+ * that some chips (e.g. AMD's Zen2) are horribly inefficient at PDEP.
+ */
+#define SPREAD_BITS_MULTIPLIER_32 0x204081U
+
+ for (int i = 0; i < nbytes; i++, isnull += 8)
+ {
+ uint64 isnull_8;
+ bits8 nullbyte = ~bits[i];
+
+ /* Convert the lower 4 bits of NULL bitmap word into a 64 bit int */
+ isnull_8 = (nullbyte & 0xf) * SPREAD_BITS_MULTIPLIER_32;
+
+ /*
+ * Convert the upper 4 bits of null bitmap word into a 64 bit int,
+ * shift into the upper 32 bit and bitwise-OR with the result of the
+ * lower 4 bits.
+ */
+ isnull_8 |= ((uint64) ((nullbyte >> 4) * SPREAD_BITS_MULTIPLIER_32)) << 32;
+
+ /* Mask out all other bits apart from the lowest bit of each byte. */
+ isnull_8 &= UINT64CONST(0x0101010101010101);
+
+#ifdef WORDS_BIGENDIAN
+
+ /*
+ * Fix byte order on big-endian machines before copying to the array.
+ */
+ isnull_8 = pg_bswap64(isnull_8);
+#endif
+ memcpy(isnull, &isnull_8, sizeof(uint64));
+ }
+}
+
#ifndef FRONTEND
/*
* Given an attbyval and an attlen from either a Form_pg_attribute or
@@ -69,6 +127,170 @@ fetch_att(const void *T, bool attbyval, int attlen)
else
return PointerGetDatum(T);
}
+
+/*
+ * Same, but no error checking for invalid attlens for byval types. This
+ * is safe to use when attlen comes from CompactAttribute as we validate the
+ * length when populating that struct.
+ */
+static inline Datum
+fetch_att_noerr(const void *T, bool attbyval, int attlen)
+{
+ if (attbyval)
+ {
+ switch (attlen)
+ {
+ case sizeof(int32):
+ return Int32GetDatum(*((const int32 *) T));
+ case sizeof(int16):
+ return Int16GetDatum(*((const int16 *) T));
+ case sizeof(char):
+ return CharGetDatum(*((const char *) T));
+ default:
+ Assert(attlen == sizeof(int64));
+ return Int64GetDatum(*((const int64 *) T));
+ }
+ }
+ else
+ return PointerGetDatum(T);
+}
+
+
+/*
+ * align_fetch_then_add
+ * Applies all the functionality of att_pointer_alignby(),
+ * fetch_att_noerr() and att_addlength_pointer(), resulting in the *off
+ * pointer to the perhaps unaligned number of bytes into 'tupptr', ready
+ * to deform the next attribute.
+ *
+ * tupptr: pointer to the beginning of the tuple, after the header and any
+ * NULL bitmask.
+ * off: offset in bytes for reading tuple data, possibly unaligned.
+ * attbyval, attlen and attalignby are values from CompactAttribute.
+ */
+static inline Datum
+align_fetch_then_add(const char *tupptr, uint32 *off, bool attbyval, int attlen,
+ uint8 attalignby)
+{
+ Datum res;
+
+ if (attlen > 0)
+ {
+ const char *offset_ptr;
+
+ *off = TYPEALIGN(attalignby, *off);
+ offset_ptr = tupptr + *off;
+ *off += attlen;
+ if (attbyval)
+ {
+ switch (attlen)
+ {
+ case sizeof(char):
+ return CharGetDatum(*((const char *) offset_ptr));
+ case sizeof(int16):
+ return Int16GetDatum(*((const int16 *) offset_ptr));
+ case sizeof(int32):
+ return Int32GetDatum(*((const int32 *) offset_ptr));
+ default:
+
+ /*
+ * populate_compact_attribute_internal() should have
+ * checked
+ */
+ Assert(attlen == sizeof(int64));
+ return Int64GetDatum(*((const int64 *) offset_ptr));
+ }
+ }
+ return PointerGetDatum(offset_ptr);
+ }
+ else if (attlen == -1)
+ {
+ if (!VARATT_IS_SHORT(tupptr + *off))
+ *off = TYPEALIGN(attalignby, *off);
+
+ res = PointerGetDatum(tupptr + *off);
+ *off += VARSIZE_ANY(DatumGetPointer(res));
+ return res;
+ }
+ else
+ {
+ Assert(attlen == -2);
+ *off = TYPEALIGN(attalignby, *off);
+ res = PointerGetDatum(tupptr + *off);
+ *off += strlen(tupptr + *off) + 1;
+ return res;
+ }
+}
+
+/*
+ * first_null_attr
+ * Inspect a NULL bitmap from a tuple and return the 0-based attnum of the
+ * first NULL attribute. Returns natts if no NULLs were found.
+ *
+ * This is coded to expect that 'bits' contains at least one 0 bit somewhere
+ * in the array, but not necessarily < natts. Note that natts may be passed
+ * as a value lower than the number of bits physically stored in the tuple's
+ * NULL bitmap, in which case we may not find a NULL and return natts.
+ *
+ * The reason we require at least one 0 bit somewhere in the NULL bitmap is
+ * that the for loop that checks 0xFF bytes would loop to the last byte in
+ * the array if all bytes were 0xFF, and the subsequent code that finds the
+ * right-most 0 bit would access the first byte beyond the bitmap. Provided
+ * we find a 0 bit before then, that won't happen. Since tuples which have no
+ * NULLs don't have a NULL bitmap, this function won't get called for that
+ * case.
+ */
+static inline int
+first_null_attr(const bits8 *bits, int natts)
+{
+ int nattByte = natts >> 3;
+ int bytenum;
+ int res;
+
+#ifdef USE_ASSERT_CHECKING
+ int firstnull_check = natts;
+
+ /* Do it the slow way and check we get the same answer. */
+ for (int i = 0; i < natts; i++)
+ {
+ if (att_isnull(i, bits))
+ {
+ firstnull_check = i;
+ break;
+ }
+ }
+#endif
+
+ /* Process all bytes up to just before the byte for the natts attribute */
+ for (bytenum = 0; bytenum < nattByte; bytenum++)
+ {
+ /* break if there's any NULL attrs (a 0 bit) */
+ if (bits[bytenum] != 0xFF)
+ break;
+ }
+
+ /*
+ * Look for the highest 0-bit in the 'bytenum' element. To do this, we
+ * promote the uint8 to uint32 before performing the bitwise NOT and
+ * looking for the first 1-bit. This works even when the byte is 0xFF, as
+ * the bitwise NOT of 0xFF in 32 bits is 0xFFFFFF00, in which case
+ * pg_rightmost_one_pos32() will return 8. We may end up with a value
+ * higher than natts here, but we'll fix that with the Min() below.
+ */
+ res = bytenum << 3;
+ res += pg_rightmost_one_pos32(~((uint32) bits[bytenum]));
+
+ /*
+ * Since we did no masking to mask out bits beyond the natt'th bit, we may
+ * have found a bit higher than natts, so we must cap res to natts
+ */
+ res = Min(res, natts);
+
+ /* Ensure we got the same answer as the att_isnull() loop got */
+ Assert(res == firstnull_check);
+
+ return res;
+}
#endif /* FRONTEND */
/*
diff --git a/src/include/executor/tuptable.h b/src/include/executor/tuptable.h
index 3b09abbf99f..ff4572a29ae 100644
--- a/src/include/executor/tuptable.h
+++ b/src/include/executor/tuptable.h
@@ -84,9 +84,6 @@
* tts_values/tts_isnull are allocated either when the slot is created (when
* the descriptor is provided), or when a descriptor is assigned to the slot;
* they are of length equal to the descriptor's natts.
- *
- * The TTS_FLAG_SLOW flag is saved state for
- * slot_deform_heap_tuple, and should not be touched by any other code.
*----------
*/
@@ -98,9 +95,13 @@
#define TTS_FLAG_SHOULDFREE (1 << 2)
#define TTS_SHOULDFREE(slot) (((slot)->tts_flags & TTS_FLAG_SHOULDFREE) != 0)
-/* saved state for slot_deform_heap_tuple */
-#define TTS_FLAG_SLOW (1 << 3)
-#define TTS_SLOW(slot) (((slot)->tts_flags & TTS_FLAG_SLOW) != 0)
+/*
+ * true = slot's formed tuple guaranteed to not have NULLs in NOT NULLable
+ * columns.
+ */
+#define TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS (1 << 3)
+#define TTS_OBEYS_NOT_NULL_CONSTRAINTS(slot) \
+ (((slot)->tts_flags & TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS) != 0)
/* fixed tuple descriptor */
#define TTS_FLAG_FIXED (1 << 4)
@@ -123,7 +124,9 @@ typedef struct TupleTableSlot
#define FIELDNO_TUPLETABLESLOT_VALUES 5
Datum *tts_values; /* current per-attribute values */
#define FIELDNO_TUPLETABLESLOT_ISNULL 6
- bool *tts_isnull; /* current per-attribute isnull flags */
+ bool *tts_isnull; /* current per-attribute isnull flags. Array
+ * size must always be rounded up to the next
+ * multiple of 8 elements. */
MemoryContext tts_mcxt; /* slot itself is in this context */
ItemPointerData tts_tid; /* stored tuple's tid */
Oid tts_tableOid; /* table oid of tuple */
diff --git a/src/test/modules/deform_bench/deform_bench.c b/src/test/modules/deform_bench/deform_bench.c
index 7838f639bef..de39fecf8fd 100644
--- a/src/test/modules/deform_bench/deform_bench.c
+++ b/src/test/modules/deform_bench/deform_bench.c
@@ -49,6 +49,7 @@ deform_bench(PG_FUNCTION_ARGS)
tupdesc = RelationGetDescr(rel);
slot = MakeTupleTableSlot(tupdesc, &TTSOpsBufferHeapTuple);
+ slot->tts_flags |= TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
scan = table_beginscan_strat(rel, GetActiveSnapshot(), 0, NULL, true, false);
/*
--
2.51.0
[application/vnd.openxmlformats-officedocument.spreadsheetml.sheet] Deform_bench_test_module_results_v11.xlsx (36.7K, 8-Deform_bench_test_module_results_v11.xlsx)
download
^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: More speedups for tuple deformation
@ 2026-03-06 04:09 David Rowley <[email protected]>
parent: David Rowley <[email protected]>
0 siblings, 1 reply; 30+ messages in thread
From: David Rowley @ 2026-03-06 04:09 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: John Naylor <[email protected]>; Chao Li <[email protected]>; PostgreSQL Developers <[email protected]>
One of my goals for proactively populating
CompactAttribute.attcacheoff is to make it so we're able to support
deforming only a subset of columns. If we only need a small number of
columns from the tuple and all those columns have a known attcacheoff
and no NULLs come prior, then we can quite efficiently just go to
those cached offsets and fetch only the attributes that we need. To
do this, we'll need an extra array to store which attnums we're
interested in, rather than deforming all attrs up to the highest
attnum that we need, as we do today. I expect that looking at this
new array will slow things down a bit when we're accessing either most
or all columns in, say, a SELECT * query. So, IMO, it'd be bad to
*replace* the current deforming code with code which does this.
Instead, I propose we add an additional deform operator and have some
heuristic which decides which one is best to use. I expect
ExecPushExprSetupSteps() could make that choice fairly easily. Perhaps
something cheap like bms_num_members(scan_attrs) is less than half the
bms_prev_member(scan_attrs, -1) (the highest member).
There's going to be many cases where the attcacheoff isn't known in
the attributes being selected. So that we still get some gains when
that's the case, I've coded it up so that we start walking the tuple
at the last attribute that has an attcacheoff. In many cases, that'll
mean we don't need to walk the entire tuple. Often, leading columns
are fixed-width, so this means that there's likely some benefit to
most cases. There might need to be a bit more education or
documentation about best column ordering practises.
There are a few hurdles to make this work, and one is the physical
tlist optimization. If the planner replaces the targetlist with a
physical tlist, the executor is going to think we need all columns,
which would have it likely choose not to do the selective deforming.
To make this work, I've added some code in createplan.c to extract the
attnums we need from the qual and tlist before the physical tlist is
installed. That's recorded in a Bitmapset and passed down to the
executor and to the code which sets up the ExprStates. Currently,
mostly to exercise this code as much as possible, I've coded it to
always do the selective deforming when the Bitmapset isn't empty. So
far, I've only done this for Seq Scan, but I expect all the scans that
deform tuples could use this.
I've attached the code which does all this in the 0006 patch.
Ideally, I'd have had this at least to the current state about 2-3
months ago, so I don't intend that 0006 is v19 material, but I wanted
to share to show where I intend this work to go.
Performance:
Using the t_1_40 table from the deform_test_setup.sh script I sent in
[1], running "select a from t_1_40 where a = 0;" ("a" is the 43rd
column in that table), on my Zen2 machine, I get the following from
perf top and pgbench:
master:
75.57% postgres [.] tts_buffer_heap_getsomeattrs
4.70% postgres [.] ExecInterpExpr
2.85% postgres [.] ExecSeqScanWithQualProject
1.94% postgres [.] heapgettup_pagemode
1.21% postgres [.] UnlockBuffer
1.15% postgres [.] slot_getsomeattrs_int
$ for i in {1..3}; do pgbench -n -f bench.sql -M prepared -T 10
postgres | grep latency; done
latency average = 154.175 ms
latency average = 156.780 ms
latency average = 157.599 ms
0001-0005:
64.24% postgres [.] tts_buffer_heap_getsomeattrs
15.01% postgres [.] ExecInterpExpr
3.22% postgres [.] ExecSeqScanWithQualProject
3.01% postgres [.] heapgettup_pagemode
1.57% postgres [.] ExecStoreBufferHeapTuple
1.53% postgres [.] heap_prepare_pagescan
$ for i in {1..3}; do pgbench -n -f bench.sql -M prepared -T 10
postgres | grep latency; done
latency average = 130.981 ms
latency average = 134.700 ms
latency average = 134.898 ms
0001-0006:
42.28% postgres [.] heapgettup_pagemode
11.38% postgres [.] ExecInterpExpr
7.13% postgres [.] ExecSeqScanWithQualProject
5.92% postgres [.] tts_buffer_heap_selectattrs <-- it's down here.
5.69% postgres [.] ExecStoreBufferHeapTuple
5.11% postgres [.] heap_getnextslot
3.87% postgres [.] heap_prepare_pagescan
$ for i in {1..3}; do pgbench -n -f bench.sql -M prepared -T 10
postgres | grep latency; done
latency average = 71.689 ms
latency average = 75.638 ms
latency average = 75.149 ms
Keep in mind that this is one of the best cases as t_1_40 has no NULLs
and only has fixed-width columns. The only slightly better case would
be to add more columns and fetch only the final one. 40 doesn't seem
excessively unrealistic, to get an idea of the gains that someone
*could* see.
You can see that perf top report that tts_buffer_heap_getsomeattrs
dropped from taking 75.57% down to 64.24% with 0001-0005. Adding 0006
sees that replaced with tts_buffer_heap_selectattrs which takes less
than 6% of the CPU time. It also highlights the next most interesting
thing we should probably make faster, heapgettup_pagemode().
I've attached v12 of the patch. There are a few changes in 0001-0005
that should help make things a bit faster than v11. I've also attached
the new selective deforming code in 0006. There's no JIT support for
0006 yet, I don't need to be told about that :-)
I'm planning on starting to go through 0002-0005 in much more detail
from mid next week with my committer hat on. If anyone wants to relook
at any of the 0002-0005 patches, there's still time. I'm also happy to
receive feedback on 0006, but I will address concerns with that at a
lower priority. One thing that's still left todo in the 0004 patch is
enable the TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS optimisation for a few
other scan types.
Thanks for reading
David
[1] https://postgr.es/m/CAApHDvo1i-ycAcWnK3L7ZASTuM8mW46kvRqMaUHD46HSuJmx7A@mail.gmail.com
From 27cf8d0007487d9dd22a7b6e0562e0399403dfef Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Tue, 27 Jan 2026 15:08:09 +1300
Subject: [PATCH v12 1/6] Introduce deform_bench test module
For benchmarking tuple deformation.
---
src/test/modules/deform_bench/.gitignore | 4 +
src/test/modules/deform_bench/Makefile | 21 ++++
.../deform_bench/deform_bench--1.0.sql | 8 ++
src/test/modules/deform_bench/deform_bench.c | 107 ++++++++++++++++++
.../modules/deform_bench/deform_bench.control | 4 +
src/test/modules/deform_bench/meson.build | 22 ++++
src/test/modules/meson.build | 1 +
7 files changed, 167 insertions(+)
create mode 100644 src/test/modules/deform_bench/.gitignore
create mode 100644 src/test/modules/deform_bench/Makefile
create mode 100644 src/test/modules/deform_bench/deform_bench--1.0.sql
create mode 100644 src/test/modules/deform_bench/deform_bench.c
create mode 100644 src/test/modules/deform_bench/deform_bench.control
create mode 100644 src/test/modules/deform_bench/meson.build
diff --git a/src/test/modules/deform_bench/.gitignore b/src/test/modules/deform_bench/.gitignore
new file mode 100644
index 00000000000..5dcb3ff9723
--- /dev/null
+++ b/src/test/modules/deform_bench/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/src/test/modules/deform_bench/Makefile b/src/test/modules/deform_bench/Makefile
new file mode 100644
index 00000000000..b5fc0f7a583
--- /dev/null
+++ b/src/test/modules/deform_bench/Makefile
@@ -0,0 +1,21 @@
+# src/test/modules/deform_bench/Makefile
+
+MODULE_big = deform_bench
+OBJS = deform_bench.o
+
+EXTENSION = deform_bench
+DATA = deform_bench--1.0.sql
+PGFILEDESC = "deform_bench - tuple deform benchmarking"
+
+REGRESS = deform_bench
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = src/test/modules/deform_bench
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/src/test/modules/deform_bench/deform_bench--1.0.sql b/src/test/modules/deform_bench/deform_bench--1.0.sql
new file mode 100644
index 00000000000..492b71dba3b
--- /dev/null
+++ b/src/test/modules/deform_bench/deform_bench--1.0.sql
@@ -0,0 +1,8 @@
+/* deform_bench--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION deform_bench" to load this file. \quit
+
+CREATE FUNCTION deform_bench(tableoid Oid, attnum int[]) RETURNS FLOAT
+AS 'MODULE_PATHNAME', 'deform_bench'
+LANGUAGE C VOLATILE STRICT;
diff --git a/src/test/modules/deform_bench/deform_bench.c b/src/test/modules/deform_bench/deform_bench.c
new file mode 100644
index 00000000000..7838f639bef
--- /dev/null
+++ b/src/test/modules/deform_bench/deform_bench.c
@@ -0,0 +1,107 @@
+/*-------------------------------------------------------------------------
+ *
+ * deform_bench.c
+ *
+ * for benchmarking tuple deformation routines
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include <time.h>
+#include <sys/time.h>
+
+#include "access/heapam.h"
+#include "access/relscan.h"
+#include "catalog/pg_am_d.h"
+#include "catalog/pg_type_d.h"
+#include "funcapi.h"
+#include "miscadmin.h"
+#include "utils/array.h"
+#include "utils/arrayaccess.h"
+#include "utils/builtins.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(deform_bench);
+
+Datum
+deform_bench(PG_FUNCTION_ARGS)
+{
+ Oid tableoid = PG_GETARG_OID(0);
+ ArrayType *array = PG_GETARG_ARRAYTYPE_P(1);
+ TableScanDesc scan;
+ Relation rel;
+ TupleDesc tupdesc;
+ TupleTableSlot *slot;
+ Datum *elem_datums = NULL;
+ bool *elem_nulls = NULL;
+ int elem_count;
+ int *attnums;
+ clock_t start,
+ end;
+
+ rel = relation_open(tableoid, AccessShareLock);
+
+ if (rel->rd_rel->relam != HEAP_TABLE_AM_OID)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("only heap AM is supported")));
+
+ tupdesc = RelationGetDescr(rel);
+ slot = MakeTupleTableSlot(tupdesc, &TTSOpsBufferHeapTuple);
+ scan = table_beginscan_strat(rel, GetActiveSnapshot(), 0, NULL, true, false);
+
+ /*
+ * The array is used to allow callers to define how many atts to deform.
+ * e.g: '{1,10}'::int[] would deform attnum=1, then in a 2nd pass deform
+ * the remainder up to attnum=10. Passing an element as NULL means all
+ * attnums. This allows simulation of incremental deformation. Generally
+ * if you're passing an array with more than 1 element, then the array
+ * should be in ascending order. Doing something like '{10,1}' would mean
+ * we've already deformed 10 attributes and on the 2nd pass there's
+ * nothing to do since attnum=1 was already deformed in the first pass.
+ *
+ * You'll get an ERROR if you pass a number higher than the number of
+ * attributes in the table.
+ */
+ deconstruct_array(array,
+ INT4OID,
+ sizeof(int32),
+ true,
+ 'i',
+ &elem_datums,
+ &elem_nulls,
+ &elem_count);
+
+ attnums = palloc_array(int, elem_count);
+
+ for (int i = 0; i < elem_count; i++)
+ {
+ /* Make a NULL element mean all attributes */
+ if (elem_nulls[i])
+ attnums[i] = tupdesc->natts;
+ else
+ attnums[i] = DatumGetInt32(elem_datums[i]);
+ }
+
+ start = clock();
+
+ while (heap_getnextslot(scan, ForwardScanDirection, slot))
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ /* Deform in stages according to the attnums array */
+ for (int i = 0; i < elem_count; i++)
+ slot_getsomeattrs(slot, attnums[i]);
+ }
+
+ end = clock();
+
+ ExecDropSingleTupleTableSlot(slot);
+ table_endscan(scan);
+ relation_close(rel, AccessShareLock);
+
+
+ /* Returns the number of milliseconds to run the test */
+ PG_RETURN_FLOAT8((double) (end - start) / (CLOCKS_PER_SEC / 1000));
+}
diff --git a/src/test/modules/deform_bench/deform_bench.control b/src/test/modules/deform_bench/deform_bench.control
new file mode 100644
index 00000000000..a2023f9d738
--- /dev/null
+++ b/src/test/modules/deform_bench/deform_bench.control
@@ -0,0 +1,4 @@
+# deform_bench extension
+comment = 'functions for benchmarking tuple deformation'
+default_version = '1.0'
+module_pathname = '$libdir/deform_bench'
diff --git a/src/test/modules/deform_bench/meson.build b/src/test/modules/deform_bench/meson.build
new file mode 100644
index 00000000000..82049585244
--- /dev/null
+++ b/src/test/modules/deform_bench/meson.build
@@ -0,0 +1,22 @@
+# Copyright (c) 2026, PostgreSQL Global Development Group
+
+deform_bench_sources = files(
+ 'deform_bench.c',
+)
+
+if host_system == 'windows'
+ deform_bench_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+ '--NAME', 'deform_bench',
+ '--FILEDESC', 'deform_bench - benchmarking tuple deformation',])
+endif
+
+deform_bench = shared_module('deform_bench',
+ deform_bench_sources,
+ kwargs: pg_test_mod_args,
+)
+test_install_libs += deform_bench
+
+test_install_data += files(
+ 'deform_bench--1.0.sql',
+ 'deform_bench.control',
+)
diff --git a/src/test/modules/meson.build b/src/test/modules/meson.build
index e2b3eef4136..1312f3b4d7b 100644
--- a/src/test/modules/meson.build
+++ b/src/test/modules/meson.build
@@ -2,6 +2,7 @@
subdir('brin')
subdir('commit_ts')
+subdir('deform_bench')
subdir('delay_execution')
subdir('dummy_index_am')
subdir('dummy_seclabel')
--
2.51.0
From ceaf2c78facbc2e118152d419a2c7bcf72a63ad5 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Mon, 16 Feb 2026 14:20:19 +1300
Subject: [PATCH v12 2/6] Allow sibling call optimization in
slot_getsomeattrs_int()
This changes the TupleTableSlotOps contract to make it so the
getsomeattrs() function is in charge of calling
slot_getmissingattrs().
Since this removes all code from slot_getsomeattrs_int() aside from the
getsomeattrs() call itself, we may as well adjust slot_getsomeattrs() so
that it calls getsomeattrs() directly. We leave slot_getsomeattrs_int()
intact as this is still called from the JIT code.
---
src/backend/executor/execTuples.c | 58 ++++++++++++++++---------------
src/include/executor/tuptable.h | 13 ++++---
2 files changed, 38 insertions(+), 33 deletions(-)
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index b768eae9e53..7effe954286 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -73,7 +73,7 @@
static TupleDesc ExecTypeFromTLInternal(List *targetList,
bool skipjunk);
static pg_attribute_always_inline void slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
- int natts);
+ int reqnatts);
static inline void tts_buffer_heap_store_tuple(TupleTableSlot *slot,
HeapTuple tuple,
Buffer buffer,
@@ -1108,7 +1108,10 @@ slot_deform_heap_tuple_internal(TupleTableSlot *slot, HeapTuple tuple,
* slot_deform_heap_tuple
* Given a TupleTableSlot, extract data from the slot's physical tuple
* into its Datum/isnull arrays. Data is extracted up through the
- * natts'th column (caller must ensure this is a legal column number).
+ * reqnatts'th column. If there are insufficient attributes in the given
+ * tuple, then slot_getmissingattrs() is called to populate the
+ * remainder. If reqnatts is above the number of attributes in the
+ * slot's TupleDesc, an error is raised.
*
* This is essentially an incremental version of heap_deform_tuple:
* on each call we extract attributes up to the one needed, without
@@ -1120,21 +1123,23 @@ slot_deform_heap_tuple_internal(TupleTableSlot *slot, HeapTuple tuple,
*/
static pg_attribute_always_inline void
slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
- int natts)
+ int reqnatts)
{
bool hasnulls = HeapTupleHasNulls(tuple);
int attnum;
+ int natts;
uint32 off; /* offset in tuple data */
bool slow; /* can we use/set attcacheoff? */
/* We can only fetch as many attributes as the tuple has. */
- natts = Min(HeapTupleHeaderGetNatts(tuple->t_data), natts);
+ natts = Min(HeapTupleHeaderGetNatts(tuple->t_data), reqnatts);
/*
* Check whether the first call for this tuple, and initialize or restore
* loop state.
*/
attnum = slot->tts_nvalid;
+ slot->tts_nvalid = reqnatts;
if (attnum == 0)
{
/* Start from the first attribute */
@@ -1199,12 +1204,15 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
/*
* Save state for next execution
*/
- slot->tts_nvalid = attnum;
*offp = off;
if (slow)
slot->tts_flags |= TTS_FLAG_SLOW;
else
slot->tts_flags &= ~TTS_FLAG_SLOW;
+
+ /* Fetch any missing attrs and raise an error if reqnatts is invalid. */
+ if (unlikely(attnum < reqnatts))
+ slot_getmissingattrs(slot, attnum, reqnatts);
}
const TupleTableSlotOps TTSOpsVirtual = {
@@ -2058,34 +2066,36 @@ slot_getmissingattrs(TupleTableSlot *slot, int startAttNum, int lastAttNum)
{
AttrMissing *attrmiss = NULL;
+ /* Check for invalid attnums */
+ if (unlikely(lastAttNum > slot->tts_tupleDescriptor->natts))
+ elog(ERROR, "invalid attribute number %d", lastAttNum);
+
if (slot->tts_tupleDescriptor->constr)
attrmiss = slot->tts_tupleDescriptor->constr->missing;
if (!attrmiss)
{
/* no missing values array at all, so just fill everything in as NULL */
- memset(slot->tts_values + startAttNum, 0,
- (lastAttNum - startAttNum) * sizeof(Datum));
- memset(slot->tts_isnull + startAttNum, 1,
- (lastAttNum - startAttNum) * sizeof(bool));
+ for (int attnum = startAttNum; attnum < lastAttNum; attnum++)
+ {
+ slot->tts_values[attnum] = (Datum) 0;
+ slot->tts_isnull[attnum] = true;
+ }
}
else
{
- int missattnum;
-
- /* if there is a missing values array we must process them one by one */
- for (missattnum = startAttNum;
- missattnum < lastAttNum;
- missattnum++)
+ /* use attrmiss to set the missing values */
+ for (int attnum = startAttNum; attnum < lastAttNum; attnum++)
{
- slot->tts_values[missattnum] = attrmiss[missattnum].am_value;
- slot->tts_isnull[missattnum] = !attrmiss[missattnum].am_present;
+ slot->tts_values[attnum] = attrmiss[attnum].am_value;
+ slot->tts_isnull[attnum] = !attrmiss[attnum].am_present;
}
}
}
/*
- * slot_getsomeattrs_int - workhorse for slot_getsomeattrs()
+ * slot_getsomeattrs_int
+ * external function to call getsomeattrs() for use in JIT
*/
void
slot_getsomeattrs_int(TupleTableSlot *slot, int attnum)
@@ -2094,21 +2104,13 @@ slot_getsomeattrs_int(TupleTableSlot *slot, int attnum)
Assert(slot->tts_nvalid < attnum); /* checked in slot_getsomeattrs */
Assert(attnum > 0);
- if (unlikely(attnum > slot->tts_tupleDescriptor->natts))
- elog(ERROR, "invalid attribute number %d", attnum);
-
/* Fetch as many attributes as possible from the underlying tuple. */
slot->tts_ops->getsomeattrs(slot, attnum);
/*
- * If the underlying tuple doesn't have enough attributes, tuple
- * descriptor must have the missing attributes.
+ * Avoid putting new code here as that would prevent the compiler from
+ * using the sibling call optimization for the above function.
*/
- if (unlikely(slot->tts_nvalid < attnum))
- {
- slot_getmissingattrs(slot, slot->tts_nvalid, attnum);
- slot->tts_nvalid = attnum;
- }
}
/* ----------------------------------------------------------------
diff --git a/src/include/executor/tuptable.h b/src/include/executor/tuptable.h
index a2dfd707e78..3b09abbf99f 100644
--- a/src/include/executor/tuptable.h
+++ b/src/include/executor/tuptable.h
@@ -151,10 +151,12 @@ struct TupleTableSlotOps
/*
* Fill up first natts entries of tts_values and tts_isnull arrays with
- * values from the tuple contained in the slot. The function may be called
- * with natts more than the number of attributes available in the tuple,
- * in which case it should set tts_nvalid to the number of returned
- * columns.
+ * values from the tuple contained in the slot and set the slot's
+ * tts_nvalid to natts. The function may be called with an natts value
+ * more than the number of attributes available in the tuple, in which
+ * case the function must call slot_getmissingattrs() to populate the
+ * remaining attributes. The function must raise an ERROR if 'natts' is
+ * higher than the number of attributes in the slot's TupleDesc.
*/
void (*getsomeattrs) (TupleTableSlot *slot, int natts);
@@ -357,8 +359,9 @@ extern void slot_getsomeattrs_int(TupleTableSlot *slot, int attnum);
static inline void
slot_getsomeattrs(TupleTableSlot *slot, int attnum)
{
+ /* Populate slot with attributes up to 'attnum', if it's not already */
if (slot->tts_nvalid < attnum)
- slot_getsomeattrs_int(slot, attnum);
+ slot->tts_ops->getsomeattrs(slot, attnum);
}
/*
--
2.51.0
From 00bc3ecefc20daef125c766fa9ca1d5e5af4d250 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Wed, 21 Jan 2026 15:41:37 +1300
Subject: [PATCH v12 3/6] Add empty TupleDescFinalize() function
Currently does nothing, but will in a future commit.
---
contrib/dblink/dblink.c | 4 ++++
contrib/pg_buffercache/pg_buffercache_pages.c | 2 ++
contrib/pg_visibility/pg_visibility.c | 2 ++
src/backend/access/brin/brin_tuple.c | 1 +
src/backend/access/common/tupdesc.c | 13 +++++++++++++
src/backend/access/gin/ginutil.c | 1 +
src/backend/access/gist/gistscan.c | 1 +
src/backend/access/spgist/spgutils.c | 1 +
src/backend/access/transam/twophase.c | 1 +
src/backend/access/transam/xlogfuncs.c | 1 +
src/backend/backup/basebackup_copy.c | 3 +++
src/backend/catalog/index.c | 2 ++
src/backend/catalog/pg_publication.c | 1 +
src/backend/catalog/toasting.c | 6 ++++++
src/backend/commands/explain.c | 1 +
src/backend/commands/functioncmds.c | 1 +
src/backend/commands/sequence.c | 1 +
src/backend/commands/tablecmds.c | 4 ++++
src/backend/commands/wait.c | 1 +
src/backend/executor/execSRF.c | 2 ++
src/backend/executor/execTuples.c | 4 ++++
src/backend/executor/nodeFunctionscan.c | 2 ++
src/backend/parser/parse_relation.c | 4 +++-
src/backend/parser/parse_target.c | 2 ++
.../replication/libpqwalreceiver/libpqwalreceiver.c | 1 +
src/backend/replication/walsender.c | 5 +++++
src/backend/utils/adt/acl.c | 1 +
src/backend/utils/adt/genfile.c | 1 +
src/backend/utils/adt/lockfuncs.c | 1 +
src/backend/utils/adt/orderedsetaggs.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 5 +++++
src/backend/utils/adt/tsvector_op.c | 1 +
src/backend/utils/cache/relcache.c | 8 ++++++++
src/backend/utils/fmgr/funcapi.c | 6 ++++++
src/backend/utils/misc/guc_funcs.c | 5 +++++
src/include/access/tupdesc.h | 1 +
src/pl/plpgsql/src/pl_comp.c | 2 ++
.../test_custom_stats/test_custom_fixed_stats.c | 1 +
src/test/modules/test_predtest/test_predtest.c | 1 +
39 files changed, 100 insertions(+), 1 deletion(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 2498d80c8e7..4038950a6ef 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -881,6 +881,7 @@ materializeResult(FunctionCallInfo fcinfo, PGconn *conn, PGresult *res)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "status",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
ntuples = 1;
nfields = 1;
}
@@ -1044,6 +1045,7 @@ materializeQueryResult(FunctionCallInfo fcinfo,
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "status",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
attinmeta = TupleDescGetAttInMetadata(tupdesc);
oldcontext = MemoryContextSwitchTo(rsinfo->econtext->ecxt_per_query_memory);
@@ -1529,6 +1531,8 @@ dblink_get_pkey(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 2, "colname",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+
/*
* Generate attribute metadata needed later to produce tuples from raw
* C strings
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 89b86855243..a6b4fb5252b 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -174,6 +174,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
INT4OID, -1, 0);
+ TupleDescFinalize(tupledesc);
fctx->tupdesc = BlessTupleDesc(tupledesc);
/* Allocate NBuffers worth of BufferCachePagesRec records. */
@@ -442,6 +443,7 @@ pg_buffercache_os_pages_internal(FunctionCallInfo fcinfo, bool include_numa)
TupleDescInitEntry(tupledesc, (AttrNumber) 3, "numa_node",
INT4OID, -1, 0);
+ TupleDescFinalize(tupledesc);
fctx->tupdesc = BlessTupleDesc(tupledesc);
fctx->include_numa = include_numa;
diff --git a/contrib/pg_visibility/pg_visibility.c b/contrib/pg_visibility/pg_visibility.c
index 9bc3a784bf7..dfab0b64cf5 100644
--- a/contrib/pg_visibility/pg_visibility.c
+++ b/contrib/pg_visibility/pg_visibility.c
@@ -469,6 +469,8 @@ pg_visibility_tupdesc(bool include_blkno, bool include_pd)
TupleDescInitEntry(tupdesc, ++a, "pd_all_visible", BOOLOID, -1, 0);
Assert(a == maxattr);
+ TupleDescFinalize(tupdesc);
+
return BlessTupleDesc(tupdesc);
}
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 69c233c62eb..742ac089a28 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -84,6 +84,7 @@ brtuple_disk_tupdesc(BrinDesc *brdesc)
MemoryContextSwitchTo(oldcxt);
+ TupleDescFinalize(tupdesc);
brdesc->bd_disktdesc = tupdesc;
}
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index b69d10f0a45..2137385a833 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -221,6 +221,9 @@ CreateTupleDesc(int natts, Form_pg_attribute *attrs)
memcpy(TupleDescAttr(desc, i), attrs[i], ATTRIBUTE_FIXED_PART_SIZE);
populate_compact_attribute(desc, i);
}
+
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -265,6 +268,8 @@ CreateTupleDescCopy(TupleDesc tupdesc)
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -311,6 +316,8 @@ CreateTupleDescTruncatedCopy(TupleDesc tupdesc, int natts)
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -396,6 +403,8 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -438,6 +447,8 @@ TupleDescCopy(TupleDesc dst, TupleDesc src)
* source's refcount would be wrong in any case.)
*/
dst->tdrefcount = -1;
+
+ TupleDescFinalize(dst);
}
/*
@@ -1065,6 +1076,8 @@ BuildDescFromLists(const List *names, const List *types, const List *typmods, co
TupleDescInitEntryCollation(desc, attnum, attcollation);
}
+ TupleDescFinalize(desc);
+
return desc;
}
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index ff927279cc3..fe7b984ff32 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -129,6 +129,7 @@ initGinState(GinState *state, Relation index)
attr->attndims);
TupleDescInitEntryCollation(state->tupdesc[i], (AttrNumber) 2,
attr->attcollation);
+ TupleDescFinalize(state->tupdesc[i]);
}
/*
diff --git a/src/backend/access/gist/gistscan.c b/src/backend/access/gist/gistscan.c
index f23bc4a6757..c65f93abdae 100644
--- a/src/backend/access/gist/gistscan.c
+++ b/src/backend/access/gist/gistscan.c
@@ -201,6 +201,7 @@ gistrescan(IndexScanDesc scan, ScanKey key, int nkeys,
attno - 1)->atttypid,
-1, 0);
}
+ TupleDescFinalize(so->giststate->fetchTupdesc);
scan->xs_hitupdesc = so->giststate->fetchTupdesc;
/* Also create a memory context that will hold the returned tuples */
diff --git a/src/backend/access/spgist/spgutils.c b/src/backend/access/spgist/spgutils.c
index 9f5379b87ac..b246e8127db 100644
--- a/src/backend/access/spgist/spgutils.c
+++ b/src/backend/access/spgist/spgutils.c
@@ -340,6 +340,7 @@ getSpGistTupleDesc(Relation index, SpGistTypeDesc *keyType)
TupleDescCompactAttr(outTupDesc, i)->attcacheoff = -1;
populate_compact_attribute(outTupDesc, spgKeyColumn);
+ TupleDescFinalize(outTupDesc);
}
return outTupDesc;
}
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 5f65fa7b80f..0281b58f40c 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -744,6 +744,7 @@ pg_prepared_xact(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 5, "dbid",
OIDOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
/*
diff --git a/src/backend/access/transam/xlogfuncs.c b/src/backend/access/transam/xlogfuncs.c
index 78543055895..be4cc24f1db 100644
--- a/src/backend/access/transam/xlogfuncs.c
+++ b/src/backend/access/transam/xlogfuncs.c
@@ -427,6 +427,7 @@ pg_walfile_name_offset(PG_FUNCTION_ARGS)
TupleDescInitEntry(resultTupleDesc, (AttrNumber) 2, "file_offset",
INT4OID, -1, 0);
+ TupleDescFinalize(resultTupleDesc);
resultTupleDesc = BlessTupleDesc(resultTupleDesc);
/*
diff --git a/src/backend/backup/basebackup_copy.c b/src/backend/backup/basebackup_copy.c
index 07f58b39d8c..6c3453efd80 100644
--- a/src/backend/backup/basebackup_copy.c
+++ b/src/backend/backup/basebackup_copy.c
@@ -357,6 +357,8 @@ SendXlogRecPtrResult(XLogRecPtr ptr, TimeLineID tli)
*/
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "tli", INT8OID, -1, 0);
+ TupleDescFinalize(tupdesc);
+
/* send RowDescription */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -388,6 +390,7 @@ SendTablespaceList(List *tablespaces)
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 1, "spcoid", OIDOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "spclocation", TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 3, "size", INT8OID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* send RowDescription */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 43de42ce39e..75e97fb394a 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -481,6 +481,8 @@ ConstructTupleDescriptor(Relation heapRelation,
populate_compact_attribute(indexTupDesc, i);
}
+ TupleDescFinalize(indexTupDesc);
+
return indexTupDesc;
}
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index aadc7c202c6..a79157c43bf 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -1322,6 +1322,7 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 4, "qual",
PG_NODE_TREEOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
funcctx->user_fctx = table_infos;
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index c78dcea98c1..078a1cf5127 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -229,6 +229,12 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attcompression = InvalidCompressionMethod;
TupleDescAttr(tupdesc, 2)->attcompression = InvalidCompressionMethod;
+ populate_compact_attribute(tupdesc, 0);
+ populate_compact_attribute(tupdesc, 1);
+ populate_compact_attribute(tupdesc, 2);
+
+ TupleDescFinalize(tupdesc);
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 93918a223b8..5f922c3f5c2 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -281,6 +281,7 @@ ExplainResultDesc(ExplainStmt *stmt)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "QUERY PLAN",
result_type, -1, 0);
+ TupleDescFinalize(tupdesc);
return tupdesc;
}
diff --git a/src/backend/commands/functioncmds.c b/src/backend/commands/functioncmds.c
index 242372b1e68..3afd762e9dc 100644
--- a/src/backend/commands/functioncmds.c
+++ b/src/backend/commands/functioncmds.c
@@ -2424,6 +2424,7 @@ CallStmtResultDesc(CallStmt *stmt)
-1,
0);
}
+ TupleDescFinalize(tupdesc);
}
return tupdesc;
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index e1b808bbb60..551667650ba 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -1808,6 +1808,7 @@ pg_get_sequence_data(PG_FUNCTION_ARGS)
BOOLOID, -1, 0);
TupleDescInitEntry(resultTupleDesc, (AttrNumber) 3, "page_lsn",
LSNOID, -1, 0);
+ TupleDescFinalize(resultTupleDesc);
resultTupleDesc = BlessTupleDesc(resultTupleDesc);
seqrel = try_relation_open(relid, AccessShareLock);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 85242dcc245..80922974970 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -1030,6 +1030,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
}
}
+ TupleDescFinalize(descriptor);
+
/*
* For relations with table AM and partitioned tables, select access
* method to use: an explicitly indicated one, or (in the case of a
@@ -1458,6 +1460,8 @@ BuildDescForRelation(const List *columns)
populate_compact_attribute(desc, attnum - 1);
}
+ TupleDescFinalize(desc);
+
return desc;
}
diff --git a/src/backend/commands/wait.c b/src/backend/commands/wait.c
index 1290df10c6f..8e920a72372 100644
--- a/src/backend/commands/wait.c
+++ b/src/backend/commands/wait.c
@@ -338,5 +338,6 @@ WaitStmtResultDesc(WaitStmt *stmt)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "status",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
return tupdesc;
}
diff --git a/src/backend/executor/execSRF.c b/src/backend/executor/execSRF.c
index a0b111dc0e4..b481e50acfb 100644
--- a/src/backend/executor/execSRF.c
+++ b/src/backend/executor/execSRF.c
@@ -272,6 +272,7 @@ ExecMakeTableFunctionResult(SetExprState *setexpr,
funcrettype,
-1,
0);
+ TupleDescFinalize(tupdesc);
rsinfo.setDesc = tupdesc;
}
MemoryContextSwitchTo(oldcontext);
@@ -776,6 +777,7 @@ init_sexpr(Oid foid, Oid input_collation, Expr *node,
funcrettype,
-1,
0);
+ TupleDescFinalize(tupdesc);
sexpr->funcResultDesc = tupdesc;
sexpr->funcReturnsTuple = false;
}
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 7effe954286..07b248aa5f3 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -2175,6 +2175,8 @@ ExecTypeFromTLInternal(List *targetList, bool skipjunk)
cur_resno++;
}
+ TupleDescFinalize(typeInfo);
+
return typeInfo;
}
@@ -2209,6 +2211,8 @@ ExecTypeFromExprList(List *exprList)
cur_resno++;
}
+ TupleDescFinalize(typeInfo);
+
return typeInfo;
}
diff --git a/src/backend/executor/nodeFunctionscan.c b/src/backend/executor/nodeFunctionscan.c
index 63e605e1f81..feb82d64967 100644
--- a/src/backend/executor/nodeFunctionscan.c
+++ b/src/backend/executor/nodeFunctionscan.c
@@ -414,6 +414,7 @@ ExecInitFunctionScan(FunctionScan *node, EState *estate, int eflags)
TupleDescInitEntryCollation(tupdesc,
(AttrNumber) 1,
exprCollation(funcexpr));
+ TupleDescFinalize(tupdesc);
}
else
{
@@ -485,6 +486,7 @@ ExecInitFunctionScan(FunctionScan *node, EState *estate, int eflags)
0);
}
+ TupleDescFinalize(scan_tupdesc);
Assert(attno == natts);
}
diff --git a/src/backend/parser/parse_relation.c b/src/backend/parser/parse_relation.c
index e003db520de..9c415e166ee 100644
--- a/src/backend/parser/parse_relation.c
+++ b/src/backend/parser/parse_relation.c
@@ -1883,6 +1883,7 @@ addRangeTableEntryForFunction(ParseState *pstate,
TupleDescInitEntryCollation(tupdesc,
(AttrNumber) 1,
exprCollation(funcexpr));
+ TupleDescFinalize(tupdesc);
}
else if (functypclass == TYPEFUNC_RECORD)
{
@@ -1940,6 +1941,7 @@ addRangeTableEntryForFunction(ParseState *pstate,
i++;
}
+ TupleDescFinalize(tupdesc);
/*
* Ensure that the coldeflist defines a legal set of names (no
@@ -2008,7 +2010,7 @@ addRangeTableEntryForFunction(ParseState *pstate,
0);
/* no need to set collation */
}
-
+ TupleDescFinalize(tupdesc);
Assert(natts == totalatts);
}
else
diff --git a/src/backend/parser/parse_target.c b/src/backend/parser/parse_target.c
index 3bcfc1f5e3d..f57c4d41080 100644
--- a/src/backend/parser/parse_target.c
+++ b/src/backend/parser/parse_target.c
@@ -1572,6 +1572,8 @@ expandRecordVariable(ParseState *pstate, Var *var, int levelsup)
}
Assert(lname == NULL && lvar == NULL); /* lists same length? */
+ TupleDescFinalize(tupleDesc);
+
return tupleDesc;
}
diff --git a/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c b/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
index 7c8639b32e9..9f04c9ed25d 100644
--- a/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
+++ b/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
@@ -1073,6 +1073,7 @@ libpqrcv_processTuples(PGresult *pgres, WalRcvExecResult *walres,
for (coln = 0; coln < nRetTypes; coln++)
TupleDescInitEntry(walres->tupledesc, (AttrNumber) coln + 1,
PQfname(pgres, coln), retTypes[coln], -1, 0);
+ TupleDescFinalize(walres->tupledesc);
attinmeta = TupleDescGetAttInMetadata(walres->tupledesc);
/* No point in doing more here if there were no tuples returned. */
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c
index 2cde8ebc729..33a9e8d7f21 100644
--- a/src/backend/replication/walsender.c
+++ b/src/backend/replication/walsender.c
@@ -451,6 +451,7 @@ IdentifySystem(void)
TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 4, "dbname",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -496,6 +497,7 @@ ReadReplicationSlot(ReadReplicationSlotCmd *cmd)
/* TimeLineID is unsigned, so int4 is not wide enough. */
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 3, "restart_tli",
INT8OID, -1, 0);
+ TupleDescFinalize(tupdesc);
memset(nulls, true, READ_REPLICATION_SLOT_COLS * sizeof(bool));
@@ -598,6 +600,7 @@ SendTimeLineHistory(TimeLineHistoryCmd *cmd)
tupdesc = CreateTemplateTupleDesc(2);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 1, "filename", TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "content", TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
TLHistoryFileName(histfname, cmd->timeline);
TLHistoryFilePath(path, cmd->timeline);
@@ -1015,6 +1018,7 @@ StartReplication(StartReplicationCmd *cmd)
INT8OID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "next_tli_startpos",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuple */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -1369,6 +1373,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd)
TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 4, "output_plugin",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
diff --git a/src/backend/utils/adt/acl.c b/src/backend/utils/adt/acl.c
index 071e3f2c49e..e210d6472be 100644
--- a/src/backend/utils/adt/acl.c
+++ b/src/backend/utils/adt/acl.c
@@ -1831,6 +1831,7 @@ aclexplode(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 4, "is_grantable",
BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
/* allocate memory for user context */
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index c083608b1d5..bfb949401d0 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -454,6 +454,7 @@ pg_stat_file(PG_FUNCTION_ARGS)
"creation", TIMESTAMPTZOID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 6,
"isdir", BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
memset(isnull, false, sizeof(isnull));
diff --git a/src/backend/utils/adt/lockfuncs.c b/src/backend/utils/adt/lockfuncs.c
index 9dadd6da672..4481c354fd6 100644
--- a/src/backend/utils/adt/lockfuncs.c
+++ b/src/backend/utils/adt/lockfuncs.c
@@ -146,6 +146,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 16, "waitstart",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
/*
diff --git a/src/backend/utils/adt/orderedsetaggs.c b/src/backend/utils/adt/orderedsetaggs.c
index 3b6da8e36ac..fd8b8676470 100644
--- a/src/backend/utils/adt/orderedsetaggs.c
+++ b/src/backend/utils/adt/orderedsetaggs.c
@@ -233,6 +233,7 @@ ordered_set_startup(FunctionCallInfo fcinfo, bool use_tuples)
-1,
0);
+ TupleDescFinalize(newdesc);
FreeTupleDesc(qstate->tupdesc);
qstate->tupdesc = newdesc;
}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index b1df96e7b0b..0b10da3b180 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -769,6 +769,7 @@ pg_stat_get_backend_subxact(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 2, "subxact_overflow",
BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
if ((local_beentry = pgstat_get_local_beentry_by_proc_number(procNumber)) != NULL)
@@ -1670,6 +1671,7 @@ pg_stat_wal_build_tuple(PgStat_WalCounters wal_counters,
TupleDescInitEntry(tupdesc, (AttrNumber) 6, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
/* Fill values and NULLs */
@@ -2097,6 +2099,7 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 7, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
/* Get statistics about the archiver process */
@@ -2178,6 +2181,7 @@ pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
TIMESTAMPTZOID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 13, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
namestrcpy(&slotname, text_to_cstring(slotname_text));
@@ -2265,6 +2269,7 @@ pg_stat_get_subscription_stats(PG_FUNCTION_ARGS)
INT8OID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 13, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
if (!subentry)
diff --git a/src/backend/utils/adt/tsvector_op.c b/src/backend/utils/adt/tsvector_op.c
index 71c7c7d3b3c..d8dece42b9b 100644
--- a/src/backend/utils/adt/tsvector_op.c
+++ b/src/backend/utils/adt/tsvector_op.c
@@ -651,6 +651,7 @@ tsvector_unnest(PG_FUNCTION_ARGS)
TEXTARRAYOID, -1, 0);
if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
elog(ERROR, "return type must be a row type");
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = tupdesc;
funcctx->user_fctx = PG_GETARG_TSVECTOR_COPY(0);
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a1c88c6b1b6..d27ac216e6d 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -729,6 +729,8 @@ RelationBuildTupleDesc(Relation relation)
pfree(constr);
relation->rd_att->constr = NULL;
}
+
+ TupleDescFinalize(relation->rd_att);
}
/*
@@ -1985,6 +1987,7 @@ formrdesc(const char *relationName, Oid relationReltype,
/* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
TupleDescCompactAttr(relation->rd_att, 0)->attcacheoff = 0;
+ TupleDescFinalize(relation->rd_att);
/* mark not-null status */
if (has_not_null)
@@ -3688,6 +3691,8 @@ RelationBuildLocalRelation(const char *relname,
for (i = 0; i < natts; i++)
TupleDescAttr(rel->rd_att, i)->attrelid = relid;
+ TupleDescFinalize(rel->rd_att);
+
rel->rd_rel->reltablespace = reltablespace;
if (mapped_relation)
@@ -4443,6 +4448,7 @@ BuildHardcodedDescriptor(int natts, const FormData_pg_attribute *attrs)
/* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
TupleDescCompactAttr(result, 0)->attcacheoff = 0;
+ TupleDescFinalize(result);
/* Note: we don't bother to set up a TupleConstr entry */
@@ -6291,6 +6297,8 @@ load_relcache_init_file(bool shared)
populate_compact_attribute(rel->rd_att, i);
}
+ TupleDescFinalize(rel->rd_att);
+
/* next read the access method specific field */
if (fread(&len, 1, sizeof(len), fp) != sizeof(len))
goto read_failed;
diff --git a/src/backend/utils/fmgr/funcapi.c b/src/backend/utils/fmgr/funcapi.c
index 8a934ea8dca..516d02cfb82 100644
--- a/src/backend/utils/fmgr/funcapi.c
+++ b/src/backend/utils/fmgr/funcapi.c
@@ -340,6 +340,8 @@ get_expr_result_type(Node *expr,
exprCollation(col));
i++;
}
+ TupleDescFinalize(tupdesc);
+
if (resultTypeId)
*resultTypeId = rexpr->row_typeid;
if (resultTupleDesc)
@@ -1044,6 +1046,7 @@ resolve_polymorphic_tupdesc(TupleDesc tupdesc, oidvector *declared_args,
}
}
+ TupleDescFinalize(tupdesc);
return true;
}
@@ -1853,6 +1856,8 @@ build_function_result_tupdesc_d(char prokind,
0);
}
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -1970,6 +1975,7 @@ TypeGetTupleDesc(Oid typeoid, List *colaliases)
typeoid,
-1,
0);
+ TupleDescFinalize(tupdesc);
}
else if (functypclass == TYPEFUNC_RECORD)
{
diff --git a/src/backend/utils/misc/guc_funcs.c b/src/backend/utils/misc/guc_funcs.c
index 8524dd3a981..472cb5393ce 100644
--- a/src/backend/utils/misc/guc_funcs.c
+++ b/src/backend/utils/misc/guc_funcs.c
@@ -444,6 +444,7 @@ GetPGVariableResultDesc(const char *name)
TupleDescInitEntry(tupdesc, (AttrNumber) 1, varname,
TEXTOID, -1, 0);
}
+ TupleDescFinalize(tupdesc);
return tupdesc;
}
@@ -465,6 +466,7 @@ ShowGUCConfigOption(const char *name, DestReceiver *dest)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 1, varname,
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -499,6 +501,7 @@ ShowAllGUCConfig(DestReceiver *dest)
TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 3, "description",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -934,6 +937,8 @@ show_all_settings(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 17, "pending_restart",
BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+
/*
* Generate attribute metadata needed later to produce tuples from raw
* C strings
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index d46cdbf7a3c..595413dbbc5 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -195,6 +195,7 @@ extern TupleDesc CreateTupleDescTruncatedCopy(TupleDesc tupdesc, int natts);
extern TupleDesc CreateTupleDescCopyConstr(TupleDesc tupdesc);
+#define TupleDescFinalize(d) ((void) 0)
#define TupleDescSize(src) \
(offsetof(struct TupleDescData, compact_attrs) + \
(src)->natts * sizeof(CompactAttribute) + \
diff --git a/src/pl/plpgsql/src/pl_comp.c b/src/pl/plpgsql/src/pl_comp.c
index 5ecc7766757..b72c963b3be 100644
--- a/src/pl/plpgsql/src/pl_comp.c
+++ b/src/pl/plpgsql/src/pl_comp.c
@@ -1912,6 +1912,8 @@ build_row_from_vars(PLpgSQL_variable **vars, int numvars)
TupleDescInitEntryCollation(row->rowtupdesc, i + 1, typcoll);
}
+ TupleDescFinalize(row->rowtupdesc);
+
return row;
}
diff --git a/src/test/modules/test_custom_stats/test_custom_fixed_stats.c b/src/test/modules/test_custom_stats/test_custom_fixed_stats.c
index 485e08e5c19..f9e7c717280 100644
--- a/src/test/modules/test_custom_stats/test_custom_fixed_stats.c
+++ b/src/test/modules/test_custom_stats/test_custom_fixed_stats.c
@@ -206,6 +206,7 @@ test_custom_stats_fixed_report(PG_FUNCTION_ARGS)
INT8OID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 2, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
values[0] = Int64GetDatum(stats->numcalls);
diff --git a/src/test/modules/test_predtest/test_predtest.c b/src/test/modules/test_predtest/test_predtest.c
index 679a5de456d..48ca2a4ea70 100644
--- a/src/test/modules/test_predtest/test_predtest.c
+++ b/src/test/modules/test_predtest/test_predtest.c
@@ -230,6 +230,7 @@ test_predtest(PG_FUNCTION_ARGS)
"s_r_holds", BOOLOID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 8,
"w_r_holds", BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
tupdesc = BlessTupleDesc(tupdesc);
values[0] = BoolGetDatum(strong_implied_by);
--
2.51.0
From d46c106325c6dd7f456fe5d28d47a83cc295a9c7 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Tue, 31 Dec 2024 09:19:24 +1300
Subject: [PATCH v12 4/6] Optimize tuple deformation
This commit includes various optimizations to improve the performance of
tuple deformation.
We now precalculate CompactAttribute's attcacheoff, which allows us to
remove the code from the deform routines which was setting the
attcacheoff. Setting the attcacheoff is handled by TupleDescFinalize(),
which must be called before the TupleDesc is used for anything. Having
this TupleDescFinalize() function means we can store the first
attribute in the TupleDesc which does not have an offset cached. That
allows us to add a dedicated deforming loop to deform all attributes up
to the final one with an attcacheoff set, or up to the first NULL
attribute, whichever comes first.
We also record the maximum attribute number which is guaranteed to exist
in the tuple, that is, has a NOT NULL constraint and isn't an
atthasmissing attribute. When deforming only attributes prior to the
guaranteed attnum, we've no need to access the tuple's natt count. As an
additional optimization, we only count fixed-width columns when
calculating the maximum guaranteed column as this eliminates the need to
emit code to fetch byref types in the deformation loop for guaranteed
attributes.
Some locations in the code deform tuples that have yet to go through NOT
NULL constraint validation. We're unable to perform the guaranteed
attribute optimization when that's the case. The optimization is opt-in
via the TupleTableSlot using the TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS
flag.
This commit also adds a more efficient way of populating the isnull
array by using a bit-wise trick which performs multiplication on the
inverse of the tuple's bitmap byte and masking out all but the lower bit
of each of the boolean's byte. This results in much more optimal code
when compared to determining the NULLness via att_isnull(). 8 isnull
elements are processed at once using this method, which means we need to
round the tts_isnull array size up to the next 8 bytes. The palloc code
does this anyway, but the round-up needed to be formalized so as not to
overwrite the sentinel byte in debug builds.
---
src/backend/access/common/heaptuple.c | 360 +++++++--------
src/backend/access/common/indextuple.c | 363 ++++++---------
src/backend/access/common/tupdesc.c | 51 +++
src/backend/access/spgist/spgutils.c | 3 -
src/backend/executor/execMain.c | 8 +-
src/backend/executor/execTuples.c | 417 ++++++++++--------
src/backend/executor/execUtils.c | 2 +-
src/backend/executor/nodeAgg.c | 2 +-
src/backend/executor/nodeBitmapHeapscan.c | 5 +-
src/backend/executor/nodeCtescan.c | 2 +-
src/backend/executor/nodeCustom.c | 4 +-
src/backend/executor/nodeForeignscan.c | 4 +-
src/backend/executor/nodeFunctionscan.c | 2 +-
src/backend/executor/nodeIndexonlyscan.c | 7 +-
src/backend/executor/nodeIndexscan.c | 5 +-
.../executor/nodeNamedtuplestorescan.c | 2 +-
src/backend/executor/nodeSamplescan.c | 6 +-
src/backend/executor/nodeSeqscan.c | 3 +-
src/backend/executor/nodeSubqueryscan.c | 3 +-
src/backend/executor/nodeTableFuncscan.c | 2 +-
src/backend/executor/nodeTidrangescan.c | 5 +-
src/backend/executor/nodeTidscan.c | 5 +-
src/backend/executor/nodeValuesscan.c | 2 +-
src/backend/executor/nodeWorktablescan.c | 2 +-
src/backend/jit/llvm/llvmjit_deform.c | 6 -
src/backend/replication/pgoutput/pgoutput.c | 4 +-
src/backend/utils/cache/relcache.c | 12 -
src/include/access/tupdesc.h | 20 +-
src/include/access/tupmacs.h | 224 +++++++++-
src/include/executor/executor.h | 3 +-
src/include/executor/tuptable.h | 28 +-
src/test/modules/deform_bench/deform_bench.c | 4 +-
32 files changed, 902 insertions(+), 664 deletions(-)
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 11bec20e82e..b2ac7fef35b 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -498,19 +498,7 @@ heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc)
* nocachegetattr
*
* This only gets called from fastgetattr(), in cases where we
- * can't use a cacheoffset and the value is not null.
- *
- * This caches attribute offsets in the attribute descriptor.
- *
- * An alternative way to speed things up would be to cache offsets
- * with the tuple, but that seems more difficult unless you take
- * the storage hit of actually putting those offsets into the
- * tuple you send to disk. Yuck.
- *
- * This scheme will be slightly slower than that, but should
- * perform well for queries which hit large #'s of tuples. After
- * you cache the offsets once, examining all the other tuples using
- * the same attribute descriptor will go much quicker. -cim 5/4/91
+ * can't use the attcacheoff and the value is not null.
*
* NOTE: if you need to change this code, see also heap_deform_tuple.
* Also see nocache_index_getattr, which is the same code for index
@@ -522,194 +510,125 @@ nocachegetattr(HeapTuple tup,
int attnum,
TupleDesc tupleDesc)
{
+ CompactAttribute *cattr;
HeapTupleHeader td = tup->t_data;
char *tp; /* ptr to data part of tuple */
bits8 *bp = td->t_bits; /* ptr to null bitmap in tuple */
- bool slow = false; /* do we have to walk attrs? */
int off; /* current offset within data */
+ int startAttr;
+ int firstNullAttr;
+ int i;
+ bool hasnulls = HeapTupleHasNulls(tup);
- /* ----------------
- * Three cases:
- *
- * 1: No nulls and no variable-width attributes.
- * 2: Has a null or a var-width AFTER att.
- * 3: Has nulls or var-widths BEFORE att.
- * ----------------
- */
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
attnum--;
- if (!HeapTupleNoNulls(tup))
+ /*
+ * To minimize the number of attributes we need to look at, start walking
+ * the tuple at the attribute with the highest attcacheoff prior to attnum
+ * or the first NULL attribute prior to attnum, whichever comes first.
+ */
+ if (hasnulls)
+ firstNullAttr = first_null_attr(bp, attnum);
+ else
+ firstNullAttr = attnum;
+
+ if (tupleDesc->firstNonCachedOffsetAttr > 0)
{
/*
- * there's a null somewhere in the tuple
- *
- * check to see if any preceding bits are null...
+ * Start at the highest attcacheoff attribute with no NULLs in prior
+ * attributes.
*/
- int byte = attnum >> 3;
- int finalbit = attnum & 0x07;
-
- /* check for nulls "before" final bit of last byte */
- if ((~bp[byte]) & ((1 << finalbit) - 1))
- slow = true;
- else
- {
- /* check for nulls in any "earlier" bytes */
- int i;
-
- for (i = 0; i < byte; i++)
- {
- if (bp[i] != 0xFF)
- {
- slow = true;
- break;
- }
- }
- }
+ startAttr = Min(tupleDesc->firstNonCachedOffsetAttr - 1, firstNullAttr);
+ off = TupleDescCompactAttr(tupleDesc, startAttr)->attcacheoff;
+ }
+ else
+ {
+ /* Otherwise, start at the beginning... */
+ startAttr = 0;
+ off = 0;
}
tp = (char *) td + td->t_hoff;
- if (!slow)
+ /*
+ * Calculate 'off' up to the first NULL attr. We use two cheaper loops
+ * when the tuple has no variable-width columns. When variable-width
+ * columns exists, we use att_addlength_pointer() to move the offset
+ * beyond the current attribute.
+ */
+ if (!HeapTupleHasVarWidth(tup))
{
- CompactAttribute *att;
-
- /*
- * If we get here, there are no nulls up to and including the target
- * attribute. If we have a cached offset, we can use it.
- */
- att = TupleDescCompactAttr(tupleDesc, attnum);
- if (att->attcacheoff >= 0)
- return fetchatt(att, tp + att->attcacheoff);
-
- /*
- * Otherwise, check for non-fixed-length attrs up to and including
- * target. If there aren't any, it's safe to cheaply initialize the
- * cached offsets for these attrs.
- */
- if (HeapTupleHasVarWidth(tup))
+ for (i = startAttr; i < firstNullAttr; i++)
{
- int j;
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- for (j = 0; j <= attnum; j++)
- {
- if (TupleDescCompactAttr(tupleDesc, j)->attlen <= 0)
- {
- slow = true;
- break;
- }
- }
+ off = att_nominal_alignby(off, cattr->attalignby);
+ off += cattr->attlen;
}
- }
-
- if (!slow)
- {
- int natts = tupleDesc->natts;
- int j = 1;
-
- /*
- * If we get here, we have a tuple with no nulls or var-widths up to
- * and including the target attribute, so we can use the cached offset
- * ... only we don't have it yet, or we'd not have got here. Since
- * it's cheap to compute offsets for fixed-width columns, we take the
- * opportunity to initialize the cached offsets for *all* the leading
- * fixed-width columns, in hope of avoiding future visits to this
- * routine.
- */
- TupleDescCompactAttr(tupleDesc, 0)->attcacheoff = 0;
-
- /* we might have set some offsets in the slow path previously */
- while (j < natts && TupleDescCompactAttr(tupleDesc, j)->attcacheoff > 0)
- j++;
-
- off = TupleDescCompactAttr(tupleDesc, j - 1)->attcacheoff +
- TupleDescCompactAttr(tupleDesc, j - 1)->attlen;
- for (; j < natts; j++)
+ for (; i < attnum; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, j);
+ if (att_isnull(i, bp))
+ continue;
- if (att->attlen <= 0)
- break;
-
- off = att_nominal_alignby(off, att->attalignby);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- att->attcacheoff = off;
-
- off += att->attlen;
+ off = att_nominal_alignby(off, cattr->attalignby);
+ off += cattr->attlen;
}
-
- Assert(j > attnum);
-
- off = TupleDescCompactAttr(tupleDesc, attnum)->attcacheoff;
}
else
{
- bool usecache = true;
- int i;
-
- /*
- * Now we know that we have to walk the tuple CAREFULLY. But we still
- * might be able to cache some offsets for next time.
- *
- * Note - This loop is a little tricky. For each non-null attribute,
- * we have to first account for alignment padding before the attr,
- * then advance over the attr based on its length. Nulls have no
- * storage and no alignment padding either. We can use/set
- * attcacheoff until we reach either a null or a var-width attribute.
- */
- off = 0;
- for (i = 0;; i++) /* loop exit is at "break" */
+ for (i = startAttr; i < firstNullAttr; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, i);
+ int attlen;
- if (HeapTupleHasNulls(tup) && att_isnull(i, bp))
- {
- usecache = false;
- continue; /* this cannot be the target att */
- }
+ cattr = TupleDescCompactAttr(tupleDesc, i);
+ attlen = cattr->attlen;
- /* If we know the next offset, we can skip the rest */
- if (usecache && att->attcacheoff >= 0)
- off = att->attcacheoff;
- else if (att->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be
- * no pad bytes in any case: then the offset will be valid for
- * either an aligned or unaligned value.
- */
- if (usecache &&
- off == att_nominal_alignby(off, att->attalignby))
- att->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, att->attalignby, -1,
- tp + off);
- usecache = false;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, att->attalignby);
+ /*
+ * cstrings don't exist in heap tuples. Use pg_assume to instruct
+ * the compiler not to emit the cstring-related code in
+ * att_addlength_pointer().
+ */
+ pg_assume(attlen > 0 || attlen == -1);
- if (usecache)
- att->attcacheoff = off;
- }
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ attlen,
+ tp + off);
+ off = att_addlength_pointer(off, attlen, tp + off);
+ }
- if (i == attnum)
- break;
+ for (; i < attnum; i++)
+ {
+ int attlen;
- off = att_addlength_pointer(off, att->attlen, tp + off);
+ if (att_isnull(i, bp))
+ continue;
- if (usecache && att->attlen <= 0)
- usecache = false;
+ cattr = TupleDescCompactAttr(tupleDesc, i);
+ attlen = cattr->attlen;
+
+ /* As above, heaptuples have no cstrings */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ off = att_pointer_alignby(off, cattr->attalignby, attlen,
+ tp + off);
+ off = att_addlength_pointer(off, attlen, tp + off);
}
}
- return fetchatt(TupleDescCompactAttr(tupleDesc, attnum), tp + off);
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+
+ return fetchatt(cattr, tp + off);
}
/* ----------------
@@ -1347,6 +1266,7 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,
Datum *values, bool *isnull)
{
HeapTupleHeader tup = tuple->t_data;
+ CompactAttribute *cattr;
bool hasnulls = HeapTupleHasNulls(tuple);
int tdesc_natts = tupleDesc->natts;
int natts; /* number of atts to extract */
@@ -1354,70 +1274,98 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,
char *tp; /* ptr to tuple data */
uint32 off; /* offset in tuple data */
bits8 *bp = tup->t_bits; /* ptr to null bitmap in tuple */
- bool slow = false; /* can we use/set attcacheoff? */
+ int firstNonCacheOffsetAttr;
+ int firstNullAttr;
natts = HeapTupleHeaderGetNatts(tup);
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
+
/*
* In inheritance situations, it is possible that the given tuple actually
* has more fields than the caller is expecting. Don't run off the end of
* the caller's arrays.
*/
natts = Min(natts, tdesc_natts);
+ firstNonCacheOffsetAttr = Min(tupleDesc->firstNonCachedOffsetAttr, natts);
+
+ if (hasnulls)
+ {
+ firstNullAttr = first_null_attr(bp, natts);
+
+ /*
+ * XXX: it'd be nice to use populate_isnull_array() here, but that
+ * requires that the isnull array's size is rounded up to the next
+ * multiple of 8. Doing that would require adjusting many locations
+ * that allocate the array.
+ */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, firstNullAttr);
+ }
+ else
+ firstNullAttr = natts;
tp = (char *) tup + tup->t_hoff;
+ attnum = 0;
- off = 0;
+ if (firstNonCacheOffsetAttr > 0)
+ {
+#ifdef USE_ASSERT_CHECKING
+ /* In Assert enabled builds, verify attcacheoff is correct */
+ int offcheck = 0;
+#endif
+ do
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ off = cattr->attcacheoff;
- for (attnum = 0; attnum < natts; attnum++)
+#ifdef USE_ASSERT_CHECKING
+ offcheck = att_nominal_alignby(offcheck, cattr->attalignby);
+ Assert(offcheck == cattr->attcacheoff);
+ offcheck += cattr->attlen;
+#endif
+
+ values[attnum] = fetch_att_noerr(tp + off,
+ cattr->attbyval,
+ cattr->attlen);
+ } while (++attnum < firstNonCacheOffsetAttr);
+ off += cattr->attlen;
+ }
+ else
+ off = 0;
+
+ for (; attnum < firstNullAttr; attnum++)
{
- CompactAttribute *thisatt = TupleDescCompactAttr(tupleDesc, attnum);
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
+ }
+
+ for (; attnum < natts; attnum++)
+ {
+ Assert(hasnulls);
- if (hasnulls && att_isnull(attnum, bp))
+ if (att_isnull(attnum, bp))
{
values[attnum] = (Datum) 0;
isnull[attnum] = true;
- slow = true; /* can't use attcacheoff anymore */
continue;
}
isnull[attnum] = false;
-
- if (!slow && thisatt->attcacheoff >= 0)
- off = thisatt->attcacheoff;
- else if (thisatt->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be no
- * pad bytes in any case: then the offset will be valid for either
- * an aligned or unaligned value.
- */
- if (!slow &&
- off == att_nominal_alignby(off, thisatt->attalignby))
- thisatt->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, thisatt->attalignby, -1,
- tp + off);
- slow = true;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, thisatt->attalignby);
-
- if (!slow)
- thisatt->attcacheoff = off;
- }
-
- values[attnum] = fetchatt(thisatt, tp + off);
-
- off = att_addlength_pointer(off, thisatt->attlen, tp + off);
-
- if (thisatt->attlen <= 0)
- slow = true; /* can't use attcacheoff anymore */
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
}
/*
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index d6350201e01..8c410853191 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -223,18 +223,6 @@ index_form_tuple_context(TupleDesc tupleDescriptor,
*
* This gets called from index_getattr() macro, and only in cases
* where we can't use cacheoffset and the value is not null.
- *
- * This caches attribute offsets in the attribute descriptor.
- *
- * An alternative way to speed things up would be to cache offsets
- * with the tuple, but that seems more difficult unless you take
- * the storage hit of actually putting those offsets into the
- * tuple you send to disk. Yuck.
- *
- * This scheme will be slightly slower than that, but should
- * perform well for queries which hit large #'s of tuples. After
- * you cache the offsets once, examining all the other tuples using
- * the same attribute descriptor will go much quicker. -cim 5/4/91
* ----------------
*/
Datum
@@ -242,205 +230,124 @@ nocache_index_getattr(IndexTuple tup,
int attnum,
TupleDesc tupleDesc)
{
+ CompactAttribute *cattr;
char *tp; /* ptr to data part of tuple */
bits8 *bp = NULL; /* ptr to null bitmap in tuple */
- bool slow = false; /* do we have to walk attrs? */
int data_off; /* tuple data offset */
int off; /* current offset within data */
+ int startAttr;
+ int firstNullAttr;
+ bool hasnulls = IndexTupleHasNulls(tup);
+ int i;
- /* ----------------
- * Three cases:
- *
- * 1: No nulls and no variable-width attributes.
- * 2: Has a null or a var-width AFTER att.
- * 3: Has nulls or var-widths BEFORE att.
- * ----------------
- */
-
- data_off = IndexInfoFindDataOffset(tup->t_info);
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
attnum--;
- if (IndexTupleHasNulls(tup))
- {
- /*
- * there's a null somewhere in the tuple
- *
- * check to see if desired att is null
- */
+ data_off = IndexInfoFindDataOffset(tup->t_info);
+ tp = (char *) tup + data_off;
- /* XXX "knows" t_bits are just after fixed tuple header! */
+ /*
+ * To minimize the number of attributes we need to look at, start walking
+ * the tuple at the attribute with the highest attcacheoff prior to attnum
+ * or the first NULL attribute prior to attnum, whichever comes first.
+ */
+ if (hasnulls)
+ {
bp = (bits8 *) ((char *) tup + sizeof(IndexTupleData));
-
- /*
- * Now check to see if any preceding bits are null...
- */
- {
- int byte = attnum >> 3;
- int finalbit = attnum & 0x07;
-
- /* check for nulls "before" final bit of last byte */
- if ((~bp[byte]) & ((1 << finalbit) - 1))
- slow = true;
- else
- {
- /* check for nulls in any "earlier" bytes */
- int i;
-
- for (i = 0; i < byte; i++)
- {
- if (bp[i] != 0xFF)
- {
- slow = true;
- break;
- }
- }
- }
- }
+ firstNullAttr = first_null_attr(bp, attnum);
}
+ else
+ firstNullAttr = attnum;
- tp = (char *) tup + data_off;
-
- if (!slow)
+ if (tupleDesc->firstNonCachedOffsetAttr > 0)
{
- CompactAttribute *att;
-
- /*
- * If we get here, there are no nulls up to and including the target
- * attribute. If we have a cached offset, we can use it.
- */
- att = TupleDescCompactAttr(tupleDesc, attnum);
- if (att->attcacheoff >= 0)
- return fetchatt(att, tp + att->attcacheoff);
-
/*
- * Otherwise, check for non-fixed-length attrs up to and including
- * target. If there aren't any, it's safe to cheaply initialize the
- * cached offsets for these attrs.
+ * Start at the highest attcacheoff attribute with no NULLs in prior
+ * attributes.
*/
- if (IndexTupleHasVarwidths(tup))
- {
- int j;
-
- for (j = 0; j <= attnum; j++)
- {
- if (TupleDescCompactAttr(tupleDesc, j)->attlen <= 0)
- {
- slow = true;
- break;
- }
- }
- }
+ startAttr = Min(tupleDesc->firstNonCachedOffsetAttr - 1, firstNullAttr);
+ off = TupleDescCompactAttr(tupleDesc, startAttr)->attcacheoff;
}
-
- if (!slow)
+ else
{
- int natts = tupleDesc->natts;
- int j = 1;
-
- /*
- * If we get here, we have a tuple with no nulls or var-widths up to
- * and including the target attribute, so we can use the cached offset
- * ... only we don't have it yet, or we'd not have got here. Since
- * it's cheap to compute offsets for fixed-width columns, we take the
- * opportunity to initialize the cached offsets for *all* the leading
- * fixed-width columns, in hope of avoiding future visits to this
- * routine.
- */
- TupleDescCompactAttr(tupleDesc, 0)->attcacheoff = 0;
+ /* Otherwise, start at the beginning... */
+ startAttr = 0;
+ off = 0;
+ }
- /* we might have set some offsets in the slow path previously */
- while (j < natts && TupleDescCompactAttr(tupleDesc, j)->attcacheoff > 0)
- j++;
+ /*
+ * Calculate 'off' up to the first NULL attr. We use two cheaper loops
+ * when the tuple has no variable-width columns. When variable-width
+ * columns exists, we use att_addlength_pointer() to move the offset
+ * beyond the current attribute.
+ */
+ if (IndexTupleHasVarwidths(tup))
+ {
+ /* Calculate the offset up until the first NULL */
+ for (i = startAttr; i < firstNullAttr; i++)
+ {
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- off = TupleDescCompactAttr(tupleDesc, j - 1)->attcacheoff +
- TupleDescCompactAttr(tupleDesc, j - 1)->attlen;
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+ off = att_addlength_pointer(off, cattr->attlen, tp + off);
+ }
- for (; j < natts; j++)
+ /* Calculate the offset for any remaining columns. */
+ for (; i < attnum; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, j);
+ Assert(hasnulls);
- if (att->attlen <= 0)
- break;
+ if (att_isnull(i, bp))
+ continue;
- off = att_nominal_alignby(off, att->attalignby);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- att->attcacheoff = off;
-
- off += att->attlen;
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+ off = att_addlength_pointer(off, cattr->attlen, tp + off);
}
-
- Assert(j > attnum);
-
- off = TupleDescCompactAttr(tupleDesc, attnum)->attcacheoff;
}
else
{
- bool usecache = true;
- int i;
+ /* Handle tuples with only fixed-width attributes */
- /*
- * Now we know that we have to walk the tuple CAREFULLY. But we still
- * might be able to cache some offsets for next time.
- *
- * Note - This loop is a little tricky. For each non-null attribute,
- * we have to first account for alignment padding before the attr,
- * then advance over the attr based on its length. Nulls have no
- * storage and no alignment padding either. We can use/set
- * attcacheoff until we reach either a null or a var-width attribute.
- */
- off = 0;
- for (i = 0;; i++) /* loop exit is at "break" */
+ /* Calculate the offset up until the first NULL */
+ for (i = startAttr; i < firstNullAttr; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, i);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- if (IndexTupleHasNulls(tup) && att_isnull(i, bp))
- {
- usecache = false;
- continue; /* this cannot be the target att */
- }
-
- /* If we know the next offset, we can skip the rest */
- if (usecache && att->attcacheoff >= 0)
- off = att->attcacheoff;
- else if (att->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be
- * no pad bytes in any case: then the offset will be valid for
- * either an aligned or unaligned value.
- */
- if (usecache &&
- off == att_nominal_alignby(off, att->attalignby))
- att->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, att->attalignby, -1,
- tp + off);
- usecache = false;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, att->attalignby);
+ Assert(cattr->attlen > 0);
+ off = att_nominal_alignby(off, cattr->attalignby);
+ off += cattr->attlen;
+ }
- if (usecache)
- att->attcacheoff = off;
- }
+ /* Calculate the offset for any remaining columns. */
+ for (; i < attnum; i++)
+ {
+ Assert(hasnulls);
- if (i == attnum)
- break;
+ if (att_isnull(i, bp))
+ continue;
- off = att_addlength_pointer(off, att->attlen, tp + off);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- if (usecache && att->attlen <= 0)
- usecache = false;
+ Assert(cattr->attlen > 0);
+ off = att_nominal_alignby(off, cattr->attalignby);
+ off += cattr->attlen;
}
}
- return fetchatt(TupleDescCompactAttr(tupleDesc, attnum), tp + off);
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ off = att_pointer_alignby(off, cattr->attalignby,
+ cattr->attlen, tp + off);
+ return fetchatt(cattr, tp + off);
}
/*
@@ -480,63 +387,87 @@ index_deform_tuple_internal(TupleDesc tupleDescriptor,
Datum *values, bool *isnull,
char *tp, bits8 *bp, int hasnulls)
{
+ CompactAttribute *cattr;
int natts = tupleDescriptor->natts; /* number of atts to extract */
- int attnum;
- int off = 0; /* offset in tuple data */
- bool slow = false; /* can we use/set attcacheoff? */
+ int attnum = 0;
+ uint32 off = 0; /* offset in tuple data */
+ int firstNonCacheOffsetAttr;
+ int firstNullAttr;
/* Assert to protect callers who allocate fixed-size arrays */
Assert(natts <= INDEX_MAX_KEYS);
- for (attnum = 0; attnum < natts; attnum++)
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDescriptor->firstNonCachedOffsetAttr >= 0);
+
+ firstNonCacheOffsetAttr = Min(tupleDescriptor->firstNonCachedOffsetAttr, natts);
+
+ if (hasnulls)
+ {
+ firstNullAttr = first_null_attr(bp, natts);
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, firstNullAttr);
+ }
+ else
+ firstNullAttr = natts;
+
+ if (firstNonCacheOffsetAttr > 0)
{
- CompactAttribute *thisatt = TupleDescCompactAttr(tupleDescriptor, attnum);
+#ifdef USE_ASSERT_CHECKING
+ /* In Assert enabled builds, verify attcacheoff is correct */
+ off = 0;
+#endif
- if (hasnulls && att_isnull(attnum, bp))
+ do
{
- values[attnum] = (Datum) 0;
- isnull[attnum] = true;
- slow = true; /* can't use attcacheoff anymore */
- continue;
- }
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDescriptor, attnum);
- isnull[attnum] = false;
+#ifdef USE_ASSERT_CHECKING
+ off = att_nominal_alignby(off, cattr->attalignby);
+ Assert(off == cattr->attcacheoff);
+ off += cattr->attlen;
+#endif
- if (!slow && thisatt->attcacheoff >= 0)
- off = thisatt->attcacheoff;
- else if (thisatt->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be no
- * pad bytes in any case: then the offset will be valid for either
- * an aligned or unaligned value.
- */
- if (!slow &&
- off == att_nominal_alignby(off, thisatt->attalignby))
- thisatt->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, thisatt->attalignby, -1,
- tp + off);
- slow = true;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, thisatt->attalignby);
+ values[attnum] = fetch_att_noerr(tp + cattr->attcacheoff, cattr->attbyval,
+ cattr->attlen);
+ } while (++attnum < firstNonCacheOffsetAttr);
- if (!slow)
- thisatt->attcacheoff = off;
- }
+ off = cattr->attcacheoff + cattr->attlen;
+ }
- values[attnum] = fetchatt(thisatt, tp + off);
+ for (; attnum < firstNullAttr; attnum++)
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDescriptor, attnum);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
+ }
- off = att_addlength_pointer(off, thisatt->attlen, tp + off);
+ for (; attnum < natts; attnum++)
+ {
+ Assert(hasnulls);
- if (thisatt->attlen <= 0)
- slow = true; /* can't use attcacheoff anymore */
+ if (att_isnull(attnum, bp))
+ {
+ values[attnum] = (Datum) 0;
+ isnull[attnum] = true;
+ continue;
+ }
+
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDescriptor, attnum);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
}
}
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 2137385a833..c68561337d7 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -197,6 +197,10 @@ CreateTemplateTupleDesc(int natts)
desc->tdtypmod = -1;
desc->tdrefcount = -1; /* assume not reference-counted */
+ /* This will be set to the correct value by TupleDescFinalize() */
+ desc->firstNonCachedOffsetAttr = -1;
+ desc->firstNonGuaranteedAttr = -1;
+
return desc;
}
@@ -457,6 +461,9 @@ TupleDescCopy(TupleDesc dst, TupleDesc src)
* descriptor to another.
*
* !!! Constraints and defaults are not copied !!!
+ *
+ * The caller must take care of calling TupleDescFinalize() on 'dst' once all
+ * TupleDesc changes have been made.
*/
void
TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
@@ -489,6 +496,50 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
populate_compact_attribute(dst, dstAttno - 1);
}
+/*
+ * TupleDescFinalize
+ * Finalize the given TupleDesc. This must be called after the
+ * attributes arrays have been populated or adjusted by any code.
+ *
+ * Must be called after populate_compact_attribute() and before
+ * BlessTupleDesc().
+ */
+void
+TupleDescFinalize(TupleDesc tupdesc)
+{
+ int firstNonCachedOffsetAttr = 0;
+ int firstNonGuaranteedAttr = tupdesc->natts;
+ int off = 0;
+
+ for (int i = 0; i < tupdesc->natts; i++)
+ {
+ CompactAttribute *cattr = TupleDescCompactAttr(tupdesc, i);
+
+ /*
+ * Find the highest attnum which is guaranteed to exist in all tuples
+ * in the table. We currently only pay attention to byval attributes
+ * to allow additional optimizations during tuple deformation.
+ */
+ if (firstNonGuaranteedAttr == tupdesc->natts &&
+ (cattr->attnullability != ATTNULLABLE_VALID || !cattr->attbyval ||
+ cattr->atthasmissing || cattr->attisdropped || cattr->attlen <= 0))
+ firstNonGuaranteedAttr = i;
+
+ if (cattr->attlen <= 0)
+ break;
+
+ off = att_nominal_alignby(off, cattr->attalignby);
+
+ cattr->attcacheoff = off;
+
+ off += cattr->attlen;
+ firstNonCachedOffsetAttr = i + 1;
+ }
+
+ tupdesc->firstNonCachedOffsetAttr = firstNonCachedOffsetAttr;
+ tupdesc->firstNonGuaranteedAttr = firstNonGuaranteedAttr;
+}
+
/*
* Free a TupleDesc including all substructure
*/
diff --git a/src/backend/access/spgist/spgutils.c b/src/backend/access/spgist/spgutils.c
index b246e8127db..a4694bd8065 100644
--- a/src/backend/access/spgist/spgutils.c
+++ b/src/backend/access/spgist/spgutils.c
@@ -335,9 +335,6 @@ getSpGistTupleDesc(Relation index, SpGistTypeDesc *keyType)
/* We shouldn't need to bother with making these valid: */
att->attcompression = InvalidCompressionMethod;
att->attcollation = InvalidOid;
- /* In case we changed typlen, we'd better reset following offsets */
- for (int i = spgFirstIncludeColumn; i < outTupDesc->natts; i++)
- TupleDescCompactAttr(outTupDesc, i)->attcacheoff = -1;
populate_compact_attribute(outTupDesc, spgKeyColumn);
TupleDescFinalize(outTupDesc);
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index bfd3ebc601e..0b635486993 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -1944,7 +1944,7 @@ ExecPartitionCheckEmitError(ResultRelInfo *resultRelInfo,
*/
if (map != NULL)
slot = execute_attr_map_slot(map, slot,
- MakeTupleTableSlot(tupdesc, &TTSOpsVirtual));
+ MakeTupleTableSlot(tupdesc, &TTSOpsVirtual, 0));
modifiedCols = bms_union(ExecGetInsertedCols(rootrel, estate),
ExecGetUpdatedCols(rootrel, estate));
}
@@ -2060,7 +2060,7 @@ ExecConstraints(ResultRelInfo *resultRelInfo,
*/
if (map != NULL)
slot = execute_attr_map_slot(map, slot,
- MakeTupleTableSlot(tupdesc, &TTSOpsVirtual));
+ MakeTupleTableSlot(tupdesc, &TTSOpsVirtual, 0));
modifiedCols = bms_union(ExecGetInsertedCols(rootrel, estate),
ExecGetUpdatedCols(rootrel, estate));
rel = rootrel->ri_RelationDesc;
@@ -2196,7 +2196,7 @@ ReportNotNullViolationError(ResultRelInfo *resultRelInfo, TupleTableSlot *slot,
*/
if (map != NULL)
slot = execute_attr_map_slot(map, slot,
- MakeTupleTableSlot(tupdesc, &TTSOpsVirtual));
+ MakeTupleTableSlot(tupdesc, &TTSOpsVirtual, 0));
modifiedCols = bms_union(ExecGetInsertedCols(rootrel, estate),
ExecGetUpdatedCols(rootrel, estate));
rel = rootrel->ri_RelationDesc;
@@ -2304,7 +2304,7 @@ ExecWithCheckOptions(WCOKind kind, ResultRelInfo *resultRelInfo,
*/
if (map != NULL)
slot = execute_attr_map_slot(map, slot,
- MakeTupleTableSlot(tupdesc, &TTSOpsVirtual));
+ MakeTupleTableSlot(tupdesc, &TTSOpsVirtual, 0));
modifiedCols = bms_union(ExecGetInsertedCols(rootrel, estate),
ExecGetUpdatedCols(rootrel, estate));
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 07b248aa5f3..14b2a9f0af6 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -992,118 +992,6 @@ tts_buffer_heap_store_tuple(TupleTableSlot *slot, HeapTuple tuple,
}
}
-/*
- * slot_deform_heap_tuple_internal
- * An always inline helper function for use in slot_deform_heap_tuple to
- * allow the compiler to emit specialized versions of this function for
- * various combinations of "slow" and "hasnulls". For example, if a
- * given tuple has no nulls, then we needn't check "hasnulls" for every
- * attribute that we're deforming. The caller can just call this
- * function with hasnulls set to constant-false and have the compiler
- * remove the constant-false branches and emit more optimal code.
- *
- * Returns the next attnum to deform, which can be equal to natts when the
- * function manages to deform all requested attributes. *offp is an input and
- * output parameter which is the byte offset within the tuple to start deforming
- * from which, on return, gets set to the offset where the next attribute
- * should be deformed from. *slowp is set to true when subsequent deforming
- * of this tuple must use a version of this function with "slow" passed as
- * true.
- *
- * Callers cannot assume when we return "attnum" (i.e. all requested
- * attributes have been deformed) that slow mode isn't required for any
- * additional deforming as the final attribute may have caused a switch to
- * slow mode.
- */
-static pg_attribute_always_inline int
-slot_deform_heap_tuple_internal(TupleTableSlot *slot, HeapTuple tuple,
- int attnum, int natts, bool slow,
- bool hasnulls, uint32 *offp, bool *slowp)
-{
- TupleDesc tupleDesc = slot->tts_tupleDescriptor;
- Datum *values = slot->tts_values;
- bool *isnull = slot->tts_isnull;
- HeapTupleHeader tup = tuple->t_data;
- char *tp; /* ptr to tuple data */
- bits8 *bp = tup->t_bits; /* ptr to null bitmap in tuple */
- bool slownext = false;
-
- tp = (char *) tup + tup->t_hoff;
-
- for (; attnum < natts; attnum++)
- {
- CompactAttribute *thisatt = TupleDescCompactAttr(tupleDesc, attnum);
-
- if (hasnulls && att_isnull(attnum, bp))
- {
- values[attnum] = (Datum) 0;
- isnull[attnum] = true;
- if (!slow)
- {
- *slowp = true;
- return attnum + 1;
- }
- else
- continue;
- }
-
- isnull[attnum] = false;
-
- /* calculate the offset of this attribute */
- if (!slow && thisatt->attcacheoff >= 0)
- *offp = thisatt->attcacheoff;
- else if (thisatt->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be no
- * pad bytes in any case: then the offset will be valid for either
- * an aligned or unaligned value.
- */
- if (!slow && *offp == att_nominal_alignby(*offp, thisatt->attalignby))
- thisatt->attcacheoff = *offp;
- else
- {
- *offp = att_pointer_alignby(*offp,
- thisatt->attalignby,
- -1,
- tp + *offp);
-
- if (!slow)
- slownext = true;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- *offp = att_nominal_alignby(*offp, thisatt->attalignby);
-
- if (!slow)
- thisatt->attcacheoff = *offp;
- }
-
- values[attnum] = fetchatt(thisatt, tp + *offp);
-
- *offp = att_addlength_pointer(*offp, thisatt->attlen, tp + *offp);
-
- /* check if we need to switch to slow mode */
- if (!slow)
- {
- /*
- * We're unable to deform any further if the above code set
- * 'slownext', or if this isn't a fixed-width attribute.
- */
- if (slownext || thisatt->attlen <= 0)
- {
- *slowp = true;
- return attnum + 1;
- }
- }
- }
-
- return natts;
-}
-
/*
* slot_deform_heap_tuple
* Given a TupleTableSlot, extract data from the slot's physical tuple
@@ -1125,94 +1013,228 @@ static pg_attribute_always_inline void
slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int reqnatts)
{
- bool hasnulls = HeapTupleHasNulls(tuple);
- int attnum;
+ CompactAttribute *cattr;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+ HeapTupleHeader tup = tuple->t_data;
+ size_t attnum;
+ int firstNonCacheOffsetAttr;
+ int firstNonGuaranteedAttr;
+ int firstNullAttr;
int natts;
+ Datum *values;
+ bool *isnull;
+ char *tp; /* ptr to tuple data */
uint32 off; /* offset in tuple data */
- bool slow; /* can we use/set attcacheoff? */
- /* We can only fetch as many attributes as the tuple has. */
- natts = Min(HeapTupleHeaderGetNatts(tuple->t_data), reqnatts);
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
+
+ isnull = slot->tts_isnull;
/*
- * Check whether the first call for this tuple, and initialize or restore
- * loop state.
+ * Some callers may form and deform tuples prior to NOT NULL constraints
+ * being checked. Here we'd like to optimize the case where we only need
+ * to fetch attributes before or up to the point where the attribute is
+ * guaranteed to exist in the tuple. We rely on the slot flag being set
+ * correctly to only enable this optimization when it's valid to do so.
+ * This optimization allows us to save fetching the number of attributes
+ * from the tuple and saves the additional cost of handling non-byval
+ * attrs.
*/
+ firstNonGuaranteedAttr = Min(reqnatts, slot->tts_first_nonguaranteed);
+
+ firstNonCacheOffsetAttr = tupleDesc->firstNonCachedOffsetAttr;
+
+ if (HeapTupleHasNulls(tuple))
+ {
+ natts = HeapTupleHeaderGetNatts(tup);
+ tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits) +
+ BITMAPLEN(natts));
+
+ natts = Min(natts, reqnatts);
+ if (natts > firstNonGuaranteedAttr)
+ {
+ bits8 *bp = tup->t_bits;
+
+ /* Find the first NULL attr */
+ firstNullAttr = first_null_attr(bp, natts);
+
+ /*
+ * And populate the isnull array for all attributes being fetched
+ * from the tuple.
+ */
+ populate_isnull_array(bp, natts, isnull);
+
+ /* We can only use any cached offsets until the first NULL attr */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr,
+ firstNullAttr);
+ }
+ else
+ {
+ /* Otherwise all required columns are guaranteed to exist */
+ firstNullAttr = natts;
+ }
+ }
+ else
+ {
+ tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits));
+
+ /*
+ * We only need to look at the tuple's natts if we need more than the
+ * guaranteed number of columns
+ */
+ if (reqnatts > firstNonGuaranteedAttr)
+ natts = Min(HeapTupleHeaderGetNatts(tup), reqnatts);
+ else
+ {
+ /* No need to access the number of attributes in the tuple */
+ natts = reqnatts;
+ }
+
+ /* All attrs can be fetched without checking for NULLs */
+ firstNullAttr = natts;
+ }
+
attnum = slot->tts_nvalid;
+ values = slot->tts_values;
slot->tts_nvalid = reqnatts;
- if (attnum == 0)
+
+ /* Ensure we calculated tp correctly */
+ Assert(tp == (char *) tup + tup->t_hoff);
+
+ if (attnum < firstNonGuaranteedAttr)
{
- /* Start from the first attribute */
- off = 0;
- slow = false;
+ int attlen;
+
+ for (; attnum < firstNonGuaranteedAttr; attnum++)
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ attlen = cattr->attlen;
+
+ /* We don't expect any non-byval types */
+ pg_assume(attlen > 0);
+ Assert(cattr->attbyval == true);
+
+ off = cattr->attcacheoff;
+ values[attnum] = fetch_att_noerr(tp + off, true, attlen);
+ }
+
+ off += attlen;
+
+ if (attnum == reqnatts)
+ goto done;
}
else
{
/* Restore state from previous execution */
off = *offp;
- slow = TTS_SLOW(slot);
+
+ /* We expect *offp to be set to 0 when attnum == 0 */
+ Assert(off == 0 || attnum > 0);
}
+ /* We can only fetch as many attributes as the tuple has. */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, natts);
+
/*
- * If 'slow' isn't set, try deforming using deforming code that does not
- * contain any of the extra checks required for non-fixed offset
- * deforming. During deforming, if or when we find a NULL or a variable
- * length attribute, we'll switch to a deforming method which includes the
- * extra code required for non-fixed offset deforming, a.k.a slow mode.
- * Because this is performance critical, we inline
- * slot_deform_heap_tuple_internal passing the 'slow' and 'hasnull'
- * parameters as constants to allow the compiler to emit specialized code
- * with the known-const false comparisons and subsequent branches removed.
+ * Handle the portion of the tuple that we have cached the offset for up
+ * to the first NULL attribute. The offset is effectively fixed for
+ * these, so we can use the CompactAttribute's attcacheoff.
*/
- if (!slow)
+ if (attnum < firstNonCacheOffsetAttr)
{
- /* Tuple without any NULLs? We can skip doing any NULL checking */
- if (!hasnulls)
- attnum = slot_deform_heap_tuple_internal(slot,
- tuple,
- attnum,
- natts,
- false, /* slow */
- false, /* hasnulls */
- &off,
- &slow);
- else
- attnum = slot_deform_heap_tuple_internal(slot,
- tuple,
- attnum,
- natts,
- false, /* slow */
- true, /* hasnulls */
- &off,
- &slow);
+ int attlen;
+
+ for (; attnum < firstNonCacheOffsetAttr; attnum++)
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ attlen = cattr->attlen;
+
+ off = cattr->attcacheoff;
+ values[attnum] = fetch_att_noerr(tp + off,
+ cattr->attbyval,
+ attlen);
+ }
+
+ /*
+ * Point the offset after the end of the last attribute with a cached
+ * offset. We expect the final cached offset attribute to have a
+ * fixed width, so just add the attlen to the attcacheoff
+ */
+ Assert(attlen > 0);
+ off += attlen;
}
- /* If there's still work to do then we must be in slow mode */
- if (attnum < natts)
+ /*
+ * Handle any portion of the tuple that doesn't have a fixed offset up
+ * until the first NULL attribute. This loop only differs from the one
+ * after it by the NULL checks.
+ */
+ for (; attnum < firstNullAttr; attnum++)
{
- /* XXX is it worth adding a separate call when hasnulls is false? */
- attnum = slot_deform_heap_tuple_internal(slot,
- tuple,
- attnum,
- natts,
- true, /* slow */
- hasnulls,
- &off,
- &slow);
+ int attlen;
+
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ attlen = cattr->attlen;
+
+ /*
+ * cstrings don't exist in heap tuples. Use pg_assume to instruct the
+ * compiler not to emit the cstring-related code in
+ * align_fetch_then_add().
+ */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ attlen,
+ cattr->attalignby);
}
/*
- * Save state for next execution
+ * Now handle any remaining attributes in the tuple up to the requested
+ * attnum. This time, include NULL checks as we're now at the first NULL
+ * attribute.
*/
- *offp = off;
- if (slow)
- slot->tts_flags |= TTS_FLAG_SLOW;
- else
- slot->tts_flags &= ~TTS_FLAG_SLOW;
+ for (; attnum < natts; attnum++)
+ {
+ int attlen;
- /* Fetch any missing attrs and raise an error if reqnatts is invalid. */
+ if (isnull[attnum])
+ {
+ values[attnum] = (Datum) 0;
+ continue;
+ }
+
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ attlen = cattr->attlen;
+
+ /* As above, we don't expect cstrings */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ attlen,
+ cattr->attalignby);
+ }
+
+ /* Fetch any missing attrs and raise an error if reqnatts is invalid */
if (unlikely(attnum < reqnatts))
+ {
+ *offp = off;
slot_getmissingattrs(slot, attnum, reqnatts);
+ return;
+ }
+done:
+
+ /* Save current offset for next execution */
+ *offp = off;
}
const TupleTableSlotOps TTSOpsVirtual = {
@@ -1307,7 +1329,7 @@ const TupleTableSlotOps TTSOpsBufferHeapTuple = {
*/
TupleTableSlot *
MakeTupleTableSlot(TupleDesc tupleDesc,
- const TupleTableSlotOps *tts_ops)
+ const TupleTableSlotOps *tts_ops, uint16 flags)
{
Size basesz,
allocsz;
@@ -1331,6 +1353,7 @@ MakeTupleTableSlot(TupleDesc tupleDesc,
*((const TupleTableSlotOps **) &slot->tts_ops) = tts_ops;
slot->type = T_TupleTableSlot;
slot->tts_flags |= TTS_FLAG_EMPTY;
+ slot->tts_flags |= flags;
if (tupleDesc != NULL)
slot->tts_flags |= TTS_FLAG_FIXED;
slot->tts_tupleDescriptor = tupleDesc;
@@ -1342,12 +1365,31 @@ MakeTupleTableSlot(TupleDesc tupleDesc,
slot->tts_values = (Datum *)
(((char *) slot)
+ MAXALIGN(basesz));
+
+ /*
+ * We round the size of tts_isnull up to the next highest multiple of
+ * 8. This is needed as populate_isnull_array() operates on 8
+ * elements at a time when converting a tuple's NULL bitmap into a
+ * boolean array.
+ */
slot->tts_isnull = (bool *)
(((char *) slot)
+ MAXALIGN(basesz)
- + MAXALIGN(tupleDesc->natts * sizeof(Datum)));
+ + TYPEALIGN(8, tupleDesc->natts * sizeof(Datum)));
PinTupleDesc(tupleDesc);
+
+ /*
+ * Precalculate the maximum guaranteed attribute that has to exist in
+ * every tuple which gets deformed into this slot. When the
+ * TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS flag is enabled, we simply take
+ * the precalculated value from the tupleDesc, otherwise the
+ * optimization is disabled, and we set the value to 0.
+ */
+ if ((flags & TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS) != 0)
+ slot->tts_first_nonguaranteed = tupleDesc->firstNonGuaranteedAttr;
+ else
+ slot->tts_first_nonguaranteed = 0;
}
/*
@@ -1366,9 +1408,9 @@ MakeTupleTableSlot(TupleDesc tupleDesc,
*/
TupleTableSlot *
ExecAllocTableSlot(List **tupleTable, TupleDesc desc,
- const TupleTableSlotOps *tts_ops)
+ const TupleTableSlotOps *tts_ops, uint16 flags)
{
- TupleTableSlot *slot = MakeTupleTableSlot(desc, tts_ops);
+ TupleTableSlot *slot = MakeTupleTableSlot(desc, tts_ops, flags);
*tupleTable = lappend(*tupleTable, slot);
@@ -1435,7 +1477,7 @@ TupleTableSlot *
MakeSingleTupleTableSlot(TupleDesc tupdesc,
const TupleTableSlotOps *tts_ops)
{
- TupleTableSlot *slot = MakeTupleTableSlot(tupdesc, tts_ops);
+ TupleTableSlot *slot = MakeTupleTableSlot(tupdesc, tts_ops, 0);
return slot;
}
@@ -1515,8 +1557,14 @@ ExecSetSlotDescriptor(TupleTableSlot *slot, /* slot to change */
*/
slot->tts_values = (Datum *)
MemoryContextAlloc(slot->tts_mcxt, tupdesc->natts * sizeof(Datum));
+
+ /*
+ * We round the size of tts_isnull up to the next highest multiple of 8.
+ * This is needed as populate_isnull_array() operates on 8 elements at a
+ * time when converting a tuple's NULL bitmap into a boolean array.
+ */
slot->tts_isnull = (bool *)
- MemoryContextAlloc(slot->tts_mcxt, tupdesc->natts * sizeof(bool));
+ MemoryContextAlloc(slot->tts_mcxt, TYPEALIGN(8, tupdesc->natts * sizeof(bool)));
}
/* --------------------------------
@@ -1978,7 +2026,7 @@ ExecInitResultSlot(PlanState *planstate, const TupleTableSlotOps *tts_ops)
TupleTableSlot *slot;
slot = ExecAllocTableSlot(&planstate->state->es_tupleTable,
- planstate->ps_ResultTupleDesc, tts_ops);
+ planstate->ps_ResultTupleDesc, tts_ops, 0);
planstate->ps_ResultTupleSlot = slot;
planstate->resultopsfixed = planstate->ps_ResultTupleDesc != NULL;
@@ -2006,10 +2054,11 @@ ExecInitResultTupleSlotTL(PlanState *planstate,
*/
void
ExecInitScanTupleSlot(EState *estate, ScanState *scanstate,
- TupleDesc tupledesc, const TupleTableSlotOps *tts_ops)
+ TupleDesc tupledesc, const TupleTableSlotOps *tts_ops,
+ uint16 flags)
{
scanstate->ss_ScanTupleSlot = ExecAllocTableSlot(&estate->es_tupleTable,
- tupledesc, tts_ops);
+ tupledesc, tts_ops, flags);
scanstate->ps.scandesc = tupledesc;
scanstate->ps.scanopsfixed = tupledesc != NULL;
scanstate->ps.scanops = tts_ops;
@@ -2029,7 +2078,7 @@ ExecInitExtraTupleSlot(EState *estate,
TupleDesc tupledesc,
const TupleTableSlotOps *tts_ops)
{
- return ExecAllocTableSlot(&estate->es_tupleTable, tupledesc, tts_ops);
+ return ExecAllocTableSlot(&estate->es_tupleTable, tupledesc, tts_ops, 0);
}
/* ----------------
@@ -2261,10 +2310,16 @@ ExecTypeSetColNames(TupleDesc typeInfo, List *namesList)
* This happens "for free" if the tupdesc came from a relcache entry, but
* not if we have manufactured a tupdesc for a transient RECORD datatype.
* In that case we have to notify typcache.c of the existence of the type.
+ *
+ * TupleDescFinalize() must be called on the TupleDesc before calling this
+ * function.
*/
TupleDesc
BlessTupleDesc(TupleDesc tupdesc)
{
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupdesc->firstNonCachedOffsetAttr >= 0);
+
if (tupdesc->tdtypeid == RECORDOID &&
tupdesc->tdtypmod < 0)
assign_record_type_typmod(tupdesc);
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index a7955e476f9..f62582859f9 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -711,7 +711,7 @@ ExecCreateScanSlotFromOuterPlan(EState *estate,
outerPlan = outerPlanState(scanstate);
tupDesc = ExecGetResultType(outerPlan);
- ExecInitScanTupleSlot(estate, scanstate, tupDesc, tts_ops);
+ ExecInitScanTupleSlot(estate, scanstate, tupDesc, tts_ops, 0);
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
index 7d487a165fa..c5c321b4f42 100644
--- a/src/backend/executor/nodeAgg.c
+++ b/src/backend/executor/nodeAgg.c
@@ -1682,7 +1682,7 @@ find_hash_columns(AggState *aggstate)
&perhash->hashfunctions);
perhash->hashslot =
ExecAllocTableSlot(&estate->es_tupleTable, hashDesc,
- &TTSOpsMinimalTuple);
+ &TTSOpsMinimalTuple, 0);
list_free(hashTlist);
bms_free(colnos);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index c68c26cbf38..dcb7599b96c 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -381,7 +381,10 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
*/
ExecInitScanTupleSlot(estate, &scanstate->ss,
RelationGetDescr(currentRelation),
- table_slot_callbacks(currentRelation));
+ table_slot_callbacks(currentRelation), 0);
+
+ scanstate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeCtescan.c b/src/backend/executor/nodeCtescan.c
index e6e476388e5..45b09d93b93 100644
--- a/src/backend/executor/nodeCtescan.c
+++ b/src/backend/executor/nodeCtescan.c
@@ -261,7 +261,7 @@ ExecInitCteScan(CteScan *node, EState *estate, int eflags)
*/
ExecInitScanTupleSlot(estate, &scanstate->ss,
ExecGetResultType(scanstate->cteplanstate),
- &TTSOpsMinimalTuple);
+ &TTSOpsMinimalTuple, 0);
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeCustom.c b/src/backend/executor/nodeCustom.c
index a9ad5af6a98..b7cc890cd20 100644
--- a/src/backend/executor/nodeCustom.c
+++ b/src/backend/executor/nodeCustom.c
@@ -79,14 +79,14 @@ ExecInitCustomScan(CustomScan *cscan, EState *estate, int eflags)
TupleDesc scan_tupdesc;
scan_tupdesc = ExecTypeFromTL(cscan->custom_scan_tlist);
- ExecInitScanTupleSlot(estate, &css->ss, scan_tupdesc, slotOps);
+ ExecInitScanTupleSlot(estate, &css->ss, scan_tupdesc, slotOps, 0);
/* Node's targetlist will contain Vars with varno = INDEX_VAR */
tlistvarno = INDEX_VAR;
}
else
{
ExecInitScanTupleSlot(estate, &css->ss, RelationGetDescr(scan_rel),
- slotOps);
+ slotOps, 0);
/* Node's targetlist will contain Vars with varno = scanrelid */
tlistvarno = scanrelid;
}
diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c
index 8721b67b7cc..6f0daddce07 100644
--- a/src/backend/executor/nodeForeignscan.c
+++ b/src/backend/executor/nodeForeignscan.c
@@ -191,7 +191,7 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
scan_tupdesc = ExecTypeFromTL(node->fdw_scan_tlist);
ExecInitScanTupleSlot(estate, &scanstate->ss, scan_tupdesc,
- &TTSOpsHeapTuple);
+ &TTSOpsHeapTuple, 0);
/* Node's targetlist will contain Vars with varno = INDEX_VAR */
tlistvarno = INDEX_VAR;
}
@@ -202,7 +202,7 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
/* don't trust FDWs to return tuples fulfilling NOT NULL constraints */
scan_tupdesc = CreateTupleDescCopy(RelationGetDescr(currentRelation));
ExecInitScanTupleSlot(estate, &scanstate->ss, scan_tupdesc,
- &TTSOpsHeapTuple);
+ &TTSOpsHeapTuple, 0);
/* Node's targetlist will contain Vars with varno = scanrelid */
tlistvarno = scanrelid;
}
diff --git a/src/backend/executor/nodeFunctionscan.c b/src/backend/executor/nodeFunctionscan.c
index feb82d64967..222741adf3b 100644
--- a/src/backend/executor/nodeFunctionscan.c
+++ b/src/backend/executor/nodeFunctionscan.c
@@ -494,7 +494,7 @@ ExecInitFunctionScan(FunctionScan *node, EState *estate, int eflags)
* Initialize scan slot and type.
*/
ExecInitScanTupleSlot(estate, &scanstate->ss, scan_tupdesc,
- &TTSOpsMinimalTuple);
+ &TTSOpsMinimalTuple, 0);
/*
* Initialize result slot, type and projection.
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index c2d09374517..144a57fde95 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -567,7 +567,10 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
*/
tupDesc = ExecTypeFromTL(node->indextlist);
ExecInitScanTupleSlot(estate, &indexstate->ss, tupDesc,
- &TTSOpsVirtual);
+ &TTSOpsVirtual, 0);
+
+ indexstate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
/*
* We need another slot, in a format that's suitable for the table AM, for
@@ -576,7 +579,7 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
indexstate->ioss_TableSlot =
ExecAllocTableSlot(&estate->es_tupleTable,
RelationGetDescr(currentRelation),
- table_slot_callbacks(currentRelation));
+ table_slot_callbacks(currentRelation), 0);
/*
* Initialize result type and projection info. The node's targetlist will
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index a616abff04c..e7bebb89517 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -938,7 +938,10 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
*/
ExecInitScanTupleSlot(estate, &indexstate->ss,
RelationGetDescr(currentRelation),
- table_slot_callbacks(currentRelation));
+ table_slot_callbacks(currentRelation), 0);
+
+ indexstate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeNamedtuplestorescan.c b/src/backend/executor/nodeNamedtuplestorescan.c
index fdfccc8169f..29d862a4001 100644
--- a/src/backend/executor/nodeNamedtuplestorescan.c
+++ b/src/backend/executor/nodeNamedtuplestorescan.c
@@ -137,7 +137,7 @@ ExecInitNamedTuplestoreScan(NamedTuplestoreScan *node, EState *estate, int eflag
* The scan tuple type is specified for the tuplestore.
*/
ExecInitScanTupleSlot(estate, &scanstate->ss, scanstate->tupdesc,
- &TTSOpsMinimalTuple);
+ &TTSOpsMinimalTuple, 0);
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c
index 1b0af70fd7a..a1da05ecc18 100644
--- a/src/backend/executor/nodeSamplescan.c
+++ b/src/backend/executor/nodeSamplescan.c
@@ -128,7 +128,11 @@ ExecInitSampleScan(SampleScan *node, EState *estate, int eflags)
/* and create slot with appropriate rowtype */
ExecInitScanTupleSlot(estate, &scanstate->ss,
RelationGetDescr(scanstate->ss.ss_currentRelation),
- table_slot_callbacks(scanstate->ss.ss_currentRelation));
+ table_slot_callbacks(scanstate->ss.ss_currentRelation),
+ 0);
+
+ scanstate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index af3c788ce8b..8f219f60a93 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -244,7 +244,8 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags)
/* and create slot with the appropriate rowtype */
ExecInitScanTupleSlot(estate, &scanstate->ss,
RelationGetDescr(scanstate->ss.ss_currentRelation),
- table_slot_callbacks(scanstate->ss.ss_currentRelation));
+ table_slot_callbacks(scanstate->ss.ss_currentRelation),
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS);
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeSubqueryscan.c b/src/backend/executor/nodeSubqueryscan.c
index 4fd6f6fb4a5..70914e8189c 100644
--- a/src/backend/executor/nodeSubqueryscan.c
+++ b/src/backend/executor/nodeSubqueryscan.c
@@ -130,7 +130,8 @@ ExecInitSubqueryScan(SubqueryScan *node, EState *estate, int eflags)
*/
ExecInitScanTupleSlot(estate, &subquerystate->ss,
ExecGetResultType(subquerystate->subplan),
- ExecGetResultSlotOps(subquerystate->subplan, NULL));
+ ExecGetResultSlotOps(subquerystate->subplan, NULL),
+ 0);
/*
* The slot used as the scantuple isn't the slot above (outside of EPQ),
diff --git a/src/backend/executor/nodeTableFuncscan.c b/src/backend/executor/nodeTableFuncscan.c
index 52070d147a4..769b9766542 100644
--- a/src/backend/executor/nodeTableFuncscan.c
+++ b/src/backend/executor/nodeTableFuncscan.c
@@ -148,7 +148,7 @@ ExecInitTableFuncScan(TableFuncScan *node, EState *estate, int eflags)
tf->colcollations);
/* and the corresponding scan slot */
ExecInitScanTupleSlot(estate, &scanstate->ss, tupdesc,
- &TTSOpsMinimalTuple);
+ &TTSOpsMinimalTuple, 0);
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index 503817da65b..decc3167ba7 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -394,7 +394,10 @@ ExecInitTidRangeScan(TidRangeScan *node, EState *estate, int eflags)
*/
ExecInitScanTupleSlot(estate, &tidrangestate->ss,
RelationGetDescr(currentRelation),
- table_slot_callbacks(currentRelation));
+ table_slot_callbacks(currentRelation), 0);
+
+ tidrangestate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeTidscan.c b/src/backend/executor/nodeTidscan.c
index 4eddb0828b5..26593b930af 100644
--- a/src/backend/executor/nodeTidscan.c
+++ b/src/backend/executor/nodeTidscan.c
@@ -536,7 +536,10 @@ ExecInitTidScan(TidScan *node, EState *estate, int eflags)
*/
ExecInitScanTupleSlot(estate, &tidstate->ss,
RelationGetDescr(currentRelation),
- table_slot_callbacks(currentRelation));
+ table_slot_callbacks(currentRelation), 0);
+
+ tidstate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeValuesscan.c b/src/backend/executor/nodeValuesscan.c
index e663fb68cfc..effc896ea1c 100644
--- a/src/backend/executor/nodeValuesscan.c
+++ b/src/backend/executor/nodeValuesscan.c
@@ -247,7 +247,7 @@ ExecInitValuesScan(ValuesScan *node, EState *estate, int eflags)
* Get info about values list, initialize scan slot with it.
*/
tupdesc = ExecTypeFromExprList((List *) linitial(node->values_lists));
- ExecInitScanTupleSlot(estate, &scanstate->ss, tupdesc, &TTSOpsVirtual);
+ ExecInitScanTupleSlot(estate, &scanstate->ss, tupdesc, &TTSOpsVirtual, 0);
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeWorktablescan.c b/src/backend/executor/nodeWorktablescan.c
index 210cc44f911..66e904c7636 100644
--- a/src/backend/executor/nodeWorktablescan.c
+++ b/src/backend/executor/nodeWorktablescan.c
@@ -165,7 +165,7 @@ ExecInitWorkTableScan(WorkTableScan *node, EState *estate, int eflags)
scanstate->ss.ps.resultopsset = true;
scanstate->ss.ps.resultopsfixed = false;
- ExecInitScanTupleSlot(estate, &scanstate->ss, NULL, &TTSOpsMinimalTuple);
+ ExecInitScanTupleSlot(estate, &scanstate->ss, NULL, &TTSOpsMinimalTuple, 0);
/*
* initialize child expressions
diff --git a/src/backend/jit/llvm/llvmjit_deform.c b/src/backend/jit/llvm/llvmjit_deform.c
index 3eb087eb56b..12521e3e46a 100644
--- a/src/backend/jit/llvm/llvmjit_deform.c
+++ b/src/backend/jit/llvm/llvmjit_deform.c
@@ -62,7 +62,6 @@ slot_compile_deform(LLVMJitContext *context, TupleDesc desc,
LLVMValueRef v_tts_values;
LLVMValueRef v_tts_nulls;
LLVMValueRef v_slotoffp;
- LLVMValueRef v_flagsp;
LLVMValueRef v_nvalidp;
LLVMValueRef v_nvalid;
LLVMValueRef v_maxatt;
@@ -178,7 +177,6 @@ slot_compile_deform(LLVMJitContext *context, TupleDesc desc,
v_tts_nulls =
l_load_struct_gep(b, StructTupleTableSlot, v_slot, FIELDNO_TUPLETABLESLOT_ISNULL,
"tts_ISNULL");
- v_flagsp = l_struct_gep(b, StructTupleTableSlot, v_slot, FIELDNO_TUPLETABLESLOT_FLAGS, "");
v_nvalidp = l_struct_gep(b, StructTupleTableSlot, v_slot, FIELDNO_TUPLETABLESLOT_NVALID, "");
if (ops == &TTSOpsHeapTuple || ops == &TTSOpsBufferHeapTuple)
@@ -747,14 +745,10 @@ slot_compile_deform(LLVMJitContext *context, TupleDesc desc,
{
LLVMValueRef v_off = l_load(b, TypeSizeT, v_offp, "");
- LLVMValueRef v_flags;
LLVMBuildStore(b, l_int16_const(lc, natts), v_nvalidp);
v_off = LLVMBuildTrunc(b, v_off, LLVMInt32TypeInContext(lc), "");
LLVMBuildStore(b, v_off, v_slotoffp);
- v_flags = l_load(b, LLVMInt16TypeInContext(lc), v_flagsp, "tts_flags");
- v_flags = LLVMBuildOr(b, v_flags, l_int16_const(lc, TTS_FLAG_SLOW), "");
- LLVMBuildStore(b, v_flags, v_flagsp);
LLVMBuildRetVoid(b);
}
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 857ebf7d6fb..4ecfcbff7ab 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -1559,7 +1559,7 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
if (relentry->attrmap)
{
TupleTableSlot *slot = MakeTupleTableSlot(RelationGetDescr(targetrel),
- &TTSOpsVirtual);
+ &TTSOpsVirtual, 0);
old_slot = execute_attr_map_slot(relentry->attrmap, old_slot, slot);
}
@@ -1574,7 +1574,7 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
if (relentry->attrmap)
{
TupleTableSlot *slot = MakeTupleTableSlot(RelationGetDescr(targetrel),
- &TTSOpsVirtual);
+ &TTSOpsVirtual, 0);
new_slot = execute_attr_map_slot(relentry->attrmap, new_slot, slot);
}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index d27ac216e6d..597de687b45 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -666,14 +666,6 @@ RelationBuildTupleDesc(Relation relation)
elog(ERROR, "pg_attribute catalog is missing %d attribute(s) for relation OID %u",
need, RelationGetRelid(relation));
- /*
- * We can easily set the attcacheoff value for the first attribute: it
- * must be zero. This eliminates the need for special cases for attnum=1
- * that used to exist in fastgetattr() and index_getattr().
- */
- if (RelationGetNumberOfAttributes(relation) > 0)
- TupleDescCompactAttr(relation->rd_att, 0)->attcacheoff = 0;
-
/*
* Set up constraint/default info
*/
@@ -1985,8 +1977,6 @@ formrdesc(const char *relationName, Oid relationReltype,
populate_compact_attribute(relation->rd_att, i);
}
- /* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
- TupleDescCompactAttr(relation->rd_att, 0)->attcacheoff = 0;
TupleDescFinalize(relation->rd_att);
/* mark not-null status */
@@ -4446,8 +4436,6 @@ BuildHardcodedDescriptor(int natts, const FormData_pg_attribute *attrs)
populate_compact_attribute(result, i);
}
- /* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
- TupleDescCompactAttr(result, 0)->attcacheoff = 0;
TupleDescFinalize(result);
/* Note: we don't bother to set up a TupleConstr entry */
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 595413dbbc5..ad7bc013812 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -131,6 +131,19 @@ typedef struct CompactAttribute
* Any code making changes manually to and fields in the FormData_pg_attribute
* array must subsequently call populate_compact_attribute() to flush the
* changes out to the corresponding 'compact_attrs' element.
+ *
+ * firstNonCachedOffsetAttr stores the index into the compact_attrs array for
+ * the first attribute that we don't have a known attcacheoff for.
+ *
+ * firstNonGuaranteedAttr stores the index to into the compact_attrs array for
+ * the first attribute that is either NULLable, missing, or !attbyval. This
+ * can be used in locations as a guarantee that attributes before this will
+ * always exist in tuples. The !attbyval part isn't required for this, but
+ * including this allows various tuple deforming routines to forego any checks
+ * for !attbyval.
+ *
+ * Once a TupleDesc has been populated, before it is used for any purpose,
+ * TupleDescFinalize() must be called on it.
*/
typedef struct TupleDescData
{
@@ -138,6 +151,11 @@ typedef struct TupleDescData
Oid tdtypeid; /* composite type ID for tuple type */
int32 tdtypmod; /* typmod for tuple type */
int tdrefcount; /* reference count, or -1 if not counting */
+ int firstNonCachedOffsetAttr; /* index of the first att without
+ * an attcacheoff */
+ int firstNonGuaranteedAttr; /* index of the first nullable,
+ * missing, dropped, or !attbyval
+ * attribute. */
TupleConstr *constr; /* constraints, or NULL if none */
/* compact_attrs[N] is the compact metadata of Attribute Number N+1 */
CompactAttribute compact_attrs[FLEXIBLE_ARRAY_MEMBER];
@@ -195,7 +213,6 @@ extern TupleDesc CreateTupleDescTruncatedCopy(TupleDesc tupdesc, int natts);
extern TupleDesc CreateTupleDescCopyConstr(TupleDesc tupdesc);
-#define TupleDescFinalize(d) ((void) 0)
#define TupleDescSize(src) \
(offsetof(struct TupleDescData, compact_attrs) + \
(src)->natts * sizeof(CompactAttribute) + \
@@ -206,6 +223,7 @@ extern void TupleDescCopy(TupleDesc dst, TupleDesc src);
extern void TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
TupleDesc src, AttrNumber srcAttno);
+extern void TupleDescFinalize(TupleDesc tupdesc);
extern void FreeTupleDesc(TupleDesc tupdesc);
extern void IncrTupleDescRefCount(TupleDesc tupdesc);
diff --git a/src/include/access/tupmacs.h b/src/include/access/tupmacs.h
index d64c18b950b..87dbeb76618 100644
--- a/src/include/access/tupmacs.h
+++ b/src/include/access/tupmacs.h
@@ -15,7 +15,9 @@
#define TUPMACS_H
#include "catalog/pg_type_d.h" /* for TYPALIGN macros */
-
+#include "port/pg_bitutils.h"
+#include "port/pg_bswap.h"
+#include "varatt.h"
/*
* Check a tuple's null bitmap to determine whether the attribute is null.
@@ -28,6 +30,62 @@ att_isnull(int ATT, const bits8 *BITS)
return !(BITS[ATT >> 3] & (1 << (ATT & 0x07)));
}
+/*
+ * populate_isnull_array
+ * Transform a tuple's null bitmap into a boolean array.
+ *
+ * Caller must ensure that the isnull array is sized so it contains
+ * at least as many elements as there are bits in the 'bits' array.
+ * Callers should be aware that isnull is populated 8 elements at a time,
+ * effectively as if natts is rounded up to the next multiple of 8.
+ */
+static inline void
+populate_isnull_array(const bits8 *bits, int natts, bool *isnull)
+{
+ int nbytes = (natts + 7) >> 3;
+
+ /*
+ * Multiplying the inverted NULL bitmap byte by this value results in the
+ * lowest bit in each byte being set the same as each bit of the inverted
+ * byte. We perform this as 2 32-bit operations rather than a single
+ * 64-bit operation as multiplying by the required value to do this in
+ * 64-bits would result in overflowing a uint64 in some cases.
+ *
+ * XXX if we ever require BMI2 (-march=x86-64-v3), then this could be done
+ * more efficiently on most X86-64 CPUs with the PDEP instruction. Beware
+ * that some chips (e.g. AMD's Zen2) are horribly inefficient at PDEP.
+ */
+#define SPREAD_BITS_MULTIPLIER_32 0x204081U
+
+ for (int i = 0; i < nbytes; i++, isnull += 8)
+ {
+ uint64 isnull_8;
+ bits8 nullbyte = ~bits[i];
+
+ /* Convert the lower 4 bits of NULL bitmap word into a 64 bit int */
+ isnull_8 = (nullbyte & 0xf) * SPREAD_BITS_MULTIPLIER_32;
+
+ /*
+ * Convert the upper 4 bits of null bitmap word into a 64 bit int,
+ * shift into the upper 32 bit and bitwise-OR with the result of the
+ * lower 4 bits.
+ */
+ isnull_8 |= ((uint64) ((nullbyte >> 4) * SPREAD_BITS_MULTIPLIER_32)) << 32;
+
+ /* Mask out all other bits apart from the lowest bit of each byte. */
+ isnull_8 &= UINT64CONST(0x0101010101010101);
+
+#ifdef WORDS_BIGENDIAN
+
+ /*
+ * Fix byte order on big-endian machines before copying to the array.
+ */
+ isnull_8 = pg_bswap64(isnull_8);
+#endif
+ memcpy(isnull, &isnull_8, sizeof(uint64));
+ }
+}
+
#ifndef FRONTEND
/*
* Given an attbyval and an attlen from either a Form_pg_attribute or
@@ -69,6 +127,170 @@ fetch_att(const void *T, bool attbyval, int attlen)
else
return PointerGetDatum(T);
}
+
+/*
+ * Same, but no error checking for invalid attlens for byval types. This
+ * is safe to use when attlen comes from CompactAttribute as we validate the
+ * length when populating that struct.
+ */
+static inline Datum
+fetch_att_noerr(const void *T, bool attbyval, int attlen)
+{
+ if (attbyval)
+ {
+ switch (attlen)
+ {
+ case sizeof(int32):
+ return Int32GetDatum(*((const int32 *) T));
+ case sizeof(int16):
+ return Int16GetDatum(*((const int16 *) T));
+ case sizeof(char):
+ return CharGetDatum(*((const char *) T));
+ default:
+ Assert(attlen == sizeof(int64));
+ return Int64GetDatum(*((const int64 *) T));
+ }
+ }
+ else
+ return PointerGetDatum(T);
+}
+
+
+/*
+ * align_fetch_then_add
+ * Applies all the functionality of att_pointer_alignby(),
+ * fetch_att_noerr() and att_addlength_pointer(), resulting in the *off
+ * pointer to the perhaps unaligned number of bytes into 'tupptr', ready
+ * to deform the next attribute.
+ *
+ * tupptr: pointer to the beginning of the tuple, after the header and any
+ * NULL bitmask.
+ * off: offset in bytes for reading tuple data, possibly unaligned.
+ * attbyval, attlen and attalignby are values from CompactAttribute.
+ */
+static inline Datum
+align_fetch_then_add(const char *tupptr, uint32 *off, bool attbyval, int attlen,
+ uint8 attalignby)
+{
+ Datum res;
+
+ if (attlen > 0)
+ {
+ const char *offset_ptr;
+
+ *off = TYPEALIGN(attalignby, *off);
+ offset_ptr = tupptr + *off;
+ *off += attlen;
+ if (attbyval)
+ {
+ switch (attlen)
+ {
+ case sizeof(char):
+ return CharGetDatum(*((const char *) offset_ptr));
+ case sizeof(int16):
+ return Int16GetDatum(*((const int16 *) offset_ptr));
+ case sizeof(int32):
+ return Int32GetDatum(*((const int32 *) offset_ptr));
+ default:
+
+ /*
+ * populate_compact_attribute_internal() should have
+ * checked
+ */
+ Assert(attlen == sizeof(int64));
+ return Int64GetDatum(*((const int64 *) offset_ptr));
+ }
+ }
+ return PointerGetDatum(offset_ptr);
+ }
+ else if (attlen == -1)
+ {
+ if (!VARATT_IS_SHORT(tupptr + *off))
+ *off = TYPEALIGN(attalignby, *off);
+
+ res = PointerGetDatum(tupptr + *off);
+ *off += VARSIZE_ANY(DatumGetPointer(res));
+ return res;
+ }
+ else
+ {
+ Assert(attlen == -2);
+ *off = TYPEALIGN(attalignby, *off);
+ res = PointerGetDatum(tupptr + *off);
+ *off += strlen(tupptr + *off) + 1;
+ return res;
+ }
+}
+
+/*
+ * first_null_attr
+ * Inspect a NULL bitmap from a tuple and return the 0-based attnum of the
+ * first NULL attribute. Returns natts if no NULLs were found.
+ *
+ * This is coded to expect that 'bits' contains at least one 0 bit somewhere
+ * in the array, but not necessarily < natts. Note that natts may be passed
+ * as a value lower than the number of bits physically stored in the tuple's
+ * NULL bitmap, in which case we may not find a NULL and return natts.
+ *
+ * The reason we require at least one 0 bit somewhere in the NULL bitmap is
+ * that the for loop that checks 0xFF bytes would loop to the last byte in
+ * the array if all bytes were 0xFF, and the subsequent code that finds the
+ * right-most 0 bit would access the first byte beyond the bitmap. Provided
+ * we find a 0 bit before then, that won't happen. Since tuples which have no
+ * NULLs don't have a NULL bitmap, this function won't get called for that
+ * case.
+ */
+static inline int
+first_null_attr(const bits8 *bits, int natts)
+{
+ int nattByte = natts >> 3;
+ int bytenum;
+ int res;
+
+#ifdef USE_ASSERT_CHECKING
+ int firstnull_check = natts;
+
+ /* Do it the slow way and check we get the same answer. */
+ for (int i = 0; i < natts; i++)
+ {
+ if (att_isnull(i, bits))
+ {
+ firstnull_check = i;
+ break;
+ }
+ }
+#endif
+
+ /* Process all bytes up to just before the byte for the natts attribute */
+ for (bytenum = 0; bytenum < nattByte; bytenum++)
+ {
+ /* break if there's any NULL attrs (a 0 bit) */
+ if (bits[bytenum] != 0xFF)
+ break;
+ }
+
+ /*
+ * Look for the highest 0-bit in the 'bytenum' element. To do this, we
+ * promote the uint8 to uint32 before performing the bitwise NOT and
+ * looking for the first 1-bit. This works even when the byte is 0xFF, as
+ * the bitwise NOT of 0xFF in 32 bits is 0xFFFFFF00, in which case
+ * pg_rightmost_one_pos32() will return 8. We may end up with a value
+ * higher than natts here, but we'll fix that with the Min() below.
+ */
+ res = bytenum << 3;
+ res += pg_rightmost_one_pos32(~((uint32) bits[bytenum]));
+
+ /*
+ * Since we did no masking to mask out bits beyond the natt'th bit, we may
+ * have found a bit higher than natts, so we must cap res to natts
+ */
+ res = Min(res, natts);
+
+ /* Ensure we got the same answer as the att_isnull() loop got */
+ Assert(res == firstnull_check);
+
+ return res;
+}
#endif /* FRONTEND */
/*
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index d46ba59895d..bf239fc156f 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -595,7 +595,8 @@ extern void ExecInitResultTupleSlotTL(PlanState *planstate,
const TupleTableSlotOps *tts_ops);
extern void ExecInitScanTupleSlot(EState *estate, ScanState *scanstate,
TupleDesc tupledesc,
- const TupleTableSlotOps *tts_ops);
+ const TupleTableSlotOps *tts_ops,
+ uint16 flags);
extern TupleTableSlot *ExecInitExtraTupleSlot(EState *estate,
TupleDesc tupledesc,
const TupleTableSlotOps *tts_ops);
diff --git a/src/include/executor/tuptable.h b/src/include/executor/tuptable.h
index 3b09abbf99f..78558098fa3 100644
--- a/src/include/executor/tuptable.h
+++ b/src/include/executor/tuptable.h
@@ -84,9 +84,6 @@
* tts_values/tts_isnull are allocated either when the slot is created (when
* the descriptor is provided), or when a descriptor is assigned to the slot;
* they are of length equal to the descriptor's natts.
- *
- * The TTS_FLAG_SLOW flag is saved state for
- * slot_deform_heap_tuple, and should not be touched by any other code.
*----------
*/
@@ -98,9 +95,13 @@
#define TTS_FLAG_SHOULDFREE (1 << 2)
#define TTS_SHOULDFREE(slot) (((slot)->tts_flags & TTS_FLAG_SHOULDFREE) != 0)
-/* saved state for slot_deform_heap_tuple */
-#define TTS_FLAG_SLOW (1 << 3)
-#define TTS_SLOW(slot) (((slot)->tts_flags & TTS_FLAG_SLOW) != 0)
+/*
+ * true = slot's formed tuple guaranteed to not have NULLs in NOT NULLable
+ * columns.
+ */
+#define TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS (1 << 3)
+#define TTS_OBEYS_NOT_NULL_CONSTRAINTS(slot) \
+ (((slot)->tts_flags & TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS) != 0)
/* fixed tuple descriptor */
#define TTS_FLAG_FIXED (1 << 4)
@@ -123,7 +124,14 @@ typedef struct TupleTableSlot
#define FIELDNO_TUPLETABLESLOT_VALUES 5
Datum *tts_values; /* current per-attribute values */
#define FIELDNO_TUPLETABLESLOT_ISNULL 6
- bool *tts_isnull; /* current per-attribute isnull flags */
+ bool *tts_isnull; /* current per-attribute isnull flags. Array
+ * size must always be rounded up to the next
+ * multiple of 8 elements. */
+ int tts_first_nonguaranteed; /* The value from the TupleDesc's
+ * firstNonGuaranteedAttr, or 0
+ * when tts_flags does not contain
+ * TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS */
+
MemoryContext tts_mcxt; /* slot itself is in this context */
ItemPointerData tts_tid; /* stored tuple's tid */
Oid tts_tableOid; /* table oid of tuple */
@@ -313,9 +321,11 @@ typedef struct MinimalTupleTableSlot
/* in executor/execTuples.c */
extern TupleTableSlot *MakeTupleTableSlot(TupleDesc tupleDesc,
- const TupleTableSlotOps *tts_ops);
+ const TupleTableSlotOps *tts_ops,
+ uint16 flags);
extern TupleTableSlot *ExecAllocTableSlot(List **tupleTable, TupleDesc desc,
- const TupleTableSlotOps *tts_ops);
+ const TupleTableSlotOps *tts_ops,
+ uint16 flags);
extern void ExecResetTupleTable(List *tupleTable, bool shouldFree);
extern TupleTableSlot *MakeSingleTupleTableSlot(TupleDesc tupdesc,
const TupleTableSlotOps *tts_ops);
diff --git a/src/test/modules/deform_bench/deform_bench.c b/src/test/modules/deform_bench/deform_bench.c
index 7838f639bef..4f104989297 100644
--- a/src/test/modules/deform_bench/deform_bench.c
+++ b/src/test/modules/deform_bench/deform_bench.c
@@ -48,7 +48,9 @@ deform_bench(PG_FUNCTION_ARGS)
errmsg("only heap AM is supported")));
tupdesc = RelationGetDescr(rel);
- slot = MakeTupleTableSlot(tupdesc, &TTSOpsBufferHeapTuple);
+ slot = MakeTupleTableSlot(tupdesc,
+ &TTSOpsBufferHeapTuple,
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS);
scan = table_beginscan_strat(rel, GetActiveSnapshot(), 0, NULL, true, false);
/*
--
2.51.0
From 8e1a945f172e3c6e9cc0092e5c2168151ed8c6d1 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Mon, 23 Feb 2026 09:39:37 +1300
Subject: [PATCH v12 5/6] Reduce size of CompactAttribute struct to 8 bytes
Previously, this was 16 bytes. With the use of some bitflags and by
reducing the attcacheoff field size to a 16-bit type, we can halve the
size of the struct.
It's unlikely that caching the offsets for offsets larger than what will
fit in a 16-bit int will help much as the tuple is very likely to have
some non-fixed-width types anyway, the offsets of which we cannot cache.
---
src/backend/access/common/tupdesc.c | 10 ++++++++++
src/backend/executor/execTuples.c | 17 ++++++++++++-----
src/include/access/tupdesc.h | 16 ++++++++--------
3 files changed, 30 insertions(+), 13 deletions(-)
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index c68561337d7..71461ba6096 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -530,6 +530,16 @@ TupleDescFinalize(TupleDesc tupdesc)
off = att_nominal_alignby(off, cattr->attalignby);
+ /*
+ * attcacheoff is an int16, so don't try to cache any offsets larger
+ * than will fit in that type. Any attributes which are offset more
+ * than 2^15 are likely due to variable-length attributes. Since we
+ * don't cache offsets for or beyond variable-length attributes, using
+ * an int16 rather than an int32 here is unlikely to cost us anything.
+ */
+ if (off > PG_INT16_MAX)
+ break;
+
cattr->attcacheoff = off;
off += cattr->attlen;
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 14b2a9f0af6..1b5e1ebd334 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -1013,6 +1013,7 @@ static pg_attribute_always_inline void
slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int reqnatts)
{
+ CompactAttribute *cattrs;
CompactAttribute *cattr;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
HeapTupleHeader tup = tuple->t_data;
@@ -1099,6 +1100,13 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
values = slot->tts_values;
slot->tts_nvalid = reqnatts;
+ /*
+ * We store the tupleDesc's CompactAttribute array in 'cattrs' as gcc
+ * seems to be unwilling to optimize accessing the CompactAttribute
+ * element efficiently when accessing it via TupleDescCompactAttr().
+ */
+ cattrs = tupleDesc->compact_attrs;
+
/* Ensure we calculated tp correctly */
Assert(tp == (char *) tup + tup->t_hoff);
@@ -1109,7 +1117,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
for (; attnum < firstNonGuaranteedAttr; attnum++)
{
isnull[attnum] = false;
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
attlen = cattr->attlen;
/* We don't expect any non-byval types */
@@ -1149,9 +1157,8 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
for (; attnum < firstNonCacheOffsetAttr; attnum++)
{
isnull[attnum] = false;
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
attlen = cattr->attlen;
-
off = cattr->attcacheoff;
values[attnum] = fetch_att_noerr(tp + off,
cattr->attbyval,
@@ -1177,7 +1184,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int attlen;
isnull[attnum] = false;
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
attlen = cattr->attlen;
/*
@@ -1210,7 +1217,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
continue;
}
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
attlen = cattr->attlen;
/* As above, we don't expect cstrings */
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index ad7bc013812..e98036b58bf 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -55,7 +55,7 @@ typedef struct TupleConstr
* directly after the FormData_pg_attribute struct is populated or
* altered in any way.
*
- * Currently, this struct is 16 bytes. Any code changes which enlarge this
+ * Currently, this struct is 8 bytes. Any code changes which enlarge this
* struct should be considered very carefully.
*
* Code which must access a TupleDesc's attribute data should always make use
@@ -67,17 +67,17 @@ typedef struct TupleConstr
*/
typedef struct CompactAttribute
{
- int32 attcacheoff; /* fixed offset into tuple, if known, or -1 */
+ int16 attcacheoff; /* fixed offset into tuple, if known, or -1 */
int16 attlen; /* attr len in bytes or -1 = varlen, -2 =
* cstring */
bool attbyval; /* as FormData_pg_attribute.attbyval */
- bool attispackable; /* FormData_pg_attribute.attstorage !=
- * TYPSTORAGE_PLAIN */
- bool atthasmissing; /* as FormData_pg_attribute.atthasmissing */
- bool attisdropped; /* as FormData_pg_attribute.attisdropped */
- bool attgenerated; /* FormData_pg_attribute.attgenerated != '\0' */
- char attnullability; /* status of not-null constraint, see below */
uint8 attalignby; /* alignment requirement in bytes */
+ bool attispackable:1; /* FormData_pg_attribute.attstorage !=
+ * TYPSTORAGE_PLAIN */
+ bool atthasmissing:1; /* as FormData_pg_attribute.atthasmissing */
+ bool attisdropped:1; /* as FormData_pg_attribute.attisdropped */
+ bool attgenerated:1; /* FormData_pg_attribute.attgenerated != '\0' */
+ char attnullability; /* status of not-null constraint, see below */
} CompactAttribute;
/* Valid values for CompactAttribute->attnullability */
--
2.51.0
From 894745116658acec4a5a53657af465255cca7068 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Wed, 4 Mar 2026 16:55:09 +1300
Subject: [PATCH v12 6/6] WIP: Introduce selective tuple deforming
Up until now, we have always deformed every attribute of each tuple up
until the last attribute that we require. This did once make sense to
do as we often had to walk the tuple in order to determine the byte
offset to any attribute that we deform. Now, since we proactively
populate the CompactAttribute.attcacheoff, in some cases we might be in
a better position to only deform the attributes that we need to deform
by directly jumping to the cached byte offset and skipping deforming any
attributes which are not required. In some cases the savings can be
very large as it not only allows tts_values and tts_isnull for unneeded
attributes to be left unpopulated, but it might also mean we can skip
loading entire cachelines when the tuple is wide enough to span multiple
cachelines.
We don't want to exclusively always deform tuples this way as doing this
means paying attention to an additional array which states which attnums
we must deform. Looking at that array for a SELECT * query, which
requires us to deform all attributes, would add overhead. To support
this a new expression evaluation operator has been added called
EEOP_SCAN_SELECTSOME and each function which builds an ExprState now
accepts a variant function that allows the caller to specify which attnums
are required from the scan side. This puts it on the caller to decide
which type of deforming should be done. When the caller provides
the attnums, the expression will be built with EEOP_SCAN_SELECTSOME
rather than EEOP_SCAN_FETCHSOME. This currently does not interact well
with the physical tlist optimization. Currently it's the planner's job
to figure out which attributes are actually required.
TODO: JIT support
---
src/backend/executor/execExpr.c | 166 ++++++++++-
src/backend/executor/execExprInterp.c | 13 +
src/backend/executor/execScan.c | 18 ++
src/backend/executor/execTuples.c | 362 +++++++++++++++++++++++-
src/backend/executor/execUtils.c | 47 ++-
src/backend/executor/nodeSeqscan.c | 8 +-
src/backend/optimizer/plan/createplan.c | 43 ++-
src/include/executor/execExpr.h | 24 +-
src/include/executor/executor.h | 19 ++
src/include/executor/tuptable.h | 22 ++
src/include/nodes/plannodes.h | 8 +
11 files changed, 702 insertions(+), 28 deletions(-)
diff --git a/src/backend/executor/execExpr.c b/src/backend/executor/execExpr.c
index 088eca24021..8a2bf04598b 100644
--- a/src/backend/executor/execExpr.c
+++ b/src/backend/executor/execExpr.c
@@ -66,6 +66,14 @@ typedef struct ExprSetupInfo
AttrNumber last_new;
/* MULTIEXPR SubPlan nodes appearing in the expression: */
List *multiexpr_subplans;
+
+ /*
+ * Fetch only these attnums from the scan with EEOP_SCAN_SELECTSOME. Empty
+ * set means use EEOP_SCAN_FETCHSOME (i.e fetch all up until last_scan).
+ * The first user attribute is based at member 0. System attributes not
+ * represented.
+ */
+ Bitmapset *scan_attrs;
} ExprSetupInfo;
static void ExecReadyExpr(ExprState *state);
@@ -77,7 +85,8 @@ static void ExecInitFunc(ExprEvalStep *scratch, Expr *node, List *args,
static void ExecInitSubPlanExpr(SubPlan *subplan,
ExprState *state,
Datum *resv, bool *resnull);
-static void ExecCreateExprSetupSteps(ExprState *state, Node *node);
+static void ExecCreateExprSetupSteps(ExprState *state, Node *node,
+ Bitmapset *scan_attrs);
static void ExecPushExprSetupSteps(ExprState *state, ExprSetupInfo *info);
static bool expr_setup_walker(Node *node, ExprSetupInfo *info);
static bool ExecComputeSlotInfo(ExprState *state, ExprEvalStep *op);
@@ -141,6 +150,19 @@ static void ExecInitJsonCoercion(ExprState *state, JsonReturning *returning,
*/
ExprState *
ExecInitExpr(Expr *node, PlanState *parent)
+{
+ return ExecInitExprWithScanAttrs(node, parent, NULL);
+}
+
+/*
+ * ExecInitExprWithScanAttrs
+ * As ExecInitExpr but when 'scan_attrs' is set, use
+ * EEOP_SCAN_SELECTSOME instead of EEOP_SCAN_FETCHSOME to deform only
+ * the mentioned 'scan_attrs' from the scan tuple.
+ */
+ExprState *
+ExecInitExprWithScanAttrs(Expr *node, PlanState *parent,
+ Bitmapset *scan_attrs)
{
ExprState *state;
ExprEvalStep scratch = {0};
@@ -156,7 +178,7 @@ ExecInitExpr(Expr *node, PlanState *parent)
state->ext_params = NULL;
/* Insert setup steps as needed */
- ExecCreateExprSetupSteps(state, (Node *) node);
+ ExecCreateExprSetupSteps(state, (Node *) node, scan_attrs);
/* Compile the expression proper */
ExecInitExprRec(node, state, &state->resvalue, &state->resnull);
@@ -193,7 +215,7 @@ ExecInitExprWithParams(Expr *node, ParamListInfo ext_params)
state->ext_params = ext_params;
/* Insert setup steps as needed */
- ExecCreateExprSetupSteps(state, (Node *) node);
+ ExecCreateExprSetupSteps(state, (Node *) node, NULL);
/* Compile the expression proper */
ExecInitExprRec(node, state, &state->resvalue, &state->resnull);
@@ -227,6 +249,19 @@ ExecInitExprWithParams(Expr *node, ParamListInfo ext_params)
*/
ExprState *
ExecInitQual(List *qual, PlanState *parent)
+{
+ return ExecInitQualWithScanAttrs(qual, parent, NULL);
+}
+
+/*
+ * ExecInitQualWithScanAttrs
+ * As ExecInitQual but when 'scan_attrs' is set, use
+ * EEOP_SCAN_SELECTSOME instead of EEOP_SCAN_FETCHSOME to deform only
+ * the mentioned 'scan_attrs' from the scan tuple.
+ */
+ExprState *
+ExecInitQualWithScanAttrs(List *qual, PlanState *parent,
+ Bitmapset *scan_attrs)
{
ExprState *state;
ExprEvalStep scratch = {0};
@@ -247,7 +282,7 @@ ExecInitQual(List *qual, PlanState *parent)
state->flags = EEO_FLAG_IS_QUAL;
/* Insert setup steps as needed */
- ExecCreateExprSetupSteps(state, (Node *) qual);
+ ExecCreateExprSetupSteps(state, (Node *) qual, scan_attrs);
/*
* ExecQual() needs to return false for an expression returning NULL. That
@@ -372,6 +407,28 @@ ExecBuildProjectionInfo(List *targetList,
TupleTableSlot *slot,
PlanState *parent,
TupleDesc inputDesc)
+{
+ return ExecBuildProjectionInfoWithScanAttrs(targetList,
+ econtext,
+ slot,
+ parent,
+ inputDesc,
+ NULL);
+}
+
+/*
+ * ExecBuildProjectionInfoWithScanAttrs
+ * As ExecBuildProjectionInfo but when 'scan_attrs' is set, use
+ * EEOP_SCAN_SELECTSOME instead of EEOP_SCAN_FETCHSOME to deform only
+ * the mentioned 'scan_attrs' from the scan tuple.
+ */
+ProjectionInfo *
+ExecBuildProjectionInfoWithScanAttrs(List *targetList,
+ ExprContext *econtext,
+ TupleTableSlot *slot,
+ PlanState *parent,
+ TupleDesc inputDesc,
+ Bitmapset *scan_attrs)
{
ProjectionInfo *projInfo = makeNode(ProjectionInfo);
ExprState *state;
@@ -389,7 +446,7 @@ ExecBuildProjectionInfo(List *targetList,
state->resultslot = slot;
/* Insert setup steps as needed */
- ExecCreateExprSetupSteps(state, (Node *) targetList);
+ ExecCreateExprSetupSteps(state, (Node *) targetList, scan_attrs);
/* Now compile each tlist column */
foreach(lc, targetList)
@@ -2871,11 +2928,19 @@ ExecInitSubPlanExpr(SubPlan *subplan,
/*
* Add expression steps performing setup that's needed before any of the
* main execution of the expression.
+ *
+ * 'scan_attrs' may be given an empty set, in which case deforming the scan
+ * tuple is done via EEOP_SCAN_FETCHSOME, which fetches every attribute from
+ * the scan tuple up until the maximum attribute used by this expression.
+ * When 'scan_attrs' is set, EEOP_SCAN_SELECTSOME is used to only fetch the
+ * attributes mentioned. Callers must create a unioned set of the attributes
+ * needed from the scan for all expressions using the given slot so that we
+ * incrementally fetch the attributes required by all ExprStates.
*/
static void
-ExecCreateExprSetupSteps(ExprState *state, Node *node)
+ExecCreateExprSetupSteps(ExprState *state, Node *node, Bitmapset *scan_attrs)
{
- ExprSetupInfo info = {0, 0, 0, 0, 0, NIL};
+ ExprSetupInfo info = {0, 0, 0, 0, 0, NIL, scan_attrs};
/* Prescan to find out what we need. */
expr_setup_walker(node, &info);
@@ -2923,11 +2988,75 @@ ExecPushExprSetupSteps(ExprState *state, ExprSetupInfo *info)
}
if (info->last_scan > 0)
{
- scratch.opcode = EEOP_SCAN_FETCHSOME;
- scratch.d.fetch.last_var = info->last_scan;
- scratch.d.fetch.fixed = false;
- scratch.d.fetch.kind = NULL;
- scratch.d.fetch.known_desc = NULL;
+ /*
+ * We have two operators for fetching attributes out of a tuple during
+ * scans. EEOP_SCAN_FETCHSOME deforms all attributes in the tuple up
+ * to the 'last_scan' attnum. This isn't ideal in some cases, as we
+ * may only need a few attributes, and those might be deep into the
+ * tuple. EEOP_SCAN_SELECTSOME is an operator that fetches only the
+ * required attributes from the tuple. When the attcacheoff for these
+ * attributes is known and no NULLs exist in the tuple prior to the
+ * required attributes, then this can be a very fast operation.
+ * EEOP_SCAN_FETCHSOME is still supported as many cases require all
+ * attributes, and EEOP_SCAN_FETCHSOME can do this more efficiently.
+ */
+ if (bms_is_empty(info->scan_attrs))
+ {
+ scratch.opcode = EEOP_SCAN_FETCHSOME;
+ scratch.d.fetch.last_var = info->last_scan;
+ scratch.d.fetch.fixed = false;
+ scratch.d.fetch.kind = NULL;
+ scratch.d.fetch.known_desc = NULL;
+ }
+ else
+ {
+ int nattrs = bms_num_members(info->scan_attrs);
+ AttrNumber *atts;
+ int a;
+ int i;
+
+ scratch.opcode = EEOP_SCAN_SELECTSOME;
+ scratch.d.fetch.last_var = info->last_scan;
+ scratch.d.fetch.fixed = false;
+ scratch.d.fetch.natts = nattrs;
+ scratch.d.fetch.kind = NULL;
+ scratch.d.fetch.known_desc = NULL;
+
+ /*
+ * Allocate these two arrays as a single allocation. The
+ * req_attnums array needs 1 element for each attnum that's being
+ * selected, plus a sentinel attnum which we set to the
+ * 'last_scan' attnum so that we correctly terminate each of the
+ * loops during selective deformation before walking off the end
+ * of the array.
+ */
+ atts = palloc_array(AttrNumber, nattrs + 1 + info->last_scan + 1);
+
+ scratch.d.fetch.req_attnums = atts;
+ scratch.d.fetch.next_req_attnums_index = &atts[nattrs + 1];
+
+ /* Store each attnum in the Bitmapset into the req_attnum array */
+ a = -1;
+ i = 0;
+ while ((a = bms_next_member(info->scan_attrs, a)) >= 0)
+ scratch.d.fetch.req_attnums[i++] = a;
+
+ /* install sentinel */
+ scratch.d.fetch.req_attnums[nattrs] = info->last_scan;
+
+ /*
+ * Populate the next_req_attnums_index array. This allows the
+ * deforming function to refind the position in the
+ * next_req_attnums_index array from tts_nvalid.
+ */
+ a = 0;
+ for (i = 0; i <= info->last_scan; i++)
+ {
+ scratch.d.fetch.next_req_attnums_index[i] = a;
+ if (bms_is_member(i, info->scan_attrs))
+ a++;
+ }
+ }
if (ExecComputeSlotInfo(state, &scratch))
ExprEvalPushStep(state, &scratch);
}
@@ -3000,6 +3129,13 @@ expr_setup_walker(Node *node, ExprSetupInfo *info)
switch (variable->varreturningtype)
{
case VAR_RETURNING_DEFAULT:
+
+ /*
+ * scan_attrs must contain a member for this attnum or
+ * be completely empty
+ */
+ Assert(attnum < 0 || bms_is_empty(info->scan_attrs) ||
+ bms_is_member(attnum - 1, info->scan_attrs));
info->last_scan = Max(info->last_scan, attnum);
break;
case VAR_RETURNING_OLD:
@@ -3066,7 +3202,8 @@ ExecComputeSlotInfo(ExprState *state, ExprEvalStep *op)
opcode == EEOP_OUTER_FETCHSOME ||
opcode == EEOP_SCAN_FETCHSOME ||
opcode == EEOP_OLD_FETCHSOME ||
- opcode == EEOP_NEW_FETCHSOME);
+ opcode == EEOP_NEW_FETCHSOME ||
+ opcode == EEOP_SCAN_SELECTSOME);
if (op->d.fetch.known_desc != NULL)
{
@@ -3119,6 +3256,7 @@ ExecComputeSlotInfo(ExprState *state, ExprEvalStep *op)
}
}
else if (opcode == EEOP_SCAN_FETCHSOME ||
+ opcode == EEOP_SCAN_SELECTSOME ||
opcode == EEOP_OLD_FETCHSOME ||
opcode == EEOP_NEW_FETCHSOME)
{
@@ -4311,7 +4449,7 @@ ExecBuildHash32Expr(TupleDesc desc, const TupleTableSlotOps *ops,
state->parent = parent;
/* Insert setup steps as needed. */
- ExecCreateExprSetupSteps(state, (Node *) hash_exprs);
+ ExecCreateExprSetupSteps(state, (Node *) hash_exprs, NULL);
/*
* Make a place to store intermediate hash values between subsequent
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 61ff5ddc74c..76965826f83 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -479,6 +479,7 @@ ExecInterpExpr(ExprState *state, ExprContext *econtext, bool *isnull)
&&CASE_EEOP_SCAN_FETCHSOME,
&&CASE_EEOP_OLD_FETCHSOME,
&&CASE_EEOP_NEW_FETCHSOME,
+ &&CASE_EEOP_SCAN_SELECTSOME,
&&CASE_EEOP_INNER_VAR,
&&CASE_EEOP_OUTER_VAR,
&&CASE_EEOP_SCAN_VAR,
@@ -676,6 +677,18 @@ ExecInterpExpr(ExprState *state, ExprContext *econtext, bool *isnull)
EEO_NEXT();
}
+ EEO_CASE(EEOP_SCAN_SELECTSOME)
+ {
+ CheckOpSlotCompatibility(op, scanslot);
+
+ slot_selectattrs(scanslot,
+ op->d.fetch.last_var,
+ op->d.fetch.req_attnums,
+ op->d.fetch.next_req_attnums_index);
+
+ EEO_NEXT();
+ }
+
EEO_CASE(EEOP_INNER_VAR)
{
int attnum = op->d.var.attnum;
diff --git a/src/backend/executor/execScan.c b/src/backend/executor/execScan.c
index 9f68be17b99..525af11aa08 100644
--- a/src/backend/executor/execScan.c
+++ b/src/backend/executor/execScan.c
@@ -86,6 +86,24 @@ ExecAssignScanProjectionInfo(ScanState *node)
ExecConditionalAssignProjectionInfo(&node->ps, tupdesc, scan->scanrelid);
}
+/*
+ * ExecAssignScanProjectionInfoWithScanAttrs
+ * As ExecAssignScanProjectionInfo but when 'scan_attrs' is set, use
+ * EEOP_SCAN_SELECTSOME instead of EEOP_SCAN_FETCHSOME to deform only
+ * the mentioned 'scan_attrs' from the scan tuple.
+ */
+void
+ExecAssignScanProjectionInfoWithScanAttrs(ScanState *node,
+ Bitmapset *scan_attrs)
+{
+ Scan *scan = (Scan *) node->ps.plan;
+ TupleDesc tupdesc = node->ss_ScanTupleSlot->tts_tupleDescriptor;
+
+ ExecConditionalAssignProjectionInfoWithScanAttrs(&node->ps, tupdesc,
+ scan->scanrelid,
+ scan_attrs);
+}
+
/*
* ExecAssignScanProjectionInfoWithVarno
* As above, but caller can specify varno expected in Vars in the tlist.
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 1b5e1ebd334..5e502bfdbb8 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -74,6 +74,12 @@ static TupleDesc ExecTypeFromTLInternal(List *targetList,
bool skipjunk);
static pg_attribute_always_inline void slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int reqnatts);
+static pg_attribute_always_inline void slot_selectively_deform_heap_tuple(TupleTableSlot *slot,
+ HeapTuple tuple,
+ uint32 *offp,
+ int last_attnum,
+ AttrNumber *attnums,
+ AttrNumber *attnum_map);
static inline void tts_buffer_heap_store_tuple(TupleTableSlot *slot,
HeapTuple tuple,
Buffer buffer,
@@ -129,7 +135,22 @@ tts_virtual_clear(TupleTableSlot *slot)
static void
tts_virtual_getsomeattrs(TupleTableSlot *slot, int natts)
{
- elog(ERROR, "getsomeattrs is not required to be called on a virtual tuple table slot");
+ elog(ERROR,
+ "%s is not required to be called on a virtual tuple table slot",
+ "getsomeattrs");
+}
+
+/*
+ * VirtualTupleTableSlots always have fully populated tts_values and
+ * tts_isnull arrays. So this function should never be called.
+ */
+static void
+tts_virtual_selectattrs(TupleTableSlot *slot, int last_attnum,
+ AttrNumber *attnums, AttrNumber *attnum_map)
+{
+ elog(ERROR,
+ "%s is not required to be called on a virtual tuple table slot",
+ "selectattrs");
}
/*
@@ -352,6 +373,22 @@ tts_heap_getsomeattrs(TupleTableSlot *slot, int natts)
slot_deform_heap_tuple(slot, hslot->tuple, &hslot->off, natts);
}
+static void
+tts_heap_selectattrs(TupleTableSlot *slot, int last_attnum,
+ AttrNumber *attnums, AttrNumber *attnum_map)
+{
+ HeapTupleTableSlot *hslot = (HeapTupleTableSlot *) slot;
+
+ Assert(!TTS_EMPTY(slot));
+
+ slot_selectively_deform_heap_tuple(slot,
+ hslot->tuple,
+ &hslot->off,
+ last_attnum,
+ attnums,
+ attnum_map);
+}
+
static Datum
tts_heap_getsysattr(TupleTableSlot *slot, int attnum, bool *isnull)
{
@@ -550,6 +587,22 @@ tts_minimal_getsomeattrs(TupleTableSlot *slot, int natts)
slot_deform_heap_tuple(slot, mslot->tuple, &mslot->off, natts);
}
+static void
+tts_minimal_selectattrs(TupleTableSlot *slot, int last_attnum,
+ AttrNumber *attnums, AttrNumber *attnum_map)
+{
+ MinimalTupleTableSlot *mslot = (MinimalTupleTableSlot *) slot;
+
+ Assert(!TTS_EMPTY(slot));
+
+ slot_selectively_deform_heap_tuple(slot,
+ mslot->tuple,
+ &mslot->off,
+ last_attnum,
+ attnums,
+ attnum_map);
+}
+
/*
* MinimalTupleTableSlots never provide system attributes. We generally
* shouldn't get here, but provide a user-friendly message if we do.
@@ -757,6 +810,23 @@ tts_buffer_heap_getsomeattrs(TupleTableSlot *slot, int natts)
slot_deform_heap_tuple(slot, bslot->base.tuple, &bslot->base.off, natts);
}
+static void
+tts_buffer_heap_selectattrs(TupleTableSlot *slot, int last_attnum,
+ AttrNumber *attnums, AttrNumber *attnum_map)
+{
+ BufferHeapTupleTableSlot *bslot = (BufferHeapTupleTableSlot *) slot;
+
+ Assert(!TTS_EMPTY(slot));
+
+ slot_selectively_deform_heap_tuple(slot,
+ bslot->base.tuple,
+ &bslot->base.off,
+ last_attnum,
+ attnums,
+ attnum_map);
+}
+
+
static Datum
tts_buffer_heap_getsysattr(TupleTableSlot *slot, int attnum, bool *isnull)
{
@@ -1244,12 +1314,299 @@ done:
*offp = off;
}
+/*
+ * slot_selectively_deform_heap_tuple
+ * Deform attributes of 'tuple' into the Datum/isnull arrays in 'slot'.
+ * Unlike slot_deform_heap_tuple, which deforms every attribute up to the
+ * given attribute number, here we deform only the attribute numbers
+ * mentioned in the 'attnums' array. When only a few attributes are
+ * required, this can be more efficient. When the attributes have a
+ * known attcacheoff and it's valid to use that, then this version can be
+ * much more efficient than slot_deform_heap_tuple when only a small
+ * number of the total attributes are required.
+ */
+static pg_attribute_always_inline void
+slot_selectively_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple,
+ uint32 *offp, int last_attnum,
+ AttrNumber *attnums,
+ AttrNumber *attnum_map)
+{
+ CompactAttribute *cattrs;
+ CompactAttribute *cattr;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+ HeapTupleHeader tup = tuple->t_data;
+ size_t attnum;
+ int attnums_idx;
+ int firstNonCacheOffsetAttr;
+ int firstNonGuaranteedAttr;
+ int firstNullAttr;
+ int natts;
+ Datum *values;
+ bool *isnull;
+ char *tp; /* ptr to tuple data */
+ uint32 off; /* offset in tuple data */
+ int off_attnum; /* the attnum that 'off' points to */
+
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
+
+ isnull = slot->tts_isnull;
+
+ /*
+ * Some callers may form and deform tuples prior to NOT NULL constraints
+ * being checked. Here we'd like to optimize the case where we only need
+ * to fetch attributes before or up to the point where the attribute is
+ * guaranteed to exist in the tuple. We rely on the slot flag being set
+ * correctly to only enable this optimization when it's valid to do so.
+ * This optimization allows us to save fetching the number of attributes
+ * from the tuple and saves the additional cost of handling non-byval
+ * attrs.
+ */
+ firstNonGuaranteedAttr = Min(last_attnum, slot->tts_first_nonguaranteed);
+
+ firstNonCacheOffsetAttr = tupleDesc->firstNonCachedOffsetAttr;
+
+ if (HeapTupleHasNulls(tuple))
+ {
+ natts = HeapTupleHeaderGetNatts(tup);
+ tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits) +
+ BITMAPLEN(natts));
+
+ natts = Min(natts, last_attnum);
+ if (natts > firstNonGuaranteedAttr)
+ {
+ bits8 *bp = tup->t_bits;
+
+ /* Find the first NULL attr */
+ firstNullAttr = first_null_attr(bp, natts);
+
+ /*
+ * And populate the isnull array for all attributes up to the last
+ * attribute we're deforming. When not using attcacheoff, we need
+ * to know if an attribute is NULL even when we're not deforming
+ * it, so that we can skip over it when calculating the offset to
+ * attributes that we are deforming.
+ */
+ populate_isnull_array(bp, natts, isnull);
+
+ /* We can only use any cached offsets until the first NULL attr */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr,
+ firstNullAttr);
+ }
+ else
+ {
+ /* Otherwise all required columns are guaranteed to exist */
+ firstNullAttr = natts;
+ }
+ }
+ else
+ {
+ tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits));
+
+ /*
+ * We only need to look at the tuple's natts if we need more than the
+ * guaranteed number of columns
+ */
+ if (last_attnum > firstNonGuaranteedAttr)
+ natts = Min(HeapTupleHeaderGetNatts(tup), last_attnum);
+ else
+ {
+ /* No need to access the number of attributes in the tuple */
+ natts = last_attnum;
+ }
+
+ /* All attrs can be fetched without checking for NULLs */
+ firstNullAttr = natts;
+ }
+
+ attnums_idx = attnum_map[slot->tts_nvalid];
+ attnum = attnums[attnums_idx];
+ values = slot->tts_values;
+
+ /*
+ * We store the tupleDesc's CompactAttribute array in 'cattrs' as gcc
+ * seems to be unwilling to optimize accessing the CompactAttribute
+ * element efficiently when accessing it via TupleDescCompactAttr().
+ */
+ cattrs = tupleDesc->compact_attrs;
+
+ /* Ensure we calculated tp correctly */
+ Assert(tp == (char *) tup + tup->t_hoff);
+
+ if (attnum < firstNonGuaranteedAttr)
+ {
+ int attlen;
+
+ for (; attnum < firstNonGuaranteedAttr; attnum = attnums[++attnums_idx])
+ {
+ isnull[attnum] = false;
+ cattr = &cattrs[attnum];
+ attlen = cattr->attlen;
+
+ /* We don't expect any non-byval types */
+ pg_assume(attlen > 0);
+ Assert(cattr->attbyval == true);
+
+ off = cattr->attcacheoff;
+ values[attnum] = fetch_att_noerr(tp + off, true, attlen);
+ }
+
+ off += attlen;
+
+ if (attnum == last_attnum)
+ goto done;
+ }
+
+ /* We can only fetch as many attributes as the tuple has. */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, natts);
+
+ /*
+ * Handle the portion of the tuple that we have cached the offset for up
+ * to the first NULL attribute. The offset is effectively fixed for
+ * these, so we can use the CompactAttribute's attcacheoff.
+ */
+ if (attnum < firstNonCacheOffsetAttr)
+ {
+ int attlen;
+
+ for (; attnum < firstNonCacheOffsetAttr; attnum = attnums[++attnums_idx])
+ {
+ isnull[attnum] = false;
+ cattr = &cattrs[attnum];
+ attlen = cattr->attlen;
+
+ off = cattr->attcacheoff;
+ values[attnum] = fetch_att_noerr(tp + off, cattr->attbyval,
+ attlen);
+ }
+
+ off += attlen;
+ Assert(attlen > 0);
+
+ if (attnum == last_attnum)
+ goto done;
+ }
+
+
+ if (slot->tts_nvalid >= firstNonCacheOffsetAttr)
+ {
+ /* Restore state from previous execution */
+ off_attnum = slot->tts_nvalid;
+ off = *offp;
+ }
+ else
+ {
+ off_attnum = firstNonCacheOffsetAttr - 1;
+ off = cattrs[off_attnum].attcacheoff;
+ }
+
+ /*
+ * We no longer have the ability to use attcacheoff, so we must look
+ * through all attributes from this point on. For attributes that we are
+ * not selecting, we only calculate the offset to skip them, and don't do
+ * the actual fetch. Here we loop up to the first NULL attribute.
+ */
+ for (; off_attnum < firstNullAttr; off_attnum++)
+ {
+ int attlen;
+
+ cattr = &cattrs[off_attnum];
+ attlen = cattr->attlen;
+
+ /*
+ * cstrings don't exist in heap tuples. Use pg_assume to instruct the
+ * compiler not to emit the cstring-related code in
+ * align_fetch_then_add().
+ */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ off = att_pointer_alignby(off, cattr->attalignby, attlen, tp + off);
+
+ /*
+ * If this is an attribute we want, do the fetch and then move attnum
+ * to the next attribute we want.
+ */
+ if (off_attnum == attnum)
+ {
+ isnull[off_attnum] = false;
+ values[off_attnum] =
+ fetch_att_noerr(tp + off, cattr->attbyval,
+ attlen);
+ attnum = attnums[++attnums_idx];
+
+ }
+ /* Move offset beyond this attribute */
+ off = att_addlength_pointer(off, attlen, tp + off);
+ }
+
+ /*
+ * Now handle any remaining attributes in the tuple up to the requested
+ * attnum. This time, include NULL checks as we're now at the first NULL
+ * attribute.
+ */
+ for (; off_attnum < natts; off_attnum++)
+ {
+ int attlen;
+
+ cattr = &cattrs[off_attnum];
+ attlen = cattr->attlen;
+
+ /* As above, we don't expect cstrings */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ /* Is this an attribute we're selecting? */
+ if (off_attnum == attnum)
+ {
+ attnum = attnums[++attnums_idx];
+
+ if (isnull[off_attnum])
+ {
+ values[off_attnum] = (Datum) 0;
+ continue;
+ }
+
+ /*
+ * align 'off', fetch the datum, and increment off beyond the
+ * datum
+ */
+ values[off_attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ attlen,
+ cattr->attalignby);
+ }
+ else if (!isnull[off_attnum])
+ {
+ /* We don't want this attribute, move beyond it */
+ off = att_pointer_alignby(off, cattr->attalignby, attlen, tp + off);
+ off = att_addlength_pointer(off, attlen, tp + off);
+ }
+
+ }
+
+ /* Fetch any missing attrs and raise an error if reqnatts is invalid */
+ if (unlikely(attnum < last_attnum))
+ {
+ *offp = off;
+ /* XXX worth doing this selectively too? */
+ slot_getmissingattrs(slot, attnum, last_attnum);
+ slot->tts_nvalid = last_attnum;
+ return;
+ }
+done:
+
+ slot->tts_nvalid = last_attnum;
+ /* Save current offset for next execution */
+ *offp = off;
+}
+
const TupleTableSlotOps TTSOpsVirtual = {
.base_slot_size = sizeof(VirtualTupleTableSlot),
.init = tts_virtual_init,
.release = tts_virtual_release,
.clear = tts_virtual_clear,
.getsomeattrs = tts_virtual_getsomeattrs,
+ .selectattrs = tts_virtual_selectattrs,
.getsysattr = tts_virtual_getsysattr,
.materialize = tts_virtual_materialize,
.is_current_xact_tuple = tts_virtual_is_current_xact_tuple,
@@ -1271,6 +1628,7 @@ const TupleTableSlotOps TTSOpsHeapTuple = {
.release = tts_heap_release,
.clear = tts_heap_clear,
.getsomeattrs = tts_heap_getsomeattrs,
+ .selectattrs = tts_heap_selectattrs,
.getsysattr = tts_heap_getsysattr,
.is_current_xact_tuple = tts_heap_is_current_xact_tuple,
.materialize = tts_heap_materialize,
@@ -1289,6 +1647,7 @@ const TupleTableSlotOps TTSOpsMinimalTuple = {
.release = tts_minimal_release,
.clear = tts_minimal_clear,
.getsomeattrs = tts_minimal_getsomeattrs,
+ .selectattrs = tts_minimal_selectattrs,
.getsysattr = tts_minimal_getsysattr,
.is_current_xact_tuple = tts_minimal_is_current_xact_tuple,
.materialize = tts_minimal_materialize,
@@ -1307,6 +1666,7 @@ const TupleTableSlotOps TTSOpsBufferHeapTuple = {
.release = tts_buffer_heap_release,
.clear = tts_buffer_heap_clear,
.getsomeattrs = tts_buffer_heap_getsomeattrs,
+ .selectattrs = tts_buffer_heap_selectattrs,
.getsysattr = tts_buffer_heap_getsysattr,
.is_current_xact_tuple = tts_buffer_is_current_xact_tuple,
.materialize = tts_buffer_heap_materialize,
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index f62582859f9..252e8306631 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -580,8 +580,7 @@ ExecGetCommonChildSlotOps(PlanState *ps)
* ----------------
*/
void
-ExecAssignProjectionInfo(PlanState *planstate,
- TupleDesc inputDesc)
+ExecAssignProjectionInfo(PlanState *planstate, TupleDesc inputDesc)
{
planstate->ps_ProjInfo =
ExecBuildProjectionInfo(planstate->plan->targetlist,
@@ -591,6 +590,28 @@ ExecAssignProjectionInfo(PlanState *planstate,
inputDesc);
}
+/* ----------------
+ * ExecAssignProjectionInfoWithScanAttrs
+ *
+ * As ExecAssignScanProjectionInfo but when 'scan_attrs' is set, use
+ * EEOP_SCAN_SELECTSOME instead of EEOP_SCAN_FETCHSOME to deform only the
+ * mentioned 'scan_attrs' from the scan tuple.
+ * ----------------
+*/
+void
+ExecAssignProjectionInfoWithScanAttrs(PlanState *planstate,
+ TupleDesc inputDesc,
+ Bitmapset *scan_attrs)
+{
+ planstate->ps_ProjInfo =
+ ExecBuildProjectionInfoWithScanAttrs(planstate->plan->targetlist,
+ planstate->ps_ExprContext,
+ planstate->ps_ResultTupleSlot,
+ planstate,
+ inputDesc,
+ scan_attrs);
+}
+
/* ----------------
* ExecConditionalAssignProjectionInfo
@@ -602,6 +623,26 @@ ExecAssignProjectionInfo(PlanState *planstate,
void
ExecConditionalAssignProjectionInfo(PlanState *planstate, TupleDesc inputDesc,
int varno)
+{
+ ExecConditionalAssignProjectionInfoWithScanAttrs(planstate,
+ inputDesc,
+ varno,
+ NULL);
+}
+
+/* ----------------
+ * ExecConditionalAssignProjectionInfoWithScanAttrs
+ *
+ * As ExecConditionalAssignProjectionInfo but when 'scan_attrs' is set, use
+ * EEOP_SCAN_SELECTSOME instead of EEOP_SCAN_FETCHSOME to deform only the
+ * mentioned 'scan_attrs' from the scan tuple.
+ * ----------------
+*/
+void
+ExecConditionalAssignProjectionInfoWithScanAttrs(PlanState *planstate,
+ TupleDesc inputDesc,
+ int varno,
+ Bitmapset *scan_attrs)
{
if (tlist_matches_tupdesc(planstate,
planstate->plan->targetlist,
@@ -622,7 +663,7 @@ ExecConditionalAssignProjectionInfo(PlanState *planstate, TupleDesc inputDesc,
planstate->resultopsfixed = true;
planstate->resultopsset = true;
}
- ExecAssignProjectionInfo(planstate, inputDesc);
+ ExecAssignProjectionInfoWithScanAttrs(planstate, inputDesc, scan_attrs);
}
}
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 8f219f60a93..41de367832c 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -251,13 +251,15 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags)
* Initialize result type and projection.
*/
ExecInitResultTypeTL(&scanstate->ss.ps);
- ExecAssignScanProjectionInfo(&scanstate->ss);
+ ExecAssignScanProjectionInfoWithScanAttrs(&scanstate->ss,
+ node->scan.scan_varattnos);
/*
* initialize child expressions
*/
- scanstate->ss.ps.qual =
- ExecInitQual(node->scan.plan.qual, (PlanState *) scanstate);
+ scanstate->ss.ps.qual = ExecInitQualWithScanAttrs(node->scan.plan.qual,
+ (PlanState *) scanstate,
+ node->scan.scan_varattnos);
/*
* When EvalPlanQual() is not in use, assign ExecProcNode for this node
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 50b0e10308b..4522ac4d4c0 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -118,7 +118,8 @@ static ModifyTable *create_modifytable_plan(PlannerInfo *root, ModifyTablePath *
static Limit *create_limit_plan(PlannerInfo *root, LimitPath *best_path,
int flags);
static SeqScan *create_seqscan_plan(PlannerInfo *root, Path *best_path,
- List *tlist, List *scan_clauses);
+ List *tlist, List *scan_clauses,
+ Bitmapset *tlist_varattnos);
static SampleScan *create_samplescan_plan(PlannerInfo *root, Path *best_path,
List *tlist, List *scan_clauses);
static Scan *create_indexscan_plan(PlannerInfo *root, IndexPath *best_path,
@@ -178,7 +179,8 @@ static void label_sort_with_costsize(PlannerInfo *root, Sort *plan,
double limit_tuples);
static void label_incrementalsort_with_costsize(PlannerInfo *root, IncrementalSort *plan,
List *pathkeys, double limit_tuples);
-static SeqScan *make_seqscan(List *qptlist, List *qpqual, Index scanrelid);
+static SeqScan *make_seqscan(List *qptlist, List *qpqual, Index scanrelid,
+ Bitmapset *scan_varattnos);
static SampleScan *make_samplescan(List *qptlist, List *qpqual, Index scanrelid,
TableSampleClause *tsc);
static IndexScan *make_indexscan(List *qptlist, List *qpqual, Index scanrelid,
@@ -550,6 +552,7 @@ create_plan_recurse(PlannerInfo *root, Path *best_path, int flags)
static Plan *
create_scan_plan(PlannerInfo *root, Path *best_path, int flags)
{
+ Bitmapset *tlist_varattnos = NULL;
RelOptInfo *rel = best_path->parent;
List *scan_clauses;
List *gating_clauses;
@@ -579,6 +582,14 @@ create_scan_plan(PlannerInfo *root, Path *best_path, int flags)
break;
}
+ /*
+ * Figure out which attributes we need from the scan before applying the
+ * physical tlist optimization.
+ */
+ pull_varattnos((Node *) best_path->pathtarget->exprs,
+ rel->relid,
+ &tlist_varattnos);
+
/*
* If this is a parameterized scan, we also need to enforce all the join
* clauses available from the outer relation(s).
@@ -672,7 +683,8 @@ create_scan_plan(PlannerInfo *root, Path *best_path, int flags)
plan = (Plan *) create_seqscan_plan(root,
best_path,
tlist,
- scan_clauses);
+ scan_clauses,
+ tlist_varattnos);
break;
case T_SampleScan:
@@ -2752,10 +2764,13 @@ create_limit_plan(PlannerInfo *root, LimitPath *best_path, int flags)
*/
static SeqScan *
create_seqscan_plan(PlannerInfo *root, Path *best_path,
- List *tlist, List *scan_clauses)
+ List *tlist, List *scan_clauses, Bitmapset *tlist_varattnos)
{
SeqScan *scan_plan;
Index scan_relid = best_path->parent->relid;
+ Bitmapset *scan_varattnos = tlist_varattnos;
+ Bitmapset *non_sys_attrs = NULL;
+ int i;
/* it should be a base rel... */
Assert(scan_relid > 0);
@@ -2767,6 +2782,19 @@ create_seqscan_plan(PlannerInfo *root, Path *best_path,
/* Reduce RestrictInfo list to bare expressions; ignore pseudoconstants */
scan_clauses = extract_actual_clauses(scan_clauses, false);
+ /* Pull varattnos from WHERE clause Vars */
+ pull_varattnos((Node *) scan_clauses, scan_relid, &scan_varattnos);
+
+ /* Don't set these when whole-row var is present */
+ if (!bms_is_member(0 - FirstLowInvalidHeapAttributeNumber, scan_varattnos))
+ {
+ /* XXX invent bms_right_shift_members()? */
+ i = 0 - FirstLowInvalidHeapAttributeNumber;
+ while ((i = bms_next_member(scan_varattnos, i)) >= 0)
+ non_sys_attrs = bms_add_member(non_sys_attrs,
+ i - 1 + FirstLowInvalidHeapAttributeNumber);
+ }
+
/* Replace any outer-relation variables with nestloop params */
if (best_path->param_info)
{
@@ -2776,7 +2804,8 @@ create_seqscan_plan(PlannerInfo *root, Path *best_path,
scan_plan = make_seqscan(tlist,
scan_clauses,
- scan_relid);
+ scan_relid,
+ non_sys_attrs);
copy_generic_path_info(&scan_plan->scan.plan, best_path);
@@ -5487,7 +5516,8 @@ bitmap_subplan_mark_shared(Plan *plan)
static SeqScan *
make_seqscan(List *qptlist,
List *qpqual,
- Index scanrelid)
+ Index scanrelid,
+ Bitmapset *scan_varattnos)
{
SeqScan *node = makeNode(SeqScan);
Plan *plan = &node->scan.plan;
@@ -5497,6 +5527,7 @@ make_seqscan(List *qptlist,
plan->lefttree = NULL;
plan->righttree = NULL;
node->scan.scanrelid = scanrelid;
+ node->scan.scan_varattnos = scan_varattnos;
return node;
}
diff --git a/src/include/executor/execExpr.h b/src/include/executor/execExpr.h
index aa9b361fa31..f29d9dd799b 100644
--- a/src/include/executor/execExpr.h
+++ b/src/include/executor/execExpr.h
@@ -78,6 +78,9 @@ typedef enum ExprEvalOp
EEOP_OLD_FETCHSOME,
EEOP_NEW_FETCHSOME,
+ /* apply slot_selectattrs on the corresponding tuple slot */
+ EEOP_SCAN_SELECTSOME,
+
/* compute non-system Var value */
EEOP_INNER_VAR,
EEOP_OUTER_VAR,
@@ -318,15 +321,34 @@ typedef struct ExprEvalStep
*/
union
{
- /* for EEOP_INNER/OUTER/SCAN/OLD/NEW_FETCHSOME */
+ /*
+ * for EEOP_INNER/OUTER/SCAN/OLD/NEW_FETCHSOME and
+ * EEOP_SCAN_SELECTSOME
+ */
struct
{
/* attribute number up to which to fetch (inclusive) */
int last_var;
/* will the type of slot be the same for every invocation */
bool fixed;
+ /* Number of elements in req_attnums array. XXX needed? */
+ AttrNumber natts;
+
+ /* One element for each attnum to select, ordered by attnum */
+ AttrNumber *req_attnums;
+
+ /*
+ * Provides mapping of 0-based attnums back to the index of the
+ * req_attnums array that deforming should continue from. This
+ * allows us to re-find the element of req_attnums using the
+ * slot's tts_nvalid so that we can continue deforming from the
+ * last defromed attribute.
+ */
+ AttrNumber *next_req_attnums_index;
+
/* tuple descriptor, if known */
TupleDesc known_desc;
+
/* type of slot, can only be relied upon if fixed is set */
const TupleTableSlotOps *kind;
} fetch;
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index bf239fc156f..69f3c5da6c5 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -324,8 +324,12 @@ ExecProcNode(PlanState *node)
* prototypes from functions in execExpr.c
*/
extern ExprState *ExecInitExpr(Expr *node, PlanState *parent);
+extern ExprState *ExecInitExprWithScanAttrs(Expr *node, PlanState *parent,
+ Bitmapset *scan_attrs);
extern ExprState *ExecInitExprWithParams(Expr *node, ParamListInfo ext_params);
extern ExprState *ExecInitQual(List *qual, PlanState *parent);
+extern ExprState *ExecInitQualWithScanAttrs(List *qual, PlanState *parent,
+ Bitmapset *scan_attrs);
extern ExprState *ExecInitCheck(List *qual, PlanState *parent);
extern List *ExecInitExprList(List *nodes, PlanState *parent);
extern ExprState *ExecBuildAggTrans(AggState *aggstate, struct AggStatePerPhaseData *phase,
@@ -364,6 +368,12 @@ extern ProjectionInfo *ExecBuildProjectionInfo(List *targetList,
TupleTableSlot *slot,
PlanState *parent,
TupleDesc inputDesc);
+extern ProjectionInfo *ExecBuildProjectionInfoWithScanAttrs(List *targetList,
+ ExprContext *econtext,
+ TupleTableSlot *slot,
+ PlanState *parent,
+ TupleDesc inputDesc,
+ Bitmapset *scan_attrs);
extern ProjectionInfo *ExecBuildUpdateProjection(List *targetList,
bool evalTargetList,
List *targetColnos,
@@ -582,6 +592,8 @@ typedef bool (*ExecScanRecheckMtd) (ScanState *node, TupleTableSlot *slot);
extern TupleTableSlot *ExecScan(ScanState *node, ExecScanAccessMtd accessMtd,
ExecScanRecheckMtd recheckMtd);
extern void ExecAssignScanProjectionInfo(ScanState *node);
+extern void ExecAssignScanProjectionInfoWithScanAttrs(ScanState *node,
+ Bitmapset *scan_attrs);
extern void ExecAssignScanProjectionInfoWithVarno(ScanState *node, int varno);
extern void ExecScanReScan(ScanState *node);
@@ -678,8 +690,15 @@ extern const TupleTableSlotOps *ExecGetCommonSlotOps(PlanState **planstates,
extern const TupleTableSlotOps *ExecGetCommonChildSlotOps(PlanState *ps);
extern void ExecAssignProjectionInfo(PlanState *planstate,
TupleDesc inputDesc);
+extern void ExecAssignProjectionInfoWithScanAttrs(PlanState *planstate,
+ TupleDesc inputDesc,
+ Bitmapset *scan_attrs);
extern void ExecConditionalAssignProjectionInfo(PlanState *planstate,
TupleDesc inputDesc, int varno);
+extern void ExecConditionalAssignProjectionInfoWithScanAttrs(PlanState *planstate,
+ TupleDesc inputDesc,
+ int varno,
+ Bitmapset *scan_attrs);
extern void ExecAssignScanType(ScanState *scanstate, TupleDesc tupDesc);
extern void ExecCreateScanSlotFromOuterPlan(EState *estate,
ScanState *scanstate,
diff --git a/src/include/executor/tuptable.h b/src/include/executor/tuptable.h
index 78558098fa3..cc2c5a257d0 100644
--- a/src/include/executor/tuptable.h
+++ b/src/include/executor/tuptable.h
@@ -168,6 +168,16 @@ struct TupleTableSlotOps
*/
void (*getsomeattrs) (TupleTableSlot *slot, int natts);
+ /*
+ * Populate the tts_values and tts_isnull elements of the given slot with
+ * the values of the corresponding attribute from the tuple stored in the
+ * slot. Populate up as far as last_attnum and store each attribute
+ * mentioned in the attnums array. Use attnum_map to determine the
+ * starting element in the attnums array from the slot's tts_nvalid.
+ */
+ void (*selectattrs) (TupleTableSlot *slot, int last_attnum,
+ AttrNumber *attnums, AttrNumber *attnum_map);
+
/*
* Returns value of the given system attribute as a datum and sets isnull
* to false, if it's not NULL. Throws an error if the slot type does not
@@ -374,6 +384,18 @@ slot_getsomeattrs(TupleTableSlot *slot, int attnum)
slot->tts_ops->getsomeattrs(slot, attnum);
}
+static inline void
+slot_selectattrs(TupleTableSlot *slot, int last_attnum, AttrNumber *attnums,
+ AttrNumber *attnum_map)
+{
+ /*
+ * Populate slot only attributes mentioned in the attnums array, up to
+ * 'last_attnum', if it's not already
+ */
+ if (slot->tts_nvalid < last_attnum)
+ slot->tts_ops->selectattrs(slot, last_attnum, attnums, attnum_map);
+}
+
/*
* slot_getallattrs
* This function forces all the entries of the slot's Datum/isnull
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 8c9321aab8c..08dcf02b8bb 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -540,6 +540,14 @@ typedef struct Scan
Plan plan;
/* relid is index into the range table */
Index scanrelid;
+
+ /*
+ * All varattnos that are required from the scanrelid. Does not include
+ * any added due to the physical tlist optimization or system attributes
+ * or whole-row attributes. User attributes are 0 based, i.e attnum==1 is
+ * member 0.
+ */
+ Bitmapset *scan_varattnos;
} Scan;
/* ----------------
--
2.51.0
Attachments:
[text/plain] v12-0001-Introduce-deform_bench-test-module.patch (7.3K, 2-v12-0001-Introduce-deform_bench-test-module.patch)
download | inline diff:
From 27cf8d0007487d9dd22a7b6e0562e0399403dfef Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Tue, 27 Jan 2026 15:08:09 +1300
Subject: [PATCH v12 1/6] Introduce deform_bench test module
For benchmarking tuple deformation.
---
src/test/modules/deform_bench/.gitignore | 4 +
src/test/modules/deform_bench/Makefile | 21 ++++
.../deform_bench/deform_bench--1.0.sql | 8 ++
src/test/modules/deform_bench/deform_bench.c | 107 ++++++++++++++++++
.../modules/deform_bench/deform_bench.control | 4 +
src/test/modules/deform_bench/meson.build | 22 ++++
src/test/modules/meson.build | 1 +
7 files changed, 167 insertions(+)
create mode 100644 src/test/modules/deform_bench/.gitignore
create mode 100644 src/test/modules/deform_bench/Makefile
create mode 100644 src/test/modules/deform_bench/deform_bench--1.0.sql
create mode 100644 src/test/modules/deform_bench/deform_bench.c
create mode 100644 src/test/modules/deform_bench/deform_bench.control
create mode 100644 src/test/modules/deform_bench/meson.build
diff --git a/src/test/modules/deform_bench/.gitignore b/src/test/modules/deform_bench/.gitignore
new file mode 100644
index 00000000000..5dcb3ff9723
--- /dev/null
+++ b/src/test/modules/deform_bench/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/src/test/modules/deform_bench/Makefile b/src/test/modules/deform_bench/Makefile
new file mode 100644
index 00000000000..b5fc0f7a583
--- /dev/null
+++ b/src/test/modules/deform_bench/Makefile
@@ -0,0 +1,21 @@
+# src/test/modules/deform_bench/Makefile
+
+MODULE_big = deform_bench
+OBJS = deform_bench.o
+
+EXTENSION = deform_bench
+DATA = deform_bench--1.0.sql
+PGFILEDESC = "deform_bench - tuple deform benchmarking"
+
+REGRESS = deform_bench
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = src/test/modules/deform_bench
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/src/test/modules/deform_bench/deform_bench--1.0.sql b/src/test/modules/deform_bench/deform_bench--1.0.sql
new file mode 100644
index 00000000000..492b71dba3b
--- /dev/null
+++ b/src/test/modules/deform_bench/deform_bench--1.0.sql
@@ -0,0 +1,8 @@
+/* deform_bench--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION deform_bench" to load this file. \quit
+
+CREATE FUNCTION deform_bench(tableoid Oid, attnum int[]) RETURNS FLOAT
+AS 'MODULE_PATHNAME', 'deform_bench'
+LANGUAGE C VOLATILE STRICT;
diff --git a/src/test/modules/deform_bench/deform_bench.c b/src/test/modules/deform_bench/deform_bench.c
new file mode 100644
index 00000000000..7838f639bef
--- /dev/null
+++ b/src/test/modules/deform_bench/deform_bench.c
@@ -0,0 +1,107 @@
+/*-------------------------------------------------------------------------
+ *
+ * deform_bench.c
+ *
+ * for benchmarking tuple deformation routines
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include <time.h>
+#include <sys/time.h>
+
+#include "access/heapam.h"
+#include "access/relscan.h"
+#include "catalog/pg_am_d.h"
+#include "catalog/pg_type_d.h"
+#include "funcapi.h"
+#include "miscadmin.h"
+#include "utils/array.h"
+#include "utils/arrayaccess.h"
+#include "utils/builtins.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(deform_bench);
+
+Datum
+deform_bench(PG_FUNCTION_ARGS)
+{
+ Oid tableoid = PG_GETARG_OID(0);
+ ArrayType *array = PG_GETARG_ARRAYTYPE_P(1);
+ TableScanDesc scan;
+ Relation rel;
+ TupleDesc tupdesc;
+ TupleTableSlot *slot;
+ Datum *elem_datums = NULL;
+ bool *elem_nulls = NULL;
+ int elem_count;
+ int *attnums;
+ clock_t start,
+ end;
+
+ rel = relation_open(tableoid, AccessShareLock);
+
+ if (rel->rd_rel->relam != HEAP_TABLE_AM_OID)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("only heap AM is supported")));
+
+ tupdesc = RelationGetDescr(rel);
+ slot = MakeTupleTableSlot(tupdesc, &TTSOpsBufferHeapTuple);
+ scan = table_beginscan_strat(rel, GetActiveSnapshot(), 0, NULL, true, false);
+
+ /*
+ * The array is used to allow callers to define how many atts to deform.
+ * e.g: '{1,10}'::int[] would deform attnum=1, then in a 2nd pass deform
+ * the remainder up to attnum=10. Passing an element as NULL means all
+ * attnums. This allows simulation of incremental deformation. Generally
+ * if you're passing an array with more than 1 element, then the array
+ * should be in ascending order. Doing something like '{10,1}' would mean
+ * we've already deformed 10 attributes and on the 2nd pass there's
+ * nothing to do since attnum=1 was already deformed in the first pass.
+ *
+ * You'll get an ERROR if you pass a number higher than the number of
+ * attributes in the table.
+ */
+ deconstruct_array(array,
+ INT4OID,
+ sizeof(int32),
+ true,
+ 'i',
+ &elem_datums,
+ &elem_nulls,
+ &elem_count);
+
+ attnums = palloc_array(int, elem_count);
+
+ for (int i = 0; i < elem_count; i++)
+ {
+ /* Make a NULL element mean all attributes */
+ if (elem_nulls[i])
+ attnums[i] = tupdesc->natts;
+ else
+ attnums[i] = DatumGetInt32(elem_datums[i]);
+ }
+
+ start = clock();
+
+ while (heap_getnextslot(scan, ForwardScanDirection, slot))
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ /* Deform in stages according to the attnums array */
+ for (int i = 0; i < elem_count; i++)
+ slot_getsomeattrs(slot, attnums[i]);
+ }
+
+ end = clock();
+
+ ExecDropSingleTupleTableSlot(slot);
+ table_endscan(scan);
+ relation_close(rel, AccessShareLock);
+
+
+ /* Returns the number of milliseconds to run the test */
+ PG_RETURN_FLOAT8((double) (end - start) / (CLOCKS_PER_SEC / 1000));
+}
diff --git a/src/test/modules/deform_bench/deform_bench.control b/src/test/modules/deform_bench/deform_bench.control
new file mode 100644
index 00000000000..a2023f9d738
--- /dev/null
+++ b/src/test/modules/deform_bench/deform_bench.control
@@ -0,0 +1,4 @@
+# deform_bench extension
+comment = 'functions for benchmarking tuple deformation'
+default_version = '1.0'
+module_pathname = '$libdir/deform_bench'
diff --git a/src/test/modules/deform_bench/meson.build b/src/test/modules/deform_bench/meson.build
new file mode 100644
index 00000000000..82049585244
--- /dev/null
+++ b/src/test/modules/deform_bench/meson.build
@@ -0,0 +1,22 @@
+# Copyright (c) 2026, PostgreSQL Global Development Group
+
+deform_bench_sources = files(
+ 'deform_bench.c',
+)
+
+if host_system == 'windows'
+ deform_bench_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+ '--NAME', 'deform_bench',
+ '--FILEDESC', 'deform_bench - benchmarking tuple deformation',])
+endif
+
+deform_bench = shared_module('deform_bench',
+ deform_bench_sources,
+ kwargs: pg_test_mod_args,
+)
+test_install_libs += deform_bench
+
+test_install_data += files(
+ 'deform_bench--1.0.sql',
+ 'deform_bench.control',
+)
diff --git a/src/test/modules/meson.build b/src/test/modules/meson.build
index e2b3eef4136..1312f3b4d7b 100644
--- a/src/test/modules/meson.build
+++ b/src/test/modules/meson.build
@@ -2,6 +2,7 @@
subdir('brin')
subdir('commit_ts')
+subdir('deform_bench')
subdir('delay_execution')
subdir('dummy_index_am')
subdir('dummy_seclabel')
--
2.51.0
[text/plain] v12-0002-Allow-sibling-call-optimization-in-slot_getsomea.patch (7.3K, 3-v12-0002-Allow-sibling-call-optimization-in-slot_getsomea.patch)
download | inline diff:
From ceaf2c78facbc2e118152d419a2c7bcf72a63ad5 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Mon, 16 Feb 2026 14:20:19 +1300
Subject: [PATCH v12 2/6] Allow sibling call optimization in
slot_getsomeattrs_int()
This changes the TupleTableSlotOps contract to make it so the
getsomeattrs() function is in charge of calling
slot_getmissingattrs().
Since this removes all code from slot_getsomeattrs_int() aside from the
getsomeattrs() call itself, we may as well adjust slot_getsomeattrs() so
that it calls getsomeattrs() directly. We leave slot_getsomeattrs_int()
intact as this is still called from the JIT code.
---
src/backend/executor/execTuples.c | 58 ++++++++++++++++---------------
src/include/executor/tuptable.h | 13 ++++---
2 files changed, 38 insertions(+), 33 deletions(-)
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index b768eae9e53..7effe954286 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -73,7 +73,7 @@
static TupleDesc ExecTypeFromTLInternal(List *targetList,
bool skipjunk);
static pg_attribute_always_inline void slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
- int natts);
+ int reqnatts);
static inline void tts_buffer_heap_store_tuple(TupleTableSlot *slot,
HeapTuple tuple,
Buffer buffer,
@@ -1108,7 +1108,10 @@ slot_deform_heap_tuple_internal(TupleTableSlot *slot, HeapTuple tuple,
* slot_deform_heap_tuple
* Given a TupleTableSlot, extract data from the slot's physical tuple
* into its Datum/isnull arrays. Data is extracted up through the
- * natts'th column (caller must ensure this is a legal column number).
+ * reqnatts'th column. If there are insufficient attributes in the given
+ * tuple, then slot_getmissingattrs() is called to populate the
+ * remainder. If reqnatts is above the number of attributes in the
+ * slot's TupleDesc, an error is raised.
*
* This is essentially an incremental version of heap_deform_tuple:
* on each call we extract attributes up to the one needed, without
@@ -1120,21 +1123,23 @@ slot_deform_heap_tuple_internal(TupleTableSlot *slot, HeapTuple tuple,
*/
static pg_attribute_always_inline void
slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
- int natts)
+ int reqnatts)
{
bool hasnulls = HeapTupleHasNulls(tuple);
int attnum;
+ int natts;
uint32 off; /* offset in tuple data */
bool slow; /* can we use/set attcacheoff? */
/* We can only fetch as many attributes as the tuple has. */
- natts = Min(HeapTupleHeaderGetNatts(tuple->t_data), natts);
+ natts = Min(HeapTupleHeaderGetNatts(tuple->t_data), reqnatts);
/*
* Check whether the first call for this tuple, and initialize or restore
* loop state.
*/
attnum = slot->tts_nvalid;
+ slot->tts_nvalid = reqnatts;
if (attnum == 0)
{
/* Start from the first attribute */
@@ -1199,12 +1204,15 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
/*
* Save state for next execution
*/
- slot->tts_nvalid = attnum;
*offp = off;
if (slow)
slot->tts_flags |= TTS_FLAG_SLOW;
else
slot->tts_flags &= ~TTS_FLAG_SLOW;
+
+ /* Fetch any missing attrs and raise an error if reqnatts is invalid. */
+ if (unlikely(attnum < reqnatts))
+ slot_getmissingattrs(slot, attnum, reqnatts);
}
const TupleTableSlotOps TTSOpsVirtual = {
@@ -2058,34 +2066,36 @@ slot_getmissingattrs(TupleTableSlot *slot, int startAttNum, int lastAttNum)
{
AttrMissing *attrmiss = NULL;
+ /* Check for invalid attnums */
+ if (unlikely(lastAttNum > slot->tts_tupleDescriptor->natts))
+ elog(ERROR, "invalid attribute number %d", lastAttNum);
+
if (slot->tts_tupleDescriptor->constr)
attrmiss = slot->tts_tupleDescriptor->constr->missing;
if (!attrmiss)
{
/* no missing values array at all, so just fill everything in as NULL */
- memset(slot->tts_values + startAttNum, 0,
- (lastAttNum - startAttNum) * sizeof(Datum));
- memset(slot->tts_isnull + startAttNum, 1,
- (lastAttNum - startAttNum) * sizeof(bool));
+ for (int attnum = startAttNum; attnum < lastAttNum; attnum++)
+ {
+ slot->tts_values[attnum] = (Datum) 0;
+ slot->tts_isnull[attnum] = true;
+ }
}
else
{
- int missattnum;
-
- /* if there is a missing values array we must process them one by one */
- for (missattnum = startAttNum;
- missattnum < lastAttNum;
- missattnum++)
+ /* use attrmiss to set the missing values */
+ for (int attnum = startAttNum; attnum < lastAttNum; attnum++)
{
- slot->tts_values[missattnum] = attrmiss[missattnum].am_value;
- slot->tts_isnull[missattnum] = !attrmiss[missattnum].am_present;
+ slot->tts_values[attnum] = attrmiss[attnum].am_value;
+ slot->tts_isnull[attnum] = !attrmiss[attnum].am_present;
}
}
}
/*
- * slot_getsomeattrs_int - workhorse for slot_getsomeattrs()
+ * slot_getsomeattrs_int
+ * external function to call getsomeattrs() for use in JIT
*/
void
slot_getsomeattrs_int(TupleTableSlot *slot, int attnum)
@@ -2094,21 +2104,13 @@ slot_getsomeattrs_int(TupleTableSlot *slot, int attnum)
Assert(slot->tts_nvalid < attnum); /* checked in slot_getsomeattrs */
Assert(attnum > 0);
- if (unlikely(attnum > slot->tts_tupleDescriptor->natts))
- elog(ERROR, "invalid attribute number %d", attnum);
-
/* Fetch as many attributes as possible from the underlying tuple. */
slot->tts_ops->getsomeattrs(slot, attnum);
/*
- * If the underlying tuple doesn't have enough attributes, tuple
- * descriptor must have the missing attributes.
+ * Avoid putting new code here as that would prevent the compiler from
+ * using the sibling call optimization for the above function.
*/
- if (unlikely(slot->tts_nvalid < attnum))
- {
- slot_getmissingattrs(slot, slot->tts_nvalid, attnum);
- slot->tts_nvalid = attnum;
- }
}
/* ----------------------------------------------------------------
diff --git a/src/include/executor/tuptable.h b/src/include/executor/tuptable.h
index a2dfd707e78..3b09abbf99f 100644
--- a/src/include/executor/tuptable.h
+++ b/src/include/executor/tuptable.h
@@ -151,10 +151,12 @@ struct TupleTableSlotOps
/*
* Fill up first natts entries of tts_values and tts_isnull arrays with
- * values from the tuple contained in the slot. The function may be called
- * with natts more than the number of attributes available in the tuple,
- * in which case it should set tts_nvalid to the number of returned
- * columns.
+ * values from the tuple contained in the slot and set the slot's
+ * tts_nvalid to natts. The function may be called with an natts value
+ * more than the number of attributes available in the tuple, in which
+ * case the function must call slot_getmissingattrs() to populate the
+ * remaining attributes. The function must raise an ERROR if 'natts' is
+ * higher than the number of attributes in the slot's TupleDesc.
*/
void (*getsomeattrs) (TupleTableSlot *slot, int natts);
@@ -357,8 +359,9 @@ extern void slot_getsomeattrs_int(TupleTableSlot *slot, int attnum);
static inline void
slot_getsomeattrs(TupleTableSlot *slot, int attnum)
{
+ /* Populate slot with attributes up to 'attnum', if it's not already */
if (slot->tts_nvalid < attnum)
- slot_getsomeattrs_int(slot, attnum);
+ slot->tts_ops->getsomeattrs(slot, attnum);
}
/*
--
2.51.0
[text/plain] v12-0003-Add-empty-TupleDescFinalize-function.patch (29.0K, 4-v12-0003-Add-empty-TupleDescFinalize-function.patch)
download | inline diff:
From 00bc3ecefc20daef125c766fa9ca1d5e5af4d250 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Wed, 21 Jan 2026 15:41:37 +1300
Subject: [PATCH v12 3/6] Add empty TupleDescFinalize() function
Currently does nothing, but will in a future commit.
---
contrib/dblink/dblink.c | 4 ++++
contrib/pg_buffercache/pg_buffercache_pages.c | 2 ++
contrib/pg_visibility/pg_visibility.c | 2 ++
src/backend/access/brin/brin_tuple.c | 1 +
src/backend/access/common/tupdesc.c | 13 +++++++++++++
src/backend/access/gin/ginutil.c | 1 +
src/backend/access/gist/gistscan.c | 1 +
src/backend/access/spgist/spgutils.c | 1 +
src/backend/access/transam/twophase.c | 1 +
src/backend/access/transam/xlogfuncs.c | 1 +
src/backend/backup/basebackup_copy.c | 3 +++
src/backend/catalog/index.c | 2 ++
src/backend/catalog/pg_publication.c | 1 +
src/backend/catalog/toasting.c | 6 ++++++
src/backend/commands/explain.c | 1 +
src/backend/commands/functioncmds.c | 1 +
src/backend/commands/sequence.c | 1 +
src/backend/commands/tablecmds.c | 4 ++++
src/backend/commands/wait.c | 1 +
src/backend/executor/execSRF.c | 2 ++
src/backend/executor/execTuples.c | 4 ++++
src/backend/executor/nodeFunctionscan.c | 2 ++
src/backend/parser/parse_relation.c | 4 +++-
src/backend/parser/parse_target.c | 2 ++
.../replication/libpqwalreceiver/libpqwalreceiver.c | 1 +
src/backend/replication/walsender.c | 5 +++++
src/backend/utils/adt/acl.c | 1 +
src/backend/utils/adt/genfile.c | 1 +
src/backend/utils/adt/lockfuncs.c | 1 +
src/backend/utils/adt/orderedsetaggs.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 5 +++++
src/backend/utils/adt/tsvector_op.c | 1 +
src/backend/utils/cache/relcache.c | 8 ++++++++
src/backend/utils/fmgr/funcapi.c | 6 ++++++
src/backend/utils/misc/guc_funcs.c | 5 +++++
src/include/access/tupdesc.h | 1 +
src/pl/plpgsql/src/pl_comp.c | 2 ++
.../test_custom_stats/test_custom_fixed_stats.c | 1 +
src/test/modules/test_predtest/test_predtest.c | 1 +
39 files changed, 100 insertions(+), 1 deletion(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 2498d80c8e7..4038950a6ef 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -881,6 +881,7 @@ materializeResult(FunctionCallInfo fcinfo, PGconn *conn, PGresult *res)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "status",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
ntuples = 1;
nfields = 1;
}
@@ -1044,6 +1045,7 @@ materializeQueryResult(FunctionCallInfo fcinfo,
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "status",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
attinmeta = TupleDescGetAttInMetadata(tupdesc);
oldcontext = MemoryContextSwitchTo(rsinfo->econtext->ecxt_per_query_memory);
@@ -1529,6 +1531,8 @@ dblink_get_pkey(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 2, "colname",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+
/*
* Generate attribute metadata needed later to produce tuples from raw
* C strings
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 89b86855243..a6b4fb5252b 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -174,6 +174,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
INT4OID, -1, 0);
+ TupleDescFinalize(tupledesc);
fctx->tupdesc = BlessTupleDesc(tupledesc);
/* Allocate NBuffers worth of BufferCachePagesRec records. */
@@ -442,6 +443,7 @@ pg_buffercache_os_pages_internal(FunctionCallInfo fcinfo, bool include_numa)
TupleDescInitEntry(tupledesc, (AttrNumber) 3, "numa_node",
INT4OID, -1, 0);
+ TupleDescFinalize(tupledesc);
fctx->tupdesc = BlessTupleDesc(tupledesc);
fctx->include_numa = include_numa;
diff --git a/contrib/pg_visibility/pg_visibility.c b/contrib/pg_visibility/pg_visibility.c
index 9bc3a784bf7..dfab0b64cf5 100644
--- a/contrib/pg_visibility/pg_visibility.c
+++ b/contrib/pg_visibility/pg_visibility.c
@@ -469,6 +469,8 @@ pg_visibility_tupdesc(bool include_blkno, bool include_pd)
TupleDescInitEntry(tupdesc, ++a, "pd_all_visible", BOOLOID, -1, 0);
Assert(a == maxattr);
+ TupleDescFinalize(tupdesc);
+
return BlessTupleDesc(tupdesc);
}
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 69c233c62eb..742ac089a28 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -84,6 +84,7 @@ brtuple_disk_tupdesc(BrinDesc *brdesc)
MemoryContextSwitchTo(oldcxt);
+ TupleDescFinalize(tupdesc);
brdesc->bd_disktdesc = tupdesc;
}
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index b69d10f0a45..2137385a833 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -221,6 +221,9 @@ CreateTupleDesc(int natts, Form_pg_attribute *attrs)
memcpy(TupleDescAttr(desc, i), attrs[i], ATTRIBUTE_FIXED_PART_SIZE);
populate_compact_attribute(desc, i);
}
+
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -265,6 +268,8 @@ CreateTupleDescCopy(TupleDesc tupdesc)
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -311,6 +316,8 @@ CreateTupleDescTruncatedCopy(TupleDesc tupdesc, int natts)
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -396,6 +403,8 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -438,6 +447,8 @@ TupleDescCopy(TupleDesc dst, TupleDesc src)
* source's refcount would be wrong in any case.)
*/
dst->tdrefcount = -1;
+
+ TupleDescFinalize(dst);
}
/*
@@ -1065,6 +1076,8 @@ BuildDescFromLists(const List *names, const List *types, const List *typmods, co
TupleDescInitEntryCollation(desc, attnum, attcollation);
}
+ TupleDescFinalize(desc);
+
return desc;
}
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index ff927279cc3..fe7b984ff32 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -129,6 +129,7 @@ initGinState(GinState *state, Relation index)
attr->attndims);
TupleDescInitEntryCollation(state->tupdesc[i], (AttrNumber) 2,
attr->attcollation);
+ TupleDescFinalize(state->tupdesc[i]);
}
/*
diff --git a/src/backend/access/gist/gistscan.c b/src/backend/access/gist/gistscan.c
index f23bc4a6757..c65f93abdae 100644
--- a/src/backend/access/gist/gistscan.c
+++ b/src/backend/access/gist/gistscan.c
@@ -201,6 +201,7 @@ gistrescan(IndexScanDesc scan, ScanKey key, int nkeys,
attno - 1)->atttypid,
-1, 0);
}
+ TupleDescFinalize(so->giststate->fetchTupdesc);
scan->xs_hitupdesc = so->giststate->fetchTupdesc;
/* Also create a memory context that will hold the returned tuples */
diff --git a/src/backend/access/spgist/spgutils.c b/src/backend/access/spgist/spgutils.c
index 9f5379b87ac..b246e8127db 100644
--- a/src/backend/access/spgist/spgutils.c
+++ b/src/backend/access/spgist/spgutils.c
@@ -340,6 +340,7 @@ getSpGistTupleDesc(Relation index, SpGistTypeDesc *keyType)
TupleDescCompactAttr(outTupDesc, i)->attcacheoff = -1;
populate_compact_attribute(outTupDesc, spgKeyColumn);
+ TupleDescFinalize(outTupDesc);
}
return outTupDesc;
}
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 5f65fa7b80f..0281b58f40c 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -744,6 +744,7 @@ pg_prepared_xact(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 5, "dbid",
OIDOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
/*
diff --git a/src/backend/access/transam/xlogfuncs.c b/src/backend/access/transam/xlogfuncs.c
index 78543055895..be4cc24f1db 100644
--- a/src/backend/access/transam/xlogfuncs.c
+++ b/src/backend/access/transam/xlogfuncs.c
@@ -427,6 +427,7 @@ pg_walfile_name_offset(PG_FUNCTION_ARGS)
TupleDescInitEntry(resultTupleDesc, (AttrNumber) 2, "file_offset",
INT4OID, -1, 0);
+ TupleDescFinalize(resultTupleDesc);
resultTupleDesc = BlessTupleDesc(resultTupleDesc);
/*
diff --git a/src/backend/backup/basebackup_copy.c b/src/backend/backup/basebackup_copy.c
index 07f58b39d8c..6c3453efd80 100644
--- a/src/backend/backup/basebackup_copy.c
+++ b/src/backend/backup/basebackup_copy.c
@@ -357,6 +357,8 @@ SendXlogRecPtrResult(XLogRecPtr ptr, TimeLineID tli)
*/
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "tli", INT8OID, -1, 0);
+ TupleDescFinalize(tupdesc);
+
/* send RowDescription */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -388,6 +390,7 @@ SendTablespaceList(List *tablespaces)
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 1, "spcoid", OIDOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "spclocation", TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 3, "size", INT8OID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* send RowDescription */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 43de42ce39e..75e97fb394a 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -481,6 +481,8 @@ ConstructTupleDescriptor(Relation heapRelation,
populate_compact_attribute(indexTupDesc, i);
}
+ TupleDescFinalize(indexTupDesc);
+
return indexTupDesc;
}
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index aadc7c202c6..a79157c43bf 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -1322,6 +1322,7 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 4, "qual",
PG_NODE_TREEOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
funcctx->user_fctx = table_infos;
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index c78dcea98c1..078a1cf5127 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -229,6 +229,12 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attcompression = InvalidCompressionMethod;
TupleDescAttr(tupdesc, 2)->attcompression = InvalidCompressionMethod;
+ populate_compact_attribute(tupdesc, 0);
+ populate_compact_attribute(tupdesc, 1);
+ populate_compact_attribute(tupdesc, 2);
+
+ TupleDescFinalize(tupdesc);
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 93918a223b8..5f922c3f5c2 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -281,6 +281,7 @@ ExplainResultDesc(ExplainStmt *stmt)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "QUERY PLAN",
result_type, -1, 0);
+ TupleDescFinalize(tupdesc);
return tupdesc;
}
diff --git a/src/backend/commands/functioncmds.c b/src/backend/commands/functioncmds.c
index 242372b1e68..3afd762e9dc 100644
--- a/src/backend/commands/functioncmds.c
+++ b/src/backend/commands/functioncmds.c
@@ -2424,6 +2424,7 @@ CallStmtResultDesc(CallStmt *stmt)
-1,
0);
}
+ TupleDescFinalize(tupdesc);
}
return tupdesc;
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index e1b808bbb60..551667650ba 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -1808,6 +1808,7 @@ pg_get_sequence_data(PG_FUNCTION_ARGS)
BOOLOID, -1, 0);
TupleDescInitEntry(resultTupleDesc, (AttrNumber) 3, "page_lsn",
LSNOID, -1, 0);
+ TupleDescFinalize(resultTupleDesc);
resultTupleDesc = BlessTupleDesc(resultTupleDesc);
seqrel = try_relation_open(relid, AccessShareLock);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 85242dcc245..80922974970 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -1030,6 +1030,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
}
}
+ TupleDescFinalize(descriptor);
+
/*
* For relations with table AM and partitioned tables, select access
* method to use: an explicitly indicated one, or (in the case of a
@@ -1458,6 +1460,8 @@ BuildDescForRelation(const List *columns)
populate_compact_attribute(desc, attnum - 1);
}
+ TupleDescFinalize(desc);
+
return desc;
}
diff --git a/src/backend/commands/wait.c b/src/backend/commands/wait.c
index 1290df10c6f..8e920a72372 100644
--- a/src/backend/commands/wait.c
+++ b/src/backend/commands/wait.c
@@ -338,5 +338,6 @@ WaitStmtResultDesc(WaitStmt *stmt)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "status",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
return tupdesc;
}
diff --git a/src/backend/executor/execSRF.c b/src/backend/executor/execSRF.c
index a0b111dc0e4..b481e50acfb 100644
--- a/src/backend/executor/execSRF.c
+++ b/src/backend/executor/execSRF.c
@@ -272,6 +272,7 @@ ExecMakeTableFunctionResult(SetExprState *setexpr,
funcrettype,
-1,
0);
+ TupleDescFinalize(tupdesc);
rsinfo.setDesc = tupdesc;
}
MemoryContextSwitchTo(oldcontext);
@@ -776,6 +777,7 @@ init_sexpr(Oid foid, Oid input_collation, Expr *node,
funcrettype,
-1,
0);
+ TupleDescFinalize(tupdesc);
sexpr->funcResultDesc = tupdesc;
sexpr->funcReturnsTuple = false;
}
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 7effe954286..07b248aa5f3 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -2175,6 +2175,8 @@ ExecTypeFromTLInternal(List *targetList, bool skipjunk)
cur_resno++;
}
+ TupleDescFinalize(typeInfo);
+
return typeInfo;
}
@@ -2209,6 +2211,8 @@ ExecTypeFromExprList(List *exprList)
cur_resno++;
}
+ TupleDescFinalize(typeInfo);
+
return typeInfo;
}
diff --git a/src/backend/executor/nodeFunctionscan.c b/src/backend/executor/nodeFunctionscan.c
index 63e605e1f81..feb82d64967 100644
--- a/src/backend/executor/nodeFunctionscan.c
+++ b/src/backend/executor/nodeFunctionscan.c
@@ -414,6 +414,7 @@ ExecInitFunctionScan(FunctionScan *node, EState *estate, int eflags)
TupleDescInitEntryCollation(tupdesc,
(AttrNumber) 1,
exprCollation(funcexpr));
+ TupleDescFinalize(tupdesc);
}
else
{
@@ -485,6 +486,7 @@ ExecInitFunctionScan(FunctionScan *node, EState *estate, int eflags)
0);
}
+ TupleDescFinalize(scan_tupdesc);
Assert(attno == natts);
}
diff --git a/src/backend/parser/parse_relation.c b/src/backend/parser/parse_relation.c
index e003db520de..9c415e166ee 100644
--- a/src/backend/parser/parse_relation.c
+++ b/src/backend/parser/parse_relation.c
@@ -1883,6 +1883,7 @@ addRangeTableEntryForFunction(ParseState *pstate,
TupleDescInitEntryCollation(tupdesc,
(AttrNumber) 1,
exprCollation(funcexpr));
+ TupleDescFinalize(tupdesc);
}
else if (functypclass == TYPEFUNC_RECORD)
{
@@ -1940,6 +1941,7 @@ addRangeTableEntryForFunction(ParseState *pstate,
i++;
}
+ TupleDescFinalize(tupdesc);
/*
* Ensure that the coldeflist defines a legal set of names (no
@@ -2008,7 +2010,7 @@ addRangeTableEntryForFunction(ParseState *pstate,
0);
/* no need to set collation */
}
-
+ TupleDescFinalize(tupdesc);
Assert(natts == totalatts);
}
else
diff --git a/src/backend/parser/parse_target.c b/src/backend/parser/parse_target.c
index 3bcfc1f5e3d..f57c4d41080 100644
--- a/src/backend/parser/parse_target.c
+++ b/src/backend/parser/parse_target.c
@@ -1572,6 +1572,8 @@ expandRecordVariable(ParseState *pstate, Var *var, int levelsup)
}
Assert(lname == NULL && lvar == NULL); /* lists same length? */
+ TupleDescFinalize(tupleDesc);
+
return tupleDesc;
}
diff --git a/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c b/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
index 7c8639b32e9..9f04c9ed25d 100644
--- a/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
+++ b/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
@@ -1073,6 +1073,7 @@ libpqrcv_processTuples(PGresult *pgres, WalRcvExecResult *walres,
for (coln = 0; coln < nRetTypes; coln++)
TupleDescInitEntry(walres->tupledesc, (AttrNumber) coln + 1,
PQfname(pgres, coln), retTypes[coln], -1, 0);
+ TupleDescFinalize(walres->tupledesc);
attinmeta = TupleDescGetAttInMetadata(walres->tupledesc);
/* No point in doing more here if there were no tuples returned. */
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c
index 2cde8ebc729..33a9e8d7f21 100644
--- a/src/backend/replication/walsender.c
+++ b/src/backend/replication/walsender.c
@@ -451,6 +451,7 @@ IdentifySystem(void)
TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 4, "dbname",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -496,6 +497,7 @@ ReadReplicationSlot(ReadReplicationSlotCmd *cmd)
/* TimeLineID is unsigned, so int4 is not wide enough. */
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 3, "restart_tli",
INT8OID, -1, 0);
+ TupleDescFinalize(tupdesc);
memset(nulls, true, READ_REPLICATION_SLOT_COLS * sizeof(bool));
@@ -598,6 +600,7 @@ SendTimeLineHistory(TimeLineHistoryCmd *cmd)
tupdesc = CreateTemplateTupleDesc(2);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 1, "filename", TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "content", TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
TLHistoryFileName(histfname, cmd->timeline);
TLHistoryFilePath(path, cmd->timeline);
@@ -1015,6 +1018,7 @@ StartReplication(StartReplicationCmd *cmd)
INT8OID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "next_tli_startpos",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuple */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -1369,6 +1373,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd)
TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 4, "output_plugin",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
diff --git a/src/backend/utils/adt/acl.c b/src/backend/utils/adt/acl.c
index 071e3f2c49e..e210d6472be 100644
--- a/src/backend/utils/adt/acl.c
+++ b/src/backend/utils/adt/acl.c
@@ -1831,6 +1831,7 @@ aclexplode(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 4, "is_grantable",
BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
/* allocate memory for user context */
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index c083608b1d5..bfb949401d0 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -454,6 +454,7 @@ pg_stat_file(PG_FUNCTION_ARGS)
"creation", TIMESTAMPTZOID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 6,
"isdir", BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
memset(isnull, false, sizeof(isnull));
diff --git a/src/backend/utils/adt/lockfuncs.c b/src/backend/utils/adt/lockfuncs.c
index 9dadd6da672..4481c354fd6 100644
--- a/src/backend/utils/adt/lockfuncs.c
+++ b/src/backend/utils/adt/lockfuncs.c
@@ -146,6 +146,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 16, "waitstart",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
/*
diff --git a/src/backend/utils/adt/orderedsetaggs.c b/src/backend/utils/adt/orderedsetaggs.c
index 3b6da8e36ac..fd8b8676470 100644
--- a/src/backend/utils/adt/orderedsetaggs.c
+++ b/src/backend/utils/adt/orderedsetaggs.c
@@ -233,6 +233,7 @@ ordered_set_startup(FunctionCallInfo fcinfo, bool use_tuples)
-1,
0);
+ TupleDescFinalize(newdesc);
FreeTupleDesc(qstate->tupdesc);
qstate->tupdesc = newdesc;
}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index b1df96e7b0b..0b10da3b180 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -769,6 +769,7 @@ pg_stat_get_backend_subxact(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 2, "subxact_overflow",
BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
if ((local_beentry = pgstat_get_local_beentry_by_proc_number(procNumber)) != NULL)
@@ -1670,6 +1671,7 @@ pg_stat_wal_build_tuple(PgStat_WalCounters wal_counters,
TupleDescInitEntry(tupdesc, (AttrNumber) 6, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
/* Fill values and NULLs */
@@ -2097,6 +2099,7 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 7, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
/* Get statistics about the archiver process */
@@ -2178,6 +2181,7 @@ pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
TIMESTAMPTZOID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 13, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
namestrcpy(&slotname, text_to_cstring(slotname_text));
@@ -2265,6 +2269,7 @@ pg_stat_get_subscription_stats(PG_FUNCTION_ARGS)
INT8OID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 13, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
if (!subentry)
diff --git a/src/backend/utils/adt/tsvector_op.c b/src/backend/utils/adt/tsvector_op.c
index 71c7c7d3b3c..d8dece42b9b 100644
--- a/src/backend/utils/adt/tsvector_op.c
+++ b/src/backend/utils/adt/tsvector_op.c
@@ -651,6 +651,7 @@ tsvector_unnest(PG_FUNCTION_ARGS)
TEXTARRAYOID, -1, 0);
if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
elog(ERROR, "return type must be a row type");
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = tupdesc;
funcctx->user_fctx = PG_GETARG_TSVECTOR_COPY(0);
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a1c88c6b1b6..d27ac216e6d 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -729,6 +729,8 @@ RelationBuildTupleDesc(Relation relation)
pfree(constr);
relation->rd_att->constr = NULL;
}
+
+ TupleDescFinalize(relation->rd_att);
}
/*
@@ -1985,6 +1987,7 @@ formrdesc(const char *relationName, Oid relationReltype,
/* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
TupleDescCompactAttr(relation->rd_att, 0)->attcacheoff = 0;
+ TupleDescFinalize(relation->rd_att);
/* mark not-null status */
if (has_not_null)
@@ -3688,6 +3691,8 @@ RelationBuildLocalRelation(const char *relname,
for (i = 0; i < natts; i++)
TupleDescAttr(rel->rd_att, i)->attrelid = relid;
+ TupleDescFinalize(rel->rd_att);
+
rel->rd_rel->reltablespace = reltablespace;
if (mapped_relation)
@@ -4443,6 +4448,7 @@ BuildHardcodedDescriptor(int natts, const FormData_pg_attribute *attrs)
/* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
TupleDescCompactAttr(result, 0)->attcacheoff = 0;
+ TupleDescFinalize(result);
/* Note: we don't bother to set up a TupleConstr entry */
@@ -6291,6 +6297,8 @@ load_relcache_init_file(bool shared)
populate_compact_attribute(rel->rd_att, i);
}
+ TupleDescFinalize(rel->rd_att);
+
/* next read the access method specific field */
if (fread(&len, 1, sizeof(len), fp) != sizeof(len))
goto read_failed;
diff --git a/src/backend/utils/fmgr/funcapi.c b/src/backend/utils/fmgr/funcapi.c
index 8a934ea8dca..516d02cfb82 100644
--- a/src/backend/utils/fmgr/funcapi.c
+++ b/src/backend/utils/fmgr/funcapi.c
@@ -340,6 +340,8 @@ get_expr_result_type(Node *expr,
exprCollation(col));
i++;
}
+ TupleDescFinalize(tupdesc);
+
if (resultTypeId)
*resultTypeId = rexpr->row_typeid;
if (resultTupleDesc)
@@ -1044,6 +1046,7 @@ resolve_polymorphic_tupdesc(TupleDesc tupdesc, oidvector *declared_args,
}
}
+ TupleDescFinalize(tupdesc);
return true;
}
@@ -1853,6 +1856,8 @@ build_function_result_tupdesc_d(char prokind,
0);
}
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -1970,6 +1975,7 @@ TypeGetTupleDesc(Oid typeoid, List *colaliases)
typeoid,
-1,
0);
+ TupleDescFinalize(tupdesc);
}
else if (functypclass == TYPEFUNC_RECORD)
{
diff --git a/src/backend/utils/misc/guc_funcs.c b/src/backend/utils/misc/guc_funcs.c
index 8524dd3a981..472cb5393ce 100644
--- a/src/backend/utils/misc/guc_funcs.c
+++ b/src/backend/utils/misc/guc_funcs.c
@@ -444,6 +444,7 @@ GetPGVariableResultDesc(const char *name)
TupleDescInitEntry(tupdesc, (AttrNumber) 1, varname,
TEXTOID, -1, 0);
}
+ TupleDescFinalize(tupdesc);
return tupdesc;
}
@@ -465,6 +466,7 @@ ShowGUCConfigOption(const char *name, DestReceiver *dest)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 1, varname,
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -499,6 +501,7 @@ ShowAllGUCConfig(DestReceiver *dest)
TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 3, "description",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -934,6 +937,8 @@ show_all_settings(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 17, "pending_restart",
BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+
/*
* Generate attribute metadata needed later to produce tuples from raw
* C strings
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index d46cdbf7a3c..595413dbbc5 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -195,6 +195,7 @@ extern TupleDesc CreateTupleDescTruncatedCopy(TupleDesc tupdesc, int natts);
extern TupleDesc CreateTupleDescCopyConstr(TupleDesc tupdesc);
+#define TupleDescFinalize(d) ((void) 0)
#define TupleDescSize(src) \
(offsetof(struct TupleDescData, compact_attrs) + \
(src)->natts * sizeof(CompactAttribute) + \
diff --git a/src/pl/plpgsql/src/pl_comp.c b/src/pl/plpgsql/src/pl_comp.c
index 5ecc7766757..b72c963b3be 100644
--- a/src/pl/plpgsql/src/pl_comp.c
+++ b/src/pl/plpgsql/src/pl_comp.c
@@ -1912,6 +1912,8 @@ build_row_from_vars(PLpgSQL_variable **vars, int numvars)
TupleDescInitEntryCollation(row->rowtupdesc, i + 1, typcoll);
}
+ TupleDescFinalize(row->rowtupdesc);
+
return row;
}
diff --git a/src/test/modules/test_custom_stats/test_custom_fixed_stats.c b/src/test/modules/test_custom_stats/test_custom_fixed_stats.c
index 485e08e5c19..f9e7c717280 100644
--- a/src/test/modules/test_custom_stats/test_custom_fixed_stats.c
+++ b/src/test/modules/test_custom_stats/test_custom_fixed_stats.c
@@ -206,6 +206,7 @@ test_custom_stats_fixed_report(PG_FUNCTION_ARGS)
INT8OID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 2, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
values[0] = Int64GetDatum(stats->numcalls);
diff --git a/src/test/modules/test_predtest/test_predtest.c b/src/test/modules/test_predtest/test_predtest.c
index 679a5de456d..48ca2a4ea70 100644
--- a/src/test/modules/test_predtest/test_predtest.c
+++ b/src/test/modules/test_predtest/test_predtest.c
@@ -230,6 +230,7 @@ test_predtest(PG_FUNCTION_ARGS)
"s_r_holds", BOOLOID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 8,
"w_r_holds", BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
tupdesc = BlessTupleDesc(tupdesc);
values[0] = BoolGetDatum(strong_implied_by);
--
2.51.0
[text/plain] v12-0004-Optimize-tuple-deformation.patch (81.3K, 5-v12-0004-Optimize-tuple-deformation.patch)
download | inline diff:
From d46c106325c6dd7f456fe5d28d47a83cc295a9c7 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Tue, 31 Dec 2024 09:19:24 +1300
Subject: [PATCH v12 4/6] Optimize tuple deformation
This commit includes various optimizations to improve the performance of
tuple deformation.
We now precalculate CompactAttribute's attcacheoff, which allows us to
remove the code from the deform routines which was setting the
attcacheoff. Setting the attcacheoff is handled by TupleDescFinalize(),
which must be called before the TupleDesc is used for anything. Having
this TupleDescFinalize() function means we can store the first
attribute in the TupleDesc which does not have an offset cached. That
allows us to add a dedicated deforming loop to deform all attributes up
to the final one with an attcacheoff set, or up to the first NULL
attribute, whichever comes first.
We also record the maximum attribute number which is guaranteed to exist
in the tuple, that is, has a NOT NULL constraint and isn't an
atthasmissing attribute. When deforming only attributes prior to the
guaranteed attnum, we've no need to access the tuple's natt count. As an
additional optimization, we only count fixed-width columns when
calculating the maximum guaranteed column as this eliminates the need to
emit code to fetch byref types in the deformation loop for guaranteed
attributes.
Some locations in the code deform tuples that have yet to go through NOT
NULL constraint validation. We're unable to perform the guaranteed
attribute optimization when that's the case. The optimization is opt-in
via the TupleTableSlot using the TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS
flag.
This commit also adds a more efficient way of populating the isnull
array by using a bit-wise trick which performs multiplication on the
inverse of the tuple's bitmap byte and masking out all but the lower bit
of each of the boolean's byte. This results in much more optimal code
when compared to determining the NULLness via att_isnull(). 8 isnull
elements are processed at once using this method, which means we need to
round the tts_isnull array size up to the next 8 bytes. The palloc code
does this anyway, but the round-up needed to be formalized so as not to
overwrite the sentinel byte in debug builds.
---
src/backend/access/common/heaptuple.c | 360 +++++++--------
src/backend/access/common/indextuple.c | 363 ++++++---------
src/backend/access/common/tupdesc.c | 51 +++
src/backend/access/spgist/spgutils.c | 3 -
src/backend/executor/execMain.c | 8 +-
src/backend/executor/execTuples.c | 417 ++++++++++--------
src/backend/executor/execUtils.c | 2 +-
src/backend/executor/nodeAgg.c | 2 +-
src/backend/executor/nodeBitmapHeapscan.c | 5 +-
src/backend/executor/nodeCtescan.c | 2 +-
src/backend/executor/nodeCustom.c | 4 +-
src/backend/executor/nodeForeignscan.c | 4 +-
src/backend/executor/nodeFunctionscan.c | 2 +-
src/backend/executor/nodeIndexonlyscan.c | 7 +-
src/backend/executor/nodeIndexscan.c | 5 +-
.../executor/nodeNamedtuplestorescan.c | 2 +-
src/backend/executor/nodeSamplescan.c | 6 +-
src/backend/executor/nodeSeqscan.c | 3 +-
src/backend/executor/nodeSubqueryscan.c | 3 +-
src/backend/executor/nodeTableFuncscan.c | 2 +-
src/backend/executor/nodeTidrangescan.c | 5 +-
src/backend/executor/nodeTidscan.c | 5 +-
src/backend/executor/nodeValuesscan.c | 2 +-
src/backend/executor/nodeWorktablescan.c | 2 +-
src/backend/jit/llvm/llvmjit_deform.c | 6 -
src/backend/replication/pgoutput/pgoutput.c | 4 +-
src/backend/utils/cache/relcache.c | 12 -
src/include/access/tupdesc.h | 20 +-
src/include/access/tupmacs.h | 224 +++++++++-
src/include/executor/executor.h | 3 +-
src/include/executor/tuptable.h | 28 +-
src/test/modules/deform_bench/deform_bench.c | 4 +-
32 files changed, 902 insertions(+), 664 deletions(-)
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 11bec20e82e..b2ac7fef35b 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -498,19 +498,7 @@ heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc)
* nocachegetattr
*
* This only gets called from fastgetattr(), in cases where we
- * can't use a cacheoffset and the value is not null.
- *
- * This caches attribute offsets in the attribute descriptor.
- *
- * An alternative way to speed things up would be to cache offsets
- * with the tuple, but that seems more difficult unless you take
- * the storage hit of actually putting those offsets into the
- * tuple you send to disk. Yuck.
- *
- * This scheme will be slightly slower than that, but should
- * perform well for queries which hit large #'s of tuples. After
- * you cache the offsets once, examining all the other tuples using
- * the same attribute descriptor will go much quicker. -cim 5/4/91
+ * can't use the attcacheoff and the value is not null.
*
* NOTE: if you need to change this code, see also heap_deform_tuple.
* Also see nocache_index_getattr, which is the same code for index
@@ -522,194 +510,125 @@ nocachegetattr(HeapTuple tup,
int attnum,
TupleDesc tupleDesc)
{
+ CompactAttribute *cattr;
HeapTupleHeader td = tup->t_data;
char *tp; /* ptr to data part of tuple */
bits8 *bp = td->t_bits; /* ptr to null bitmap in tuple */
- bool slow = false; /* do we have to walk attrs? */
int off; /* current offset within data */
+ int startAttr;
+ int firstNullAttr;
+ int i;
+ bool hasnulls = HeapTupleHasNulls(tup);
- /* ----------------
- * Three cases:
- *
- * 1: No nulls and no variable-width attributes.
- * 2: Has a null or a var-width AFTER att.
- * 3: Has nulls or var-widths BEFORE att.
- * ----------------
- */
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
attnum--;
- if (!HeapTupleNoNulls(tup))
+ /*
+ * To minimize the number of attributes we need to look at, start walking
+ * the tuple at the attribute with the highest attcacheoff prior to attnum
+ * or the first NULL attribute prior to attnum, whichever comes first.
+ */
+ if (hasnulls)
+ firstNullAttr = first_null_attr(bp, attnum);
+ else
+ firstNullAttr = attnum;
+
+ if (tupleDesc->firstNonCachedOffsetAttr > 0)
{
/*
- * there's a null somewhere in the tuple
- *
- * check to see if any preceding bits are null...
+ * Start at the highest attcacheoff attribute with no NULLs in prior
+ * attributes.
*/
- int byte = attnum >> 3;
- int finalbit = attnum & 0x07;
-
- /* check for nulls "before" final bit of last byte */
- if ((~bp[byte]) & ((1 << finalbit) - 1))
- slow = true;
- else
- {
- /* check for nulls in any "earlier" bytes */
- int i;
-
- for (i = 0; i < byte; i++)
- {
- if (bp[i] != 0xFF)
- {
- slow = true;
- break;
- }
- }
- }
+ startAttr = Min(tupleDesc->firstNonCachedOffsetAttr - 1, firstNullAttr);
+ off = TupleDescCompactAttr(tupleDesc, startAttr)->attcacheoff;
+ }
+ else
+ {
+ /* Otherwise, start at the beginning... */
+ startAttr = 0;
+ off = 0;
}
tp = (char *) td + td->t_hoff;
- if (!slow)
+ /*
+ * Calculate 'off' up to the first NULL attr. We use two cheaper loops
+ * when the tuple has no variable-width columns. When variable-width
+ * columns exists, we use att_addlength_pointer() to move the offset
+ * beyond the current attribute.
+ */
+ if (!HeapTupleHasVarWidth(tup))
{
- CompactAttribute *att;
-
- /*
- * If we get here, there are no nulls up to and including the target
- * attribute. If we have a cached offset, we can use it.
- */
- att = TupleDescCompactAttr(tupleDesc, attnum);
- if (att->attcacheoff >= 0)
- return fetchatt(att, tp + att->attcacheoff);
-
- /*
- * Otherwise, check for non-fixed-length attrs up to and including
- * target. If there aren't any, it's safe to cheaply initialize the
- * cached offsets for these attrs.
- */
- if (HeapTupleHasVarWidth(tup))
+ for (i = startAttr; i < firstNullAttr; i++)
{
- int j;
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- for (j = 0; j <= attnum; j++)
- {
- if (TupleDescCompactAttr(tupleDesc, j)->attlen <= 0)
- {
- slow = true;
- break;
- }
- }
+ off = att_nominal_alignby(off, cattr->attalignby);
+ off += cattr->attlen;
}
- }
-
- if (!slow)
- {
- int natts = tupleDesc->natts;
- int j = 1;
-
- /*
- * If we get here, we have a tuple with no nulls or var-widths up to
- * and including the target attribute, so we can use the cached offset
- * ... only we don't have it yet, or we'd not have got here. Since
- * it's cheap to compute offsets for fixed-width columns, we take the
- * opportunity to initialize the cached offsets for *all* the leading
- * fixed-width columns, in hope of avoiding future visits to this
- * routine.
- */
- TupleDescCompactAttr(tupleDesc, 0)->attcacheoff = 0;
-
- /* we might have set some offsets in the slow path previously */
- while (j < natts && TupleDescCompactAttr(tupleDesc, j)->attcacheoff > 0)
- j++;
-
- off = TupleDescCompactAttr(tupleDesc, j - 1)->attcacheoff +
- TupleDescCompactAttr(tupleDesc, j - 1)->attlen;
- for (; j < natts; j++)
+ for (; i < attnum; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, j);
+ if (att_isnull(i, bp))
+ continue;
- if (att->attlen <= 0)
- break;
-
- off = att_nominal_alignby(off, att->attalignby);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- att->attcacheoff = off;
-
- off += att->attlen;
+ off = att_nominal_alignby(off, cattr->attalignby);
+ off += cattr->attlen;
}
-
- Assert(j > attnum);
-
- off = TupleDescCompactAttr(tupleDesc, attnum)->attcacheoff;
}
else
{
- bool usecache = true;
- int i;
-
- /*
- * Now we know that we have to walk the tuple CAREFULLY. But we still
- * might be able to cache some offsets for next time.
- *
- * Note - This loop is a little tricky. For each non-null attribute,
- * we have to first account for alignment padding before the attr,
- * then advance over the attr based on its length. Nulls have no
- * storage and no alignment padding either. We can use/set
- * attcacheoff until we reach either a null or a var-width attribute.
- */
- off = 0;
- for (i = 0;; i++) /* loop exit is at "break" */
+ for (i = startAttr; i < firstNullAttr; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, i);
+ int attlen;
- if (HeapTupleHasNulls(tup) && att_isnull(i, bp))
- {
- usecache = false;
- continue; /* this cannot be the target att */
- }
+ cattr = TupleDescCompactAttr(tupleDesc, i);
+ attlen = cattr->attlen;
- /* If we know the next offset, we can skip the rest */
- if (usecache && att->attcacheoff >= 0)
- off = att->attcacheoff;
- else if (att->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be
- * no pad bytes in any case: then the offset will be valid for
- * either an aligned or unaligned value.
- */
- if (usecache &&
- off == att_nominal_alignby(off, att->attalignby))
- att->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, att->attalignby, -1,
- tp + off);
- usecache = false;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, att->attalignby);
+ /*
+ * cstrings don't exist in heap tuples. Use pg_assume to instruct
+ * the compiler not to emit the cstring-related code in
+ * att_addlength_pointer().
+ */
+ pg_assume(attlen > 0 || attlen == -1);
- if (usecache)
- att->attcacheoff = off;
- }
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ attlen,
+ tp + off);
+ off = att_addlength_pointer(off, attlen, tp + off);
+ }
- if (i == attnum)
- break;
+ for (; i < attnum; i++)
+ {
+ int attlen;
- off = att_addlength_pointer(off, att->attlen, tp + off);
+ if (att_isnull(i, bp))
+ continue;
- if (usecache && att->attlen <= 0)
- usecache = false;
+ cattr = TupleDescCompactAttr(tupleDesc, i);
+ attlen = cattr->attlen;
+
+ /* As above, heaptuples have no cstrings */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ off = att_pointer_alignby(off, cattr->attalignby, attlen,
+ tp + off);
+ off = att_addlength_pointer(off, attlen, tp + off);
}
}
- return fetchatt(TupleDescCompactAttr(tupleDesc, attnum), tp + off);
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+
+ return fetchatt(cattr, tp + off);
}
/* ----------------
@@ -1347,6 +1266,7 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,
Datum *values, bool *isnull)
{
HeapTupleHeader tup = tuple->t_data;
+ CompactAttribute *cattr;
bool hasnulls = HeapTupleHasNulls(tuple);
int tdesc_natts = tupleDesc->natts;
int natts; /* number of atts to extract */
@@ -1354,70 +1274,98 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,
char *tp; /* ptr to tuple data */
uint32 off; /* offset in tuple data */
bits8 *bp = tup->t_bits; /* ptr to null bitmap in tuple */
- bool slow = false; /* can we use/set attcacheoff? */
+ int firstNonCacheOffsetAttr;
+ int firstNullAttr;
natts = HeapTupleHeaderGetNatts(tup);
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
+
/*
* In inheritance situations, it is possible that the given tuple actually
* has more fields than the caller is expecting. Don't run off the end of
* the caller's arrays.
*/
natts = Min(natts, tdesc_natts);
+ firstNonCacheOffsetAttr = Min(tupleDesc->firstNonCachedOffsetAttr, natts);
+
+ if (hasnulls)
+ {
+ firstNullAttr = first_null_attr(bp, natts);
+
+ /*
+ * XXX: it'd be nice to use populate_isnull_array() here, but that
+ * requires that the isnull array's size is rounded up to the next
+ * multiple of 8. Doing that would require adjusting many locations
+ * that allocate the array.
+ */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, firstNullAttr);
+ }
+ else
+ firstNullAttr = natts;
tp = (char *) tup + tup->t_hoff;
+ attnum = 0;
- off = 0;
+ if (firstNonCacheOffsetAttr > 0)
+ {
+#ifdef USE_ASSERT_CHECKING
+ /* In Assert enabled builds, verify attcacheoff is correct */
+ int offcheck = 0;
+#endif
+ do
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ off = cattr->attcacheoff;
- for (attnum = 0; attnum < natts; attnum++)
+#ifdef USE_ASSERT_CHECKING
+ offcheck = att_nominal_alignby(offcheck, cattr->attalignby);
+ Assert(offcheck == cattr->attcacheoff);
+ offcheck += cattr->attlen;
+#endif
+
+ values[attnum] = fetch_att_noerr(tp + off,
+ cattr->attbyval,
+ cattr->attlen);
+ } while (++attnum < firstNonCacheOffsetAttr);
+ off += cattr->attlen;
+ }
+ else
+ off = 0;
+
+ for (; attnum < firstNullAttr; attnum++)
{
- CompactAttribute *thisatt = TupleDescCompactAttr(tupleDesc, attnum);
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
+ }
+
+ for (; attnum < natts; attnum++)
+ {
+ Assert(hasnulls);
- if (hasnulls && att_isnull(attnum, bp))
+ if (att_isnull(attnum, bp))
{
values[attnum] = (Datum) 0;
isnull[attnum] = true;
- slow = true; /* can't use attcacheoff anymore */
continue;
}
isnull[attnum] = false;
-
- if (!slow && thisatt->attcacheoff >= 0)
- off = thisatt->attcacheoff;
- else if (thisatt->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be no
- * pad bytes in any case: then the offset will be valid for either
- * an aligned or unaligned value.
- */
- if (!slow &&
- off == att_nominal_alignby(off, thisatt->attalignby))
- thisatt->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, thisatt->attalignby, -1,
- tp + off);
- slow = true;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, thisatt->attalignby);
-
- if (!slow)
- thisatt->attcacheoff = off;
- }
-
- values[attnum] = fetchatt(thisatt, tp + off);
-
- off = att_addlength_pointer(off, thisatt->attlen, tp + off);
-
- if (thisatt->attlen <= 0)
- slow = true; /* can't use attcacheoff anymore */
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
}
/*
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index d6350201e01..8c410853191 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -223,18 +223,6 @@ index_form_tuple_context(TupleDesc tupleDescriptor,
*
* This gets called from index_getattr() macro, and only in cases
* where we can't use cacheoffset and the value is not null.
- *
- * This caches attribute offsets in the attribute descriptor.
- *
- * An alternative way to speed things up would be to cache offsets
- * with the tuple, but that seems more difficult unless you take
- * the storage hit of actually putting those offsets into the
- * tuple you send to disk. Yuck.
- *
- * This scheme will be slightly slower than that, but should
- * perform well for queries which hit large #'s of tuples. After
- * you cache the offsets once, examining all the other tuples using
- * the same attribute descriptor will go much quicker. -cim 5/4/91
* ----------------
*/
Datum
@@ -242,205 +230,124 @@ nocache_index_getattr(IndexTuple tup,
int attnum,
TupleDesc tupleDesc)
{
+ CompactAttribute *cattr;
char *tp; /* ptr to data part of tuple */
bits8 *bp = NULL; /* ptr to null bitmap in tuple */
- bool slow = false; /* do we have to walk attrs? */
int data_off; /* tuple data offset */
int off; /* current offset within data */
+ int startAttr;
+ int firstNullAttr;
+ bool hasnulls = IndexTupleHasNulls(tup);
+ int i;
- /* ----------------
- * Three cases:
- *
- * 1: No nulls and no variable-width attributes.
- * 2: Has a null or a var-width AFTER att.
- * 3: Has nulls or var-widths BEFORE att.
- * ----------------
- */
-
- data_off = IndexInfoFindDataOffset(tup->t_info);
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
attnum--;
- if (IndexTupleHasNulls(tup))
- {
- /*
- * there's a null somewhere in the tuple
- *
- * check to see if desired att is null
- */
+ data_off = IndexInfoFindDataOffset(tup->t_info);
+ tp = (char *) tup + data_off;
- /* XXX "knows" t_bits are just after fixed tuple header! */
+ /*
+ * To minimize the number of attributes we need to look at, start walking
+ * the tuple at the attribute with the highest attcacheoff prior to attnum
+ * or the first NULL attribute prior to attnum, whichever comes first.
+ */
+ if (hasnulls)
+ {
bp = (bits8 *) ((char *) tup + sizeof(IndexTupleData));
-
- /*
- * Now check to see if any preceding bits are null...
- */
- {
- int byte = attnum >> 3;
- int finalbit = attnum & 0x07;
-
- /* check for nulls "before" final bit of last byte */
- if ((~bp[byte]) & ((1 << finalbit) - 1))
- slow = true;
- else
- {
- /* check for nulls in any "earlier" bytes */
- int i;
-
- for (i = 0; i < byte; i++)
- {
- if (bp[i] != 0xFF)
- {
- slow = true;
- break;
- }
- }
- }
- }
+ firstNullAttr = first_null_attr(bp, attnum);
}
+ else
+ firstNullAttr = attnum;
- tp = (char *) tup + data_off;
-
- if (!slow)
+ if (tupleDesc->firstNonCachedOffsetAttr > 0)
{
- CompactAttribute *att;
-
- /*
- * If we get here, there are no nulls up to and including the target
- * attribute. If we have a cached offset, we can use it.
- */
- att = TupleDescCompactAttr(tupleDesc, attnum);
- if (att->attcacheoff >= 0)
- return fetchatt(att, tp + att->attcacheoff);
-
/*
- * Otherwise, check for non-fixed-length attrs up to and including
- * target. If there aren't any, it's safe to cheaply initialize the
- * cached offsets for these attrs.
+ * Start at the highest attcacheoff attribute with no NULLs in prior
+ * attributes.
*/
- if (IndexTupleHasVarwidths(tup))
- {
- int j;
-
- for (j = 0; j <= attnum; j++)
- {
- if (TupleDescCompactAttr(tupleDesc, j)->attlen <= 0)
- {
- slow = true;
- break;
- }
- }
- }
+ startAttr = Min(tupleDesc->firstNonCachedOffsetAttr - 1, firstNullAttr);
+ off = TupleDescCompactAttr(tupleDesc, startAttr)->attcacheoff;
}
-
- if (!slow)
+ else
{
- int natts = tupleDesc->natts;
- int j = 1;
-
- /*
- * If we get here, we have a tuple with no nulls or var-widths up to
- * and including the target attribute, so we can use the cached offset
- * ... only we don't have it yet, or we'd not have got here. Since
- * it's cheap to compute offsets for fixed-width columns, we take the
- * opportunity to initialize the cached offsets for *all* the leading
- * fixed-width columns, in hope of avoiding future visits to this
- * routine.
- */
- TupleDescCompactAttr(tupleDesc, 0)->attcacheoff = 0;
+ /* Otherwise, start at the beginning... */
+ startAttr = 0;
+ off = 0;
+ }
- /* we might have set some offsets in the slow path previously */
- while (j < natts && TupleDescCompactAttr(tupleDesc, j)->attcacheoff > 0)
- j++;
+ /*
+ * Calculate 'off' up to the first NULL attr. We use two cheaper loops
+ * when the tuple has no variable-width columns. When variable-width
+ * columns exists, we use att_addlength_pointer() to move the offset
+ * beyond the current attribute.
+ */
+ if (IndexTupleHasVarwidths(tup))
+ {
+ /* Calculate the offset up until the first NULL */
+ for (i = startAttr; i < firstNullAttr; i++)
+ {
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- off = TupleDescCompactAttr(tupleDesc, j - 1)->attcacheoff +
- TupleDescCompactAttr(tupleDesc, j - 1)->attlen;
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+ off = att_addlength_pointer(off, cattr->attlen, tp + off);
+ }
- for (; j < natts; j++)
+ /* Calculate the offset for any remaining columns. */
+ for (; i < attnum; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, j);
+ Assert(hasnulls);
- if (att->attlen <= 0)
- break;
+ if (att_isnull(i, bp))
+ continue;
- off = att_nominal_alignby(off, att->attalignby);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- att->attcacheoff = off;
-
- off += att->attlen;
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+ off = att_addlength_pointer(off, cattr->attlen, tp + off);
}
-
- Assert(j > attnum);
-
- off = TupleDescCompactAttr(tupleDesc, attnum)->attcacheoff;
}
else
{
- bool usecache = true;
- int i;
+ /* Handle tuples with only fixed-width attributes */
- /*
- * Now we know that we have to walk the tuple CAREFULLY. But we still
- * might be able to cache some offsets for next time.
- *
- * Note - This loop is a little tricky. For each non-null attribute,
- * we have to first account for alignment padding before the attr,
- * then advance over the attr based on its length. Nulls have no
- * storage and no alignment padding either. We can use/set
- * attcacheoff until we reach either a null or a var-width attribute.
- */
- off = 0;
- for (i = 0;; i++) /* loop exit is at "break" */
+ /* Calculate the offset up until the first NULL */
+ for (i = startAttr; i < firstNullAttr; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, i);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- if (IndexTupleHasNulls(tup) && att_isnull(i, bp))
- {
- usecache = false;
- continue; /* this cannot be the target att */
- }
-
- /* If we know the next offset, we can skip the rest */
- if (usecache && att->attcacheoff >= 0)
- off = att->attcacheoff;
- else if (att->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be
- * no pad bytes in any case: then the offset will be valid for
- * either an aligned or unaligned value.
- */
- if (usecache &&
- off == att_nominal_alignby(off, att->attalignby))
- att->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, att->attalignby, -1,
- tp + off);
- usecache = false;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, att->attalignby);
+ Assert(cattr->attlen > 0);
+ off = att_nominal_alignby(off, cattr->attalignby);
+ off += cattr->attlen;
+ }
- if (usecache)
- att->attcacheoff = off;
- }
+ /* Calculate the offset for any remaining columns. */
+ for (; i < attnum; i++)
+ {
+ Assert(hasnulls);
- if (i == attnum)
- break;
+ if (att_isnull(i, bp))
+ continue;
- off = att_addlength_pointer(off, att->attlen, tp + off);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- if (usecache && att->attlen <= 0)
- usecache = false;
+ Assert(cattr->attlen > 0);
+ off = att_nominal_alignby(off, cattr->attalignby);
+ off += cattr->attlen;
}
}
- return fetchatt(TupleDescCompactAttr(tupleDesc, attnum), tp + off);
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ off = att_pointer_alignby(off, cattr->attalignby,
+ cattr->attlen, tp + off);
+ return fetchatt(cattr, tp + off);
}
/*
@@ -480,63 +387,87 @@ index_deform_tuple_internal(TupleDesc tupleDescriptor,
Datum *values, bool *isnull,
char *tp, bits8 *bp, int hasnulls)
{
+ CompactAttribute *cattr;
int natts = tupleDescriptor->natts; /* number of atts to extract */
- int attnum;
- int off = 0; /* offset in tuple data */
- bool slow = false; /* can we use/set attcacheoff? */
+ int attnum = 0;
+ uint32 off = 0; /* offset in tuple data */
+ int firstNonCacheOffsetAttr;
+ int firstNullAttr;
/* Assert to protect callers who allocate fixed-size arrays */
Assert(natts <= INDEX_MAX_KEYS);
- for (attnum = 0; attnum < natts; attnum++)
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDescriptor->firstNonCachedOffsetAttr >= 0);
+
+ firstNonCacheOffsetAttr = Min(tupleDescriptor->firstNonCachedOffsetAttr, natts);
+
+ if (hasnulls)
+ {
+ firstNullAttr = first_null_attr(bp, natts);
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, firstNullAttr);
+ }
+ else
+ firstNullAttr = natts;
+
+ if (firstNonCacheOffsetAttr > 0)
{
- CompactAttribute *thisatt = TupleDescCompactAttr(tupleDescriptor, attnum);
+#ifdef USE_ASSERT_CHECKING
+ /* In Assert enabled builds, verify attcacheoff is correct */
+ off = 0;
+#endif
- if (hasnulls && att_isnull(attnum, bp))
+ do
{
- values[attnum] = (Datum) 0;
- isnull[attnum] = true;
- slow = true; /* can't use attcacheoff anymore */
- continue;
- }
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDescriptor, attnum);
- isnull[attnum] = false;
+#ifdef USE_ASSERT_CHECKING
+ off = att_nominal_alignby(off, cattr->attalignby);
+ Assert(off == cattr->attcacheoff);
+ off += cattr->attlen;
+#endif
- if (!slow && thisatt->attcacheoff >= 0)
- off = thisatt->attcacheoff;
- else if (thisatt->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be no
- * pad bytes in any case: then the offset will be valid for either
- * an aligned or unaligned value.
- */
- if (!slow &&
- off == att_nominal_alignby(off, thisatt->attalignby))
- thisatt->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, thisatt->attalignby, -1,
- tp + off);
- slow = true;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, thisatt->attalignby);
+ values[attnum] = fetch_att_noerr(tp + cattr->attcacheoff, cattr->attbyval,
+ cattr->attlen);
+ } while (++attnum < firstNonCacheOffsetAttr);
- if (!slow)
- thisatt->attcacheoff = off;
- }
+ off = cattr->attcacheoff + cattr->attlen;
+ }
- values[attnum] = fetchatt(thisatt, tp + off);
+ for (; attnum < firstNullAttr; attnum++)
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDescriptor, attnum);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
+ }
- off = att_addlength_pointer(off, thisatt->attlen, tp + off);
+ for (; attnum < natts; attnum++)
+ {
+ Assert(hasnulls);
- if (thisatt->attlen <= 0)
- slow = true; /* can't use attcacheoff anymore */
+ if (att_isnull(attnum, bp))
+ {
+ values[attnum] = (Datum) 0;
+ isnull[attnum] = true;
+ continue;
+ }
+
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDescriptor, attnum);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
}
}
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 2137385a833..c68561337d7 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -197,6 +197,10 @@ CreateTemplateTupleDesc(int natts)
desc->tdtypmod = -1;
desc->tdrefcount = -1; /* assume not reference-counted */
+ /* This will be set to the correct value by TupleDescFinalize() */
+ desc->firstNonCachedOffsetAttr = -1;
+ desc->firstNonGuaranteedAttr = -1;
+
return desc;
}
@@ -457,6 +461,9 @@ TupleDescCopy(TupleDesc dst, TupleDesc src)
* descriptor to another.
*
* !!! Constraints and defaults are not copied !!!
+ *
+ * The caller must take care of calling TupleDescFinalize() on 'dst' once all
+ * TupleDesc changes have been made.
*/
void
TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
@@ -489,6 +496,50 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
populate_compact_attribute(dst, dstAttno - 1);
}
+/*
+ * TupleDescFinalize
+ * Finalize the given TupleDesc. This must be called after the
+ * attributes arrays have been populated or adjusted by any code.
+ *
+ * Must be called after populate_compact_attribute() and before
+ * BlessTupleDesc().
+ */
+void
+TupleDescFinalize(TupleDesc tupdesc)
+{
+ int firstNonCachedOffsetAttr = 0;
+ int firstNonGuaranteedAttr = tupdesc->natts;
+ int off = 0;
+
+ for (int i = 0; i < tupdesc->natts; i++)
+ {
+ CompactAttribute *cattr = TupleDescCompactAttr(tupdesc, i);
+
+ /*
+ * Find the highest attnum which is guaranteed to exist in all tuples
+ * in the table. We currently only pay attention to byval attributes
+ * to allow additional optimizations during tuple deformation.
+ */
+ if (firstNonGuaranteedAttr == tupdesc->natts &&
+ (cattr->attnullability != ATTNULLABLE_VALID || !cattr->attbyval ||
+ cattr->atthasmissing || cattr->attisdropped || cattr->attlen <= 0))
+ firstNonGuaranteedAttr = i;
+
+ if (cattr->attlen <= 0)
+ break;
+
+ off = att_nominal_alignby(off, cattr->attalignby);
+
+ cattr->attcacheoff = off;
+
+ off += cattr->attlen;
+ firstNonCachedOffsetAttr = i + 1;
+ }
+
+ tupdesc->firstNonCachedOffsetAttr = firstNonCachedOffsetAttr;
+ tupdesc->firstNonGuaranteedAttr = firstNonGuaranteedAttr;
+}
+
/*
* Free a TupleDesc including all substructure
*/
diff --git a/src/backend/access/spgist/spgutils.c b/src/backend/access/spgist/spgutils.c
index b246e8127db..a4694bd8065 100644
--- a/src/backend/access/spgist/spgutils.c
+++ b/src/backend/access/spgist/spgutils.c
@@ -335,9 +335,6 @@ getSpGistTupleDesc(Relation index, SpGistTypeDesc *keyType)
/* We shouldn't need to bother with making these valid: */
att->attcompression = InvalidCompressionMethod;
att->attcollation = InvalidOid;
- /* In case we changed typlen, we'd better reset following offsets */
- for (int i = spgFirstIncludeColumn; i < outTupDesc->natts; i++)
- TupleDescCompactAttr(outTupDesc, i)->attcacheoff = -1;
populate_compact_attribute(outTupDesc, spgKeyColumn);
TupleDescFinalize(outTupDesc);
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index bfd3ebc601e..0b635486993 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -1944,7 +1944,7 @@ ExecPartitionCheckEmitError(ResultRelInfo *resultRelInfo,
*/
if (map != NULL)
slot = execute_attr_map_slot(map, slot,
- MakeTupleTableSlot(tupdesc, &TTSOpsVirtual));
+ MakeTupleTableSlot(tupdesc, &TTSOpsVirtual, 0));
modifiedCols = bms_union(ExecGetInsertedCols(rootrel, estate),
ExecGetUpdatedCols(rootrel, estate));
}
@@ -2060,7 +2060,7 @@ ExecConstraints(ResultRelInfo *resultRelInfo,
*/
if (map != NULL)
slot = execute_attr_map_slot(map, slot,
- MakeTupleTableSlot(tupdesc, &TTSOpsVirtual));
+ MakeTupleTableSlot(tupdesc, &TTSOpsVirtual, 0));
modifiedCols = bms_union(ExecGetInsertedCols(rootrel, estate),
ExecGetUpdatedCols(rootrel, estate));
rel = rootrel->ri_RelationDesc;
@@ -2196,7 +2196,7 @@ ReportNotNullViolationError(ResultRelInfo *resultRelInfo, TupleTableSlot *slot,
*/
if (map != NULL)
slot = execute_attr_map_slot(map, slot,
- MakeTupleTableSlot(tupdesc, &TTSOpsVirtual));
+ MakeTupleTableSlot(tupdesc, &TTSOpsVirtual, 0));
modifiedCols = bms_union(ExecGetInsertedCols(rootrel, estate),
ExecGetUpdatedCols(rootrel, estate));
rel = rootrel->ri_RelationDesc;
@@ -2304,7 +2304,7 @@ ExecWithCheckOptions(WCOKind kind, ResultRelInfo *resultRelInfo,
*/
if (map != NULL)
slot = execute_attr_map_slot(map, slot,
- MakeTupleTableSlot(tupdesc, &TTSOpsVirtual));
+ MakeTupleTableSlot(tupdesc, &TTSOpsVirtual, 0));
modifiedCols = bms_union(ExecGetInsertedCols(rootrel, estate),
ExecGetUpdatedCols(rootrel, estate));
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 07b248aa5f3..14b2a9f0af6 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -992,118 +992,6 @@ tts_buffer_heap_store_tuple(TupleTableSlot *slot, HeapTuple tuple,
}
}
-/*
- * slot_deform_heap_tuple_internal
- * An always inline helper function for use in slot_deform_heap_tuple to
- * allow the compiler to emit specialized versions of this function for
- * various combinations of "slow" and "hasnulls". For example, if a
- * given tuple has no nulls, then we needn't check "hasnulls" for every
- * attribute that we're deforming. The caller can just call this
- * function with hasnulls set to constant-false and have the compiler
- * remove the constant-false branches and emit more optimal code.
- *
- * Returns the next attnum to deform, which can be equal to natts when the
- * function manages to deform all requested attributes. *offp is an input and
- * output parameter which is the byte offset within the tuple to start deforming
- * from which, on return, gets set to the offset where the next attribute
- * should be deformed from. *slowp is set to true when subsequent deforming
- * of this tuple must use a version of this function with "slow" passed as
- * true.
- *
- * Callers cannot assume when we return "attnum" (i.e. all requested
- * attributes have been deformed) that slow mode isn't required for any
- * additional deforming as the final attribute may have caused a switch to
- * slow mode.
- */
-static pg_attribute_always_inline int
-slot_deform_heap_tuple_internal(TupleTableSlot *slot, HeapTuple tuple,
- int attnum, int natts, bool slow,
- bool hasnulls, uint32 *offp, bool *slowp)
-{
- TupleDesc tupleDesc = slot->tts_tupleDescriptor;
- Datum *values = slot->tts_values;
- bool *isnull = slot->tts_isnull;
- HeapTupleHeader tup = tuple->t_data;
- char *tp; /* ptr to tuple data */
- bits8 *bp = tup->t_bits; /* ptr to null bitmap in tuple */
- bool slownext = false;
-
- tp = (char *) tup + tup->t_hoff;
-
- for (; attnum < natts; attnum++)
- {
- CompactAttribute *thisatt = TupleDescCompactAttr(tupleDesc, attnum);
-
- if (hasnulls && att_isnull(attnum, bp))
- {
- values[attnum] = (Datum) 0;
- isnull[attnum] = true;
- if (!slow)
- {
- *slowp = true;
- return attnum + 1;
- }
- else
- continue;
- }
-
- isnull[attnum] = false;
-
- /* calculate the offset of this attribute */
- if (!slow && thisatt->attcacheoff >= 0)
- *offp = thisatt->attcacheoff;
- else if (thisatt->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be no
- * pad bytes in any case: then the offset will be valid for either
- * an aligned or unaligned value.
- */
- if (!slow && *offp == att_nominal_alignby(*offp, thisatt->attalignby))
- thisatt->attcacheoff = *offp;
- else
- {
- *offp = att_pointer_alignby(*offp,
- thisatt->attalignby,
- -1,
- tp + *offp);
-
- if (!slow)
- slownext = true;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- *offp = att_nominal_alignby(*offp, thisatt->attalignby);
-
- if (!slow)
- thisatt->attcacheoff = *offp;
- }
-
- values[attnum] = fetchatt(thisatt, tp + *offp);
-
- *offp = att_addlength_pointer(*offp, thisatt->attlen, tp + *offp);
-
- /* check if we need to switch to slow mode */
- if (!slow)
- {
- /*
- * We're unable to deform any further if the above code set
- * 'slownext', or if this isn't a fixed-width attribute.
- */
- if (slownext || thisatt->attlen <= 0)
- {
- *slowp = true;
- return attnum + 1;
- }
- }
- }
-
- return natts;
-}
-
/*
* slot_deform_heap_tuple
* Given a TupleTableSlot, extract data from the slot's physical tuple
@@ -1125,94 +1013,228 @@ static pg_attribute_always_inline void
slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int reqnatts)
{
- bool hasnulls = HeapTupleHasNulls(tuple);
- int attnum;
+ CompactAttribute *cattr;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+ HeapTupleHeader tup = tuple->t_data;
+ size_t attnum;
+ int firstNonCacheOffsetAttr;
+ int firstNonGuaranteedAttr;
+ int firstNullAttr;
int natts;
+ Datum *values;
+ bool *isnull;
+ char *tp; /* ptr to tuple data */
uint32 off; /* offset in tuple data */
- bool slow; /* can we use/set attcacheoff? */
- /* We can only fetch as many attributes as the tuple has. */
- natts = Min(HeapTupleHeaderGetNatts(tuple->t_data), reqnatts);
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
+
+ isnull = slot->tts_isnull;
/*
- * Check whether the first call for this tuple, and initialize or restore
- * loop state.
+ * Some callers may form and deform tuples prior to NOT NULL constraints
+ * being checked. Here we'd like to optimize the case where we only need
+ * to fetch attributes before or up to the point where the attribute is
+ * guaranteed to exist in the tuple. We rely on the slot flag being set
+ * correctly to only enable this optimization when it's valid to do so.
+ * This optimization allows us to save fetching the number of attributes
+ * from the tuple and saves the additional cost of handling non-byval
+ * attrs.
*/
+ firstNonGuaranteedAttr = Min(reqnatts, slot->tts_first_nonguaranteed);
+
+ firstNonCacheOffsetAttr = tupleDesc->firstNonCachedOffsetAttr;
+
+ if (HeapTupleHasNulls(tuple))
+ {
+ natts = HeapTupleHeaderGetNatts(tup);
+ tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits) +
+ BITMAPLEN(natts));
+
+ natts = Min(natts, reqnatts);
+ if (natts > firstNonGuaranteedAttr)
+ {
+ bits8 *bp = tup->t_bits;
+
+ /* Find the first NULL attr */
+ firstNullAttr = first_null_attr(bp, natts);
+
+ /*
+ * And populate the isnull array for all attributes being fetched
+ * from the tuple.
+ */
+ populate_isnull_array(bp, natts, isnull);
+
+ /* We can only use any cached offsets until the first NULL attr */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr,
+ firstNullAttr);
+ }
+ else
+ {
+ /* Otherwise all required columns are guaranteed to exist */
+ firstNullAttr = natts;
+ }
+ }
+ else
+ {
+ tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits));
+
+ /*
+ * We only need to look at the tuple's natts if we need more than the
+ * guaranteed number of columns
+ */
+ if (reqnatts > firstNonGuaranteedAttr)
+ natts = Min(HeapTupleHeaderGetNatts(tup), reqnatts);
+ else
+ {
+ /* No need to access the number of attributes in the tuple */
+ natts = reqnatts;
+ }
+
+ /* All attrs can be fetched without checking for NULLs */
+ firstNullAttr = natts;
+ }
+
attnum = slot->tts_nvalid;
+ values = slot->tts_values;
slot->tts_nvalid = reqnatts;
- if (attnum == 0)
+
+ /* Ensure we calculated tp correctly */
+ Assert(tp == (char *) tup + tup->t_hoff);
+
+ if (attnum < firstNonGuaranteedAttr)
{
- /* Start from the first attribute */
- off = 0;
- slow = false;
+ int attlen;
+
+ for (; attnum < firstNonGuaranteedAttr; attnum++)
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ attlen = cattr->attlen;
+
+ /* We don't expect any non-byval types */
+ pg_assume(attlen > 0);
+ Assert(cattr->attbyval == true);
+
+ off = cattr->attcacheoff;
+ values[attnum] = fetch_att_noerr(tp + off, true, attlen);
+ }
+
+ off += attlen;
+
+ if (attnum == reqnatts)
+ goto done;
}
else
{
/* Restore state from previous execution */
off = *offp;
- slow = TTS_SLOW(slot);
+
+ /* We expect *offp to be set to 0 when attnum == 0 */
+ Assert(off == 0 || attnum > 0);
}
+ /* We can only fetch as many attributes as the tuple has. */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, natts);
+
/*
- * If 'slow' isn't set, try deforming using deforming code that does not
- * contain any of the extra checks required for non-fixed offset
- * deforming. During deforming, if or when we find a NULL or a variable
- * length attribute, we'll switch to a deforming method which includes the
- * extra code required for non-fixed offset deforming, a.k.a slow mode.
- * Because this is performance critical, we inline
- * slot_deform_heap_tuple_internal passing the 'slow' and 'hasnull'
- * parameters as constants to allow the compiler to emit specialized code
- * with the known-const false comparisons and subsequent branches removed.
+ * Handle the portion of the tuple that we have cached the offset for up
+ * to the first NULL attribute. The offset is effectively fixed for
+ * these, so we can use the CompactAttribute's attcacheoff.
*/
- if (!slow)
+ if (attnum < firstNonCacheOffsetAttr)
{
- /* Tuple without any NULLs? We can skip doing any NULL checking */
- if (!hasnulls)
- attnum = slot_deform_heap_tuple_internal(slot,
- tuple,
- attnum,
- natts,
- false, /* slow */
- false, /* hasnulls */
- &off,
- &slow);
- else
- attnum = slot_deform_heap_tuple_internal(slot,
- tuple,
- attnum,
- natts,
- false, /* slow */
- true, /* hasnulls */
- &off,
- &slow);
+ int attlen;
+
+ for (; attnum < firstNonCacheOffsetAttr; attnum++)
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ attlen = cattr->attlen;
+
+ off = cattr->attcacheoff;
+ values[attnum] = fetch_att_noerr(tp + off,
+ cattr->attbyval,
+ attlen);
+ }
+
+ /*
+ * Point the offset after the end of the last attribute with a cached
+ * offset. We expect the final cached offset attribute to have a
+ * fixed width, so just add the attlen to the attcacheoff
+ */
+ Assert(attlen > 0);
+ off += attlen;
}
- /* If there's still work to do then we must be in slow mode */
- if (attnum < natts)
+ /*
+ * Handle any portion of the tuple that doesn't have a fixed offset up
+ * until the first NULL attribute. This loop only differs from the one
+ * after it by the NULL checks.
+ */
+ for (; attnum < firstNullAttr; attnum++)
{
- /* XXX is it worth adding a separate call when hasnulls is false? */
- attnum = slot_deform_heap_tuple_internal(slot,
- tuple,
- attnum,
- natts,
- true, /* slow */
- hasnulls,
- &off,
- &slow);
+ int attlen;
+
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ attlen = cattr->attlen;
+
+ /*
+ * cstrings don't exist in heap tuples. Use pg_assume to instruct the
+ * compiler not to emit the cstring-related code in
+ * align_fetch_then_add().
+ */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ attlen,
+ cattr->attalignby);
}
/*
- * Save state for next execution
+ * Now handle any remaining attributes in the tuple up to the requested
+ * attnum. This time, include NULL checks as we're now at the first NULL
+ * attribute.
*/
- *offp = off;
- if (slow)
- slot->tts_flags |= TTS_FLAG_SLOW;
- else
- slot->tts_flags &= ~TTS_FLAG_SLOW;
+ for (; attnum < natts; attnum++)
+ {
+ int attlen;
- /* Fetch any missing attrs and raise an error if reqnatts is invalid. */
+ if (isnull[attnum])
+ {
+ values[attnum] = (Datum) 0;
+ continue;
+ }
+
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ attlen = cattr->attlen;
+
+ /* As above, we don't expect cstrings */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ attlen,
+ cattr->attalignby);
+ }
+
+ /* Fetch any missing attrs and raise an error if reqnatts is invalid */
if (unlikely(attnum < reqnatts))
+ {
+ *offp = off;
slot_getmissingattrs(slot, attnum, reqnatts);
+ return;
+ }
+done:
+
+ /* Save current offset for next execution */
+ *offp = off;
}
const TupleTableSlotOps TTSOpsVirtual = {
@@ -1307,7 +1329,7 @@ const TupleTableSlotOps TTSOpsBufferHeapTuple = {
*/
TupleTableSlot *
MakeTupleTableSlot(TupleDesc tupleDesc,
- const TupleTableSlotOps *tts_ops)
+ const TupleTableSlotOps *tts_ops, uint16 flags)
{
Size basesz,
allocsz;
@@ -1331,6 +1353,7 @@ MakeTupleTableSlot(TupleDesc tupleDesc,
*((const TupleTableSlotOps **) &slot->tts_ops) = tts_ops;
slot->type = T_TupleTableSlot;
slot->tts_flags |= TTS_FLAG_EMPTY;
+ slot->tts_flags |= flags;
if (tupleDesc != NULL)
slot->tts_flags |= TTS_FLAG_FIXED;
slot->tts_tupleDescriptor = tupleDesc;
@@ -1342,12 +1365,31 @@ MakeTupleTableSlot(TupleDesc tupleDesc,
slot->tts_values = (Datum *)
(((char *) slot)
+ MAXALIGN(basesz));
+
+ /*
+ * We round the size of tts_isnull up to the next highest multiple of
+ * 8. This is needed as populate_isnull_array() operates on 8
+ * elements at a time when converting a tuple's NULL bitmap into a
+ * boolean array.
+ */
slot->tts_isnull = (bool *)
(((char *) slot)
+ MAXALIGN(basesz)
- + MAXALIGN(tupleDesc->natts * sizeof(Datum)));
+ + TYPEALIGN(8, tupleDesc->natts * sizeof(Datum)));
PinTupleDesc(tupleDesc);
+
+ /*
+ * Precalculate the maximum guaranteed attribute that has to exist in
+ * every tuple which gets deformed into this slot. When the
+ * TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS flag is enabled, we simply take
+ * the precalculated value from the tupleDesc, otherwise the
+ * optimization is disabled, and we set the value to 0.
+ */
+ if ((flags & TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS) != 0)
+ slot->tts_first_nonguaranteed = tupleDesc->firstNonGuaranteedAttr;
+ else
+ slot->tts_first_nonguaranteed = 0;
}
/*
@@ -1366,9 +1408,9 @@ MakeTupleTableSlot(TupleDesc tupleDesc,
*/
TupleTableSlot *
ExecAllocTableSlot(List **tupleTable, TupleDesc desc,
- const TupleTableSlotOps *tts_ops)
+ const TupleTableSlotOps *tts_ops, uint16 flags)
{
- TupleTableSlot *slot = MakeTupleTableSlot(desc, tts_ops);
+ TupleTableSlot *slot = MakeTupleTableSlot(desc, tts_ops, flags);
*tupleTable = lappend(*tupleTable, slot);
@@ -1435,7 +1477,7 @@ TupleTableSlot *
MakeSingleTupleTableSlot(TupleDesc tupdesc,
const TupleTableSlotOps *tts_ops)
{
- TupleTableSlot *slot = MakeTupleTableSlot(tupdesc, tts_ops);
+ TupleTableSlot *slot = MakeTupleTableSlot(tupdesc, tts_ops, 0);
return slot;
}
@@ -1515,8 +1557,14 @@ ExecSetSlotDescriptor(TupleTableSlot *slot, /* slot to change */
*/
slot->tts_values = (Datum *)
MemoryContextAlloc(slot->tts_mcxt, tupdesc->natts * sizeof(Datum));
+
+ /*
+ * We round the size of tts_isnull up to the next highest multiple of 8.
+ * This is needed as populate_isnull_array() operates on 8 elements at a
+ * time when converting a tuple's NULL bitmap into a boolean array.
+ */
slot->tts_isnull = (bool *)
- MemoryContextAlloc(slot->tts_mcxt, tupdesc->natts * sizeof(bool));
+ MemoryContextAlloc(slot->tts_mcxt, TYPEALIGN(8, tupdesc->natts * sizeof(bool)));
}
/* --------------------------------
@@ -1978,7 +2026,7 @@ ExecInitResultSlot(PlanState *planstate, const TupleTableSlotOps *tts_ops)
TupleTableSlot *slot;
slot = ExecAllocTableSlot(&planstate->state->es_tupleTable,
- planstate->ps_ResultTupleDesc, tts_ops);
+ planstate->ps_ResultTupleDesc, tts_ops, 0);
planstate->ps_ResultTupleSlot = slot;
planstate->resultopsfixed = planstate->ps_ResultTupleDesc != NULL;
@@ -2006,10 +2054,11 @@ ExecInitResultTupleSlotTL(PlanState *planstate,
*/
void
ExecInitScanTupleSlot(EState *estate, ScanState *scanstate,
- TupleDesc tupledesc, const TupleTableSlotOps *tts_ops)
+ TupleDesc tupledesc, const TupleTableSlotOps *tts_ops,
+ uint16 flags)
{
scanstate->ss_ScanTupleSlot = ExecAllocTableSlot(&estate->es_tupleTable,
- tupledesc, tts_ops);
+ tupledesc, tts_ops, flags);
scanstate->ps.scandesc = tupledesc;
scanstate->ps.scanopsfixed = tupledesc != NULL;
scanstate->ps.scanops = tts_ops;
@@ -2029,7 +2078,7 @@ ExecInitExtraTupleSlot(EState *estate,
TupleDesc tupledesc,
const TupleTableSlotOps *tts_ops)
{
- return ExecAllocTableSlot(&estate->es_tupleTable, tupledesc, tts_ops);
+ return ExecAllocTableSlot(&estate->es_tupleTable, tupledesc, tts_ops, 0);
}
/* ----------------
@@ -2261,10 +2310,16 @@ ExecTypeSetColNames(TupleDesc typeInfo, List *namesList)
* This happens "for free" if the tupdesc came from a relcache entry, but
* not if we have manufactured a tupdesc for a transient RECORD datatype.
* In that case we have to notify typcache.c of the existence of the type.
+ *
+ * TupleDescFinalize() must be called on the TupleDesc before calling this
+ * function.
*/
TupleDesc
BlessTupleDesc(TupleDesc tupdesc)
{
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupdesc->firstNonCachedOffsetAttr >= 0);
+
if (tupdesc->tdtypeid == RECORDOID &&
tupdesc->tdtypmod < 0)
assign_record_type_typmod(tupdesc);
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index a7955e476f9..f62582859f9 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -711,7 +711,7 @@ ExecCreateScanSlotFromOuterPlan(EState *estate,
outerPlan = outerPlanState(scanstate);
tupDesc = ExecGetResultType(outerPlan);
- ExecInitScanTupleSlot(estate, scanstate, tupDesc, tts_ops);
+ ExecInitScanTupleSlot(estate, scanstate, tupDesc, tts_ops, 0);
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
index 7d487a165fa..c5c321b4f42 100644
--- a/src/backend/executor/nodeAgg.c
+++ b/src/backend/executor/nodeAgg.c
@@ -1682,7 +1682,7 @@ find_hash_columns(AggState *aggstate)
&perhash->hashfunctions);
perhash->hashslot =
ExecAllocTableSlot(&estate->es_tupleTable, hashDesc,
- &TTSOpsMinimalTuple);
+ &TTSOpsMinimalTuple, 0);
list_free(hashTlist);
bms_free(colnos);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index c68c26cbf38..dcb7599b96c 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -381,7 +381,10 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
*/
ExecInitScanTupleSlot(estate, &scanstate->ss,
RelationGetDescr(currentRelation),
- table_slot_callbacks(currentRelation));
+ table_slot_callbacks(currentRelation), 0);
+
+ scanstate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeCtescan.c b/src/backend/executor/nodeCtescan.c
index e6e476388e5..45b09d93b93 100644
--- a/src/backend/executor/nodeCtescan.c
+++ b/src/backend/executor/nodeCtescan.c
@@ -261,7 +261,7 @@ ExecInitCteScan(CteScan *node, EState *estate, int eflags)
*/
ExecInitScanTupleSlot(estate, &scanstate->ss,
ExecGetResultType(scanstate->cteplanstate),
- &TTSOpsMinimalTuple);
+ &TTSOpsMinimalTuple, 0);
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeCustom.c b/src/backend/executor/nodeCustom.c
index a9ad5af6a98..b7cc890cd20 100644
--- a/src/backend/executor/nodeCustom.c
+++ b/src/backend/executor/nodeCustom.c
@@ -79,14 +79,14 @@ ExecInitCustomScan(CustomScan *cscan, EState *estate, int eflags)
TupleDesc scan_tupdesc;
scan_tupdesc = ExecTypeFromTL(cscan->custom_scan_tlist);
- ExecInitScanTupleSlot(estate, &css->ss, scan_tupdesc, slotOps);
+ ExecInitScanTupleSlot(estate, &css->ss, scan_tupdesc, slotOps, 0);
/* Node's targetlist will contain Vars with varno = INDEX_VAR */
tlistvarno = INDEX_VAR;
}
else
{
ExecInitScanTupleSlot(estate, &css->ss, RelationGetDescr(scan_rel),
- slotOps);
+ slotOps, 0);
/* Node's targetlist will contain Vars with varno = scanrelid */
tlistvarno = scanrelid;
}
diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c
index 8721b67b7cc..6f0daddce07 100644
--- a/src/backend/executor/nodeForeignscan.c
+++ b/src/backend/executor/nodeForeignscan.c
@@ -191,7 +191,7 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
scan_tupdesc = ExecTypeFromTL(node->fdw_scan_tlist);
ExecInitScanTupleSlot(estate, &scanstate->ss, scan_tupdesc,
- &TTSOpsHeapTuple);
+ &TTSOpsHeapTuple, 0);
/* Node's targetlist will contain Vars with varno = INDEX_VAR */
tlistvarno = INDEX_VAR;
}
@@ -202,7 +202,7 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
/* don't trust FDWs to return tuples fulfilling NOT NULL constraints */
scan_tupdesc = CreateTupleDescCopy(RelationGetDescr(currentRelation));
ExecInitScanTupleSlot(estate, &scanstate->ss, scan_tupdesc,
- &TTSOpsHeapTuple);
+ &TTSOpsHeapTuple, 0);
/* Node's targetlist will contain Vars with varno = scanrelid */
tlistvarno = scanrelid;
}
diff --git a/src/backend/executor/nodeFunctionscan.c b/src/backend/executor/nodeFunctionscan.c
index feb82d64967..222741adf3b 100644
--- a/src/backend/executor/nodeFunctionscan.c
+++ b/src/backend/executor/nodeFunctionscan.c
@@ -494,7 +494,7 @@ ExecInitFunctionScan(FunctionScan *node, EState *estate, int eflags)
* Initialize scan slot and type.
*/
ExecInitScanTupleSlot(estate, &scanstate->ss, scan_tupdesc,
- &TTSOpsMinimalTuple);
+ &TTSOpsMinimalTuple, 0);
/*
* Initialize result slot, type and projection.
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index c2d09374517..144a57fde95 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -567,7 +567,10 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
*/
tupDesc = ExecTypeFromTL(node->indextlist);
ExecInitScanTupleSlot(estate, &indexstate->ss, tupDesc,
- &TTSOpsVirtual);
+ &TTSOpsVirtual, 0);
+
+ indexstate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
/*
* We need another slot, in a format that's suitable for the table AM, for
@@ -576,7 +579,7 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
indexstate->ioss_TableSlot =
ExecAllocTableSlot(&estate->es_tupleTable,
RelationGetDescr(currentRelation),
- table_slot_callbacks(currentRelation));
+ table_slot_callbacks(currentRelation), 0);
/*
* Initialize result type and projection info. The node's targetlist will
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index a616abff04c..e7bebb89517 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -938,7 +938,10 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
*/
ExecInitScanTupleSlot(estate, &indexstate->ss,
RelationGetDescr(currentRelation),
- table_slot_callbacks(currentRelation));
+ table_slot_callbacks(currentRelation), 0);
+
+ indexstate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeNamedtuplestorescan.c b/src/backend/executor/nodeNamedtuplestorescan.c
index fdfccc8169f..29d862a4001 100644
--- a/src/backend/executor/nodeNamedtuplestorescan.c
+++ b/src/backend/executor/nodeNamedtuplestorescan.c
@@ -137,7 +137,7 @@ ExecInitNamedTuplestoreScan(NamedTuplestoreScan *node, EState *estate, int eflag
* The scan tuple type is specified for the tuplestore.
*/
ExecInitScanTupleSlot(estate, &scanstate->ss, scanstate->tupdesc,
- &TTSOpsMinimalTuple);
+ &TTSOpsMinimalTuple, 0);
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c
index 1b0af70fd7a..a1da05ecc18 100644
--- a/src/backend/executor/nodeSamplescan.c
+++ b/src/backend/executor/nodeSamplescan.c
@@ -128,7 +128,11 @@ ExecInitSampleScan(SampleScan *node, EState *estate, int eflags)
/* and create slot with appropriate rowtype */
ExecInitScanTupleSlot(estate, &scanstate->ss,
RelationGetDescr(scanstate->ss.ss_currentRelation),
- table_slot_callbacks(scanstate->ss.ss_currentRelation));
+ table_slot_callbacks(scanstate->ss.ss_currentRelation),
+ 0);
+
+ scanstate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index af3c788ce8b..8f219f60a93 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -244,7 +244,8 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags)
/* and create slot with the appropriate rowtype */
ExecInitScanTupleSlot(estate, &scanstate->ss,
RelationGetDescr(scanstate->ss.ss_currentRelation),
- table_slot_callbacks(scanstate->ss.ss_currentRelation));
+ table_slot_callbacks(scanstate->ss.ss_currentRelation),
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS);
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeSubqueryscan.c b/src/backend/executor/nodeSubqueryscan.c
index 4fd6f6fb4a5..70914e8189c 100644
--- a/src/backend/executor/nodeSubqueryscan.c
+++ b/src/backend/executor/nodeSubqueryscan.c
@@ -130,7 +130,8 @@ ExecInitSubqueryScan(SubqueryScan *node, EState *estate, int eflags)
*/
ExecInitScanTupleSlot(estate, &subquerystate->ss,
ExecGetResultType(subquerystate->subplan),
- ExecGetResultSlotOps(subquerystate->subplan, NULL));
+ ExecGetResultSlotOps(subquerystate->subplan, NULL),
+ 0);
/*
* The slot used as the scantuple isn't the slot above (outside of EPQ),
diff --git a/src/backend/executor/nodeTableFuncscan.c b/src/backend/executor/nodeTableFuncscan.c
index 52070d147a4..769b9766542 100644
--- a/src/backend/executor/nodeTableFuncscan.c
+++ b/src/backend/executor/nodeTableFuncscan.c
@@ -148,7 +148,7 @@ ExecInitTableFuncScan(TableFuncScan *node, EState *estate, int eflags)
tf->colcollations);
/* and the corresponding scan slot */
ExecInitScanTupleSlot(estate, &scanstate->ss, tupdesc,
- &TTSOpsMinimalTuple);
+ &TTSOpsMinimalTuple, 0);
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index 503817da65b..decc3167ba7 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -394,7 +394,10 @@ ExecInitTidRangeScan(TidRangeScan *node, EState *estate, int eflags)
*/
ExecInitScanTupleSlot(estate, &tidrangestate->ss,
RelationGetDescr(currentRelation),
- table_slot_callbacks(currentRelation));
+ table_slot_callbacks(currentRelation), 0);
+
+ tidrangestate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeTidscan.c b/src/backend/executor/nodeTidscan.c
index 4eddb0828b5..26593b930af 100644
--- a/src/backend/executor/nodeTidscan.c
+++ b/src/backend/executor/nodeTidscan.c
@@ -536,7 +536,10 @@ ExecInitTidScan(TidScan *node, EState *estate, int eflags)
*/
ExecInitScanTupleSlot(estate, &tidstate->ss,
RelationGetDescr(currentRelation),
- table_slot_callbacks(currentRelation));
+ table_slot_callbacks(currentRelation), 0);
+
+ tidstate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeValuesscan.c b/src/backend/executor/nodeValuesscan.c
index e663fb68cfc..effc896ea1c 100644
--- a/src/backend/executor/nodeValuesscan.c
+++ b/src/backend/executor/nodeValuesscan.c
@@ -247,7 +247,7 @@ ExecInitValuesScan(ValuesScan *node, EState *estate, int eflags)
* Get info about values list, initialize scan slot with it.
*/
tupdesc = ExecTypeFromExprList((List *) linitial(node->values_lists));
- ExecInitScanTupleSlot(estate, &scanstate->ss, tupdesc, &TTSOpsVirtual);
+ ExecInitScanTupleSlot(estate, &scanstate->ss, tupdesc, &TTSOpsVirtual, 0);
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeWorktablescan.c b/src/backend/executor/nodeWorktablescan.c
index 210cc44f911..66e904c7636 100644
--- a/src/backend/executor/nodeWorktablescan.c
+++ b/src/backend/executor/nodeWorktablescan.c
@@ -165,7 +165,7 @@ ExecInitWorkTableScan(WorkTableScan *node, EState *estate, int eflags)
scanstate->ss.ps.resultopsset = true;
scanstate->ss.ps.resultopsfixed = false;
- ExecInitScanTupleSlot(estate, &scanstate->ss, NULL, &TTSOpsMinimalTuple);
+ ExecInitScanTupleSlot(estate, &scanstate->ss, NULL, &TTSOpsMinimalTuple, 0);
/*
* initialize child expressions
diff --git a/src/backend/jit/llvm/llvmjit_deform.c b/src/backend/jit/llvm/llvmjit_deform.c
index 3eb087eb56b..12521e3e46a 100644
--- a/src/backend/jit/llvm/llvmjit_deform.c
+++ b/src/backend/jit/llvm/llvmjit_deform.c
@@ -62,7 +62,6 @@ slot_compile_deform(LLVMJitContext *context, TupleDesc desc,
LLVMValueRef v_tts_values;
LLVMValueRef v_tts_nulls;
LLVMValueRef v_slotoffp;
- LLVMValueRef v_flagsp;
LLVMValueRef v_nvalidp;
LLVMValueRef v_nvalid;
LLVMValueRef v_maxatt;
@@ -178,7 +177,6 @@ slot_compile_deform(LLVMJitContext *context, TupleDesc desc,
v_tts_nulls =
l_load_struct_gep(b, StructTupleTableSlot, v_slot, FIELDNO_TUPLETABLESLOT_ISNULL,
"tts_ISNULL");
- v_flagsp = l_struct_gep(b, StructTupleTableSlot, v_slot, FIELDNO_TUPLETABLESLOT_FLAGS, "");
v_nvalidp = l_struct_gep(b, StructTupleTableSlot, v_slot, FIELDNO_TUPLETABLESLOT_NVALID, "");
if (ops == &TTSOpsHeapTuple || ops == &TTSOpsBufferHeapTuple)
@@ -747,14 +745,10 @@ slot_compile_deform(LLVMJitContext *context, TupleDesc desc,
{
LLVMValueRef v_off = l_load(b, TypeSizeT, v_offp, "");
- LLVMValueRef v_flags;
LLVMBuildStore(b, l_int16_const(lc, natts), v_nvalidp);
v_off = LLVMBuildTrunc(b, v_off, LLVMInt32TypeInContext(lc), "");
LLVMBuildStore(b, v_off, v_slotoffp);
- v_flags = l_load(b, LLVMInt16TypeInContext(lc), v_flagsp, "tts_flags");
- v_flags = LLVMBuildOr(b, v_flags, l_int16_const(lc, TTS_FLAG_SLOW), "");
- LLVMBuildStore(b, v_flags, v_flagsp);
LLVMBuildRetVoid(b);
}
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 857ebf7d6fb..4ecfcbff7ab 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -1559,7 +1559,7 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
if (relentry->attrmap)
{
TupleTableSlot *slot = MakeTupleTableSlot(RelationGetDescr(targetrel),
- &TTSOpsVirtual);
+ &TTSOpsVirtual, 0);
old_slot = execute_attr_map_slot(relentry->attrmap, old_slot, slot);
}
@@ -1574,7 +1574,7 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
if (relentry->attrmap)
{
TupleTableSlot *slot = MakeTupleTableSlot(RelationGetDescr(targetrel),
- &TTSOpsVirtual);
+ &TTSOpsVirtual, 0);
new_slot = execute_attr_map_slot(relentry->attrmap, new_slot, slot);
}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index d27ac216e6d..597de687b45 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -666,14 +666,6 @@ RelationBuildTupleDesc(Relation relation)
elog(ERROR, "pg_attribute catalog is missing %d attribute(s) for relation OID %u",
need, RelationGetRelid(relation));
- /*
- * We can easily set the attcacheoff value for the first attribute: it
- * must be zero. This eliminates the need for special cases for attnum=1
- * that used to exist in fastgetattr() and index_getattr().
- */
- if (RelationGetNumberOfAttributes(relation) > 0)
- TupleDescCompactAttr(relation->rd_att, 0)->attcacheoff = 0;
-
/*
* Set up constraint/default info
*/
@@ -1985,8 +1977,6 @@ formrdesc(const char *relationName, Oid relationReltype,
populate_compact_attribute(relation->rd_att, i);
}
- /* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
- TupleDescCompactAttr(relation->rd_att, 0)->attcacheoff = 0;
TupleDescFinalize(relation->rd_att);
/* mark not-null status */
@@ -4446,8 +4436,6 @@ BuildHardcodedDescriptor(int natts, const FormData_pg_attribute *attrs)
populate_compact_attribute(result, i);
}
- /* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
- TupleDescCompactAttr(result, 0)->attcacheoff = 0;
TupleDescFinalize(result);
/* Note: we don't bother to set up a TupleConstr entry */
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 595413dbbc5..ad7bc013812 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -131,6 +131,19 @@ typedef struct CompactAttribute
* Any code making changes manually to and fields in the FormData_pg_attribute
* array must subsequently call populate_compact_attribute() to flush the
* changes out to the corresponding 'compact_attrs' element.
+ *
+ * firstNonCachedOffsetAttr stores the index into the compact_attrs array for
+ * the first attribute that we don't have a known attcacheoff for.
+ *
+ * firstNonGuaranteedAttr stores the index to into the compact_attrs array for
+ * the first attribute that is either NULLable, missing, or !attbyval. This
+ * can be used in locations as a guarantee that attributes before this will
+ * always exist in tuples. The !attbyval part isn't required for this, but
+ * including this allows various tuple deforming routines to forego any checks
+ * for !attbyval.
+ *
+ * Once a TupleDesc has been populated, before it is used for any purpose,
+ * TupleDescFinalize() must be called on it.
*/
typedef struct TupleDescData
{
@@ -138,6 +151,11 @@ typedef struct TupleDescData
Oid tdtypeid; /* composite type ID for tuple type */
int32 tdtypmod; /* typmod for tuple type */
int tdrefcount; /* reference count, or -1 if not counting */
+ int firstNonCachedOffsetAttr; /* index of the first att without
+ * an attcacheoff */
+ int firstNonGuaranteedAttr; /* index of the first nullable,
+ * missing, dropped, or !attbyval
+ * attribute. */
TupleConstr *constr; /* constraints, or NULL if none */
/* compact_attrs[N] is the compact metadata of Attribute Number N+1 */
CompactAttribute compact_attrs[FLEXIBLE_ARRAY_MEMBER];
@@ -195,7 +213,6 @@ extern TupleDesc CreateTupleDescTruncatedCopy(TupleDesc tupdesc, int natts);
extern TupleDesc CreateTupleDescCopyConstr(TupleDesc tupdesc);
-#define TupleDescFinalize(d) ((void) 0)
#define TupleDescSize(src) \
(offsetof(struct TupleDescData, compact_attrs) + \
(src)->natts * sizeof(CompactAttribute) + \
@@ -206,6 +223,7 @@ extern void TupleDescCopy(TupleDesc dst, TupleDesc src);
extern void TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
TupleDesc src, AttrNumber srcAttno);
+extern void TupleDescFinalize(TupleDesc tupdesc);
extern void FreeTupleDesc(TupleDesc tupdesc);
extern void IncrTupleDescRefCount(TupleDesc tupdesc);
diff --git a/src/include/access/tupmacs.h b/src/include/access/tupmacs.h
index d64c18b950b..87dbeb76618 100644
--- a/src/include/access/tupmacs.h
+++ b/src/include/access/tupmacs.h
@@ -15,7 +15,9 @@
#define TUPMACS_H
#include "catalog/pg_type_d.h" /* for TYPALIGN macros */
-
+#include "port/pg_bitutils.h"
+#include "port/pg_bswap.h"
+#include "varatt.h"
/*
* Check a tuple's null bitmap to determine whether the attribute is null.
@@ -28,6 +30,62 @@ att_isnull(int ATT, const bits8 *BITS)
return !(BITS[ATT >> 3] & (1 << (ATT & 0x07)));
}
+/*
+ * populate_isnull_array
+ * Transform a tuple's null bitmap into a boolean array.
+ *
+ * Caller must ensure that the isnull array is sized so it contains
+ * at least as many elements as there are bits in the 'bits' array.
+ * Callers should be aware that isnull is populated 8 elements at a time,
+ * effectively as if natts is rounded up to the next multiple of 8.
+ */
+static inline void
+populate_isnull_array(const bits8 *bits, int natts, bool *isnull)
+{
+ int nbytes = (natts + 7) >> 3;
+
+ /*
+ * Multiplying the inverted NULL bitmap byte by this value results in the
+ * lowest bit in each byte being set the same as each bit of the inverted
+ * byte. We perform this as 2 32-bit operations rather than a single
+ * 64-bit operation as multiplying by the required value to do this in
+ * 64-bits would result in overflowing a uint64 in some cases.
+ *
+ * XXX if we ever require BMI2 (-march=x86-64-v3), then this could be done
+ * more efficiently on most X86-64 CPUs with the PDEP instruction. Beware
+ * that some chips (e.g. AMD's Zen2) are horribly inefficient at PDEP.
+ */
+#define SPREAD_BITS_MULTIPLIER_32 0x204081U
+
+ for (int i = 0; i < nbytes; i++, isnull += 8)
+ {
+ uint64 isnull_8;
+ bits8 nullbyte = ~bits[i];
+
+ /* Convert the lower 4 bits of NULL bitmap word into a 64 bit int */
+ isnull_8 = (nullbyte & 0xf) * SPREAD_BITS_MULTIPLIER_32;
+
+ /*
+ * Convert the upper 4 bits of null bitmap word into a 64 bit int,
+ * shift into the upper 32 bit and bitwise-OR with the result of the
+ * lower 4 bits.
+ */
+ isnull_8 |= ((uint64) ((nullbyte >> 4) * SPREAD_BITS_MULTIPLIER_32)) << 32;
+
+ /* Mask out all other bits apart from the lowest bit of each byte. */
+ isnull_8 &= UINT64CONST(0x0101010101010101);
+
+#ifdef WORDS_BIGENDIAN
+
+ /*
+ * Fix byte order on big-endian machines before copying to the array.
+ */
+ isnull_8 = pg_bswap64(isnull_8);
+#endif
+ memcpy(isnull, &isnull_8, sizeof(uint64));
+ }
+}
+
#ifndef FRONTEND
/*
* Given an attbyval and an attlen from either a Form_pg_attribute or
@@ -69,6 +127,170 @@ fetch_att(const void *T, bool attbyval, int attlen)
else
return PointerGetDatum(T);
}
+
+/*
+ * Same, but no error checking for invalid attlens for byval types. This
+ * is safe to use when attlen comes from CompactAttribute as we validate the
+ * length when populating that struct.
+ */
+static inline Datum
+fetch_att_noerr(const void *T, bool attbyval, int attlen)
+{
+ if (attbyval)
+ {
+ switch (attlen)
+ {
+ case sizeof(int32):
+ return Int32GetDatum(*((const int32 *) T));
+ case sizeof(int16):
+ return Int16GetDatum(*((const int16 *) T));
+ case sizeof(char):
+ return CharGetDatum(*((const char *) T));
+ default:
+ Assert(attlen == sizeof(int64));
+ return Int64GetDatum(*((const int64 *) T));
+ }
+ }
+ else
+ return PointerGetDatum(T);
+}
+
+
+/*
+ * align_fetch_then_add
+ * Applies all the functionality of att_pointer_alignby(),
+ * fetch_att_noerr() and att_addlength_pointer(), resulting in the *off
+ * pointer to the perhaps unaligned number of bytes into 'tupptr', ready
+ * to deform the next attribute.
+ *
+ * tupptr: pointer to the beginning of the tuple, after the header and any
+ * NULL bitmask.
+ * off: offset in bytes for reading tuple data, possibly unaligned.
+ * attbyval, attlen and attalignby are values from CompactAttribute.
+ */
+static inline Datum
+align_fetch_then_add(const char *tupptr, uint32 *off, bool attbyval, int attlen,
+ uint8 attalignby)
+{
+ Datum res;
+
+ if (attlen > 0)
+ {
+ const char *offset_ptr;
+
+ *off = TYPEALIGN(attalignby, *off);
+ offset_ptr = tupptr + *off;
+ *off += attlen;
+ if (attbyval)
+ {
+ switch (attlen)
+ {
+ case sizeof(char):
+ return CharGetDatum(*((const char *) offset_ptr));
+ case sizeof(int16):
+ return Int16GetDatum(*((const int16 *) offset_ptr));
+ case sizeof(int32):
+ return Int32GetDatum(*((const int32 *) offset_ptr));
+ default:
+
+ /*
+ * populate_compact_attribute_internal() should have
+ * checked
+ */
+ Assert(attlen == sizeof(int64));
+ return Int64GetDatum(*((const int64 *) offset_ptr));
+ }
+ }
+ return PointerGetDatum(offset_ptr);
+ }
+ else if (attlen == -1)
+ {
+ if (!VARATT_IS_SHORT(tupptr + *off))
+ *off = TYPEALIGN(attalignby, *off);
+
+ res = PointerGetDatum(tupptr + *off);
+ *off += VARSIZE_ANY(DatumGetPointer(res));
+ return res;
+ }
+ else
+ {
+ Assert(attlen == -2);
+ *off = TYPEALIGN(attalignby, *off);
+ res = PointerGetDatum(tupptr + *off);
+ *off += strlen(tupptr + *off) + 1;
+ return res;
+ }
+}
+
+/*
+ * first_null_attr
+ * Inspect a NULL bitmap from a tuple and return the 0-based attnum of the
+ * first NULL attribute. Returns natts if no NULLs were found.
+ *
+ * This is coded to expect that 'bits' contains at least one 0 bit somewhere
+ * in the array, but not necessarily < natts. Note that natts may be passed
+ * as a value lower than the number of bits physically stored in the tuple's
+ * NULL bitmap, in which case we may not find a NULL and return natts.
+ *
+ * The reason we require at least one 0 bit somewhere in the NULL bitmap is
+ * that the for loop that checks 0xFF bytes would loop to the last byte in
+ * the array if all bytes were 0xFF, and the subsequent code that finds the
+ * right-most 0 bit would access the first byte beyond the bitmap. Provided
+ * we find a 0 bit before then, that won't happen. Since tuples which have no
+ * NULLs don't have a NULL bitmap, this function won't get called for that
+ * case.
+ */
+static inline int
+first_null_attr(const bits8 *bits, int natts)
+{
+ int nattByte = natts >> 3;
+ int bytenum;
+ int res;
+
+#ifdef USE_ASSERT_CHECKING
+ int firstnull_check = natts;
+
+ /* Do it the slow way and check we get the same answer. */
+ for (int i = 0; i < natts; i++)
+ {
+ if (att_isnull(i, bits))
+ {
+ firstnull_check = i;
+ break;
+ }
+ }
+#endif
+
+ /* Process all bytes up to just before the byte for the natts attribute */
+ for (bytenum = 0; bytenum < nattByte; bytenum++)
+ {
+ /* break if there's any NULL attrs (a 0 bit) */
+ if (bits[bytenum] != 0xFF)
+ break;
+ }
+
+ /*
+ * Look for the highest 0-bit in the 'bytenum' element. To do this, we
+ * promote the uint8 to uint32 before performing the bitwise NOT and
+ * looking for the first 1-bit. This works even when the byte is 0xFF, as
+ * the bitwise NOT of 0xFF in 32 bits is 0xFFFFFF00, in which case
+ * pg_rightmost_one_pos32() will return 8. We may end up with a value
+ * higher than natts here, but we'll fix that with the Min() below.
+ */
+ res = bytenum << 3;
+ res += pg_rightmost_one_pos32(~((uint32) bits[bytenum]));
+
+ /*
+ * Since we did no masking to mask out bits beyond the natt'th bit, we may
+ * have found a bit higher than natts, so we must cap res to natts
+ */
+ res = Min(res, natts);
+
+ /* Ensure we got the same answer as the att_isnull() loop got */
+ Assert(res == firstnull_check);
+
+ return res;
+}
#endif /* FRONTEND */
/*
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index d46ba59895d..bf239fc156f 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -595,7 +595,8 @@ extern void ExecInitResultTupleSlotTL(PlanState *planstate,
const TupleTableSlotOps *tts_ops);
extern void ExecInitScanTupleSlot(EState *estate, ScanState *scanstate,
TupleDesc tupledesc,
- const TupleTableSlotOps *tts_ops);
+ const TupleTableSlotOps *tts_ops,
+ uint16 flags);
extern TupleTableSlot *ExecInitExtraTupleSlot(EState *estate,
TupleDesc tupledesc,
const TupleTableSlotOps *tts_ops);
diff --git a/src/include/executor/tuptable.h b/src/include/executor/tuptable.h
index 3b09abbf99f..78558098fa3 100644
--- a/src/include/executor/tuptable.h
+++ b/src/include/executor/tuptable.h
@@ -84,9 +84,6 @@
* tts_values/tts_isnull are allocated either when the slot is created (when
* the descriptor is provided), or when a descriptor is assigned to the slot;
* they are of length equal to the descriptor's natts.
- *
- * The TTS_FLAG_SLOW flag is saved state for
- * slot_deform_heap_tuple, and should not be touched by any other code.
*----------
*/
@@ -98,9 +95,13 @@
#define TTS_FLAG_SHOULDFREE (1 << 2)
#define TTS_SHOULDFREE(slot) (((slot)->tts_flags & TTS_FLAG_SHOULDFREE) != 0)
-/* saved state for slot_deform_heap_tuple */
-#define TTS_FLAG_SLOW (1 << 3)
-#define TTS_SLOW(slot) (((slot)->tts_flags & TTS_FLAG_SLOW) != 0)
+/*
+ * true = slot's formed tuple guaranteed to not have NULLs in NOT NULLable
+ * columns.
+ */
+#define TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS (1 << 3)
+#define TTS_OBEYS_NOT_NULL_CONSTRAINTS(slot) \
+ (((slot)->tts_flags & TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS) != 0)
/* fixed tuple descriptor */
#define TTS_FLAG_FIXED (1 << 4)
@@ -123,7 +124,14 @@ typedef struct TupleTableSlot
#define FIELDNO_TUPLETABLESLOT_VALUES 5
Datum *tts_values; /* current per-attribute values */
#define FIELDNO_TUPLETABLESLOT_ISNULL 6
- bool *tts_isnull; /* current per-attribute isnull flags */
+ bool *tts_isnull; /* current per-attribute isnull flags. Array
+ * size must always be rounded up to the next
+ * multiple of 8 elements. */
+ int tts_first_nonguaranteed; /* The value from the TupleDesc's
+ * firstNonGuaranteedAttr, or 0
+ * when tts_flags does not contain
+ * TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS */
+
MemoryContext tts_mcxt; /* slot itself is in this context */
ItemPointerData tts_tid; /* stored tuple's tid */
Oid tts_tableOid; /* table oid of tuple */
@@ -313,9 +321,11 @@ typedef struct MinimalTupleTableSlot
/* in executor/execTuples.c */
extern TupleTableSlot *MakeTupleTableSlot(TupleDesc tupleDesc,
- const TupleTableSlotOps *tts_ops);
+ const TupleTableSlotOps *tts_ops,
+ uint16 flags);
extern TupleTableSlot *ExecAllocTableSlot(List **tupleTable, TupleDesc desc,
- const TupleTableSlotOps *tts_ops);
+ const TupleTableSlotOps *tts_ops,
+ uint16 flags);
extern void ExecResetTupleTable(List *tupleTable, bool shouldFree);
extern TupleTableSlot *MakeSingleTupleTableSlot(TupleDesc tupdesc,
const TupleTableSlotOps *tts_ops);
diff --git a/src/test/modules/deform_bench/deform_bench.c b/src/test/modules/deform_bench/deform_bench.c
index 7838f639bef..4f104989297 100644
--- a/src/test/modules/deform_bench/deform_bench.c
+++ b/src/test/modules/deform_bench/deform_bench.c
@@ -48,7 +48,9 @@ deform_bench(PG_FUNCTION_ARGS)
errmsg("only heap AM is supported")));
tupdesc = RelationGetDescr(rel);
- slot = MakeTupleTableSlot(tupdesc, &TTSOpsBufferHeapTuple);
+ slot = MakeTupleTableSlot(tupdesc,
+ &TTSOpsBufferHeapTuple,
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS);
scan = table_beginscan_strat(rel, GetActiveSnapshot(), 0, NULL, true, false);
/*
--
2.51.0
[text/plain] v12-0005-Reduce-size-of-CompactAttribute-struct-to-8-byte.patch (5.7K, 6-v12-0005-Reduce-size-of-CompactAttribute-struct-to-8-byte.patch)
download | inline diff:
From 8e1a945f172e3c6e9cc0092e5c2168151ed8c6d1 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Mon, 23 Feb 2026 09:39:37 +1300
Subject: [PATCH v12 5/6] Reduce size of CompactAttribute struct to 8 bytes
Previously, this was 16 bytes. With the use of some bitflags and by
reducing the attcacheoff field size to a 16-bit type, we can halve the
size of the struct.
It's unlikely that caching the offsets for offsets larger than what will
fit in a 16-bit int will help much as the tuple is very likely to have
some non-fixed-width types anyway, the offsets of which we cannot cache.
---
src/backend/access/common/tupdesc.c | 10 ++++++++++
src/backend/executor/execTuples.c | 17 ++++++++++++-----
src/include/access/tupdesc.h | 16 ++++++++--------
3 files changed, 30 insertions(+), 13 deletions(-)
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index c68561337d7..71461ba6096 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -530,6 +530,16 @@ TupleDescFinalize(TupleDesc tupdesc)
off = att_nominal_alignby(off, cattr->attalignby);
+ /*
+ * attcacheoff is an int16, so don't try to cache any offsets larger
+ * than will fit in that type. Any attributes which are offset more
+ * than 2^15 are likely due to variable-length attributes. Since we
+ * don't cache offsets for or beyond variable-length attributes, using
+ * an int16 rather than an int32 here is unlikely to cost us anything.
+ */
+ if (off > PG_INT16_MAX)
+ break;
+
cattr->attcacheoff = off;
off += cattr->attlen;
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 14b2a9f0af6..1b5e1ebd334 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -1013,6 +1013,7 @@ static pg_attribute_always_inline void
slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int reqnatts)
{
+ CompactAttribute *cattrs;
CompactAttribute *cattr;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
HeapTupleHeader tup = tuple->t_data;
@@ -1099,6 +1100,13 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
values = slot->tts_values;
slot->tts_nvalid = reqnatts;
+ /*
+ * We store the tupleDesc's CompactAttribute array in 'cattrs' as gcc
+ * seems to be unwilling to optimize accessing the CompactAttribute
+ * element efficiently when accessing it via TupleDescCompactAttr().
+ */
+ cattrs = tupleDesc->compact_attrs;
+
/* Ensure we calculated tp correctly */
Assert(tp == (char *) tup + tup->t_hoff);
@@ -1109,7 +1117,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
for (; attnum < firstNonGuaranteedAttr; attnum++)
{
isnull[attnum] = false;
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
attlen = cattr->attlen;
/* We don't expect any non-byval types */
@@ -1149,9 +1157,8 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
for (; attnum < firstNonCacheOffsetAttr; attnum++)
{
isnull[attnum] = false;
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
attlen = cattr->attlen;
-
off = cattr->attcacheoff;
values[attnum] = fetch_att_noerr(tp + off,
cattr->attbyval,
@@ -1177,7 +1184,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int attlen;
isnull[attnum] = false;
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
attlen = cattr->attlen;
/*
@@ -1210,7 +1217,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
continue;
}
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
attlen = cattr->attlen;
/* As above, we don't expect cstrings */
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index ad7bc013812..e98036b58bf 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -55,7 +55,7 @@ typedef struct TupleConstr
* directly after the FormData_pg_attribute struct is populated or
* altered in any way.
*
- * Currently, this struct is 16 bytes. Any code changes which enlarge this
+ * Currently, this struct is 8 bytes. Any code changes which enlarge this
* struct should be considered very carefully.
*
* Code which must access a TupleDesc's attribute data should always make use
@@ -67,17 +67,17 @@ typedef struct TupleConstr
*/
typedef struct CompactAttribute
{
- int32 attcacheoff; /* fixed offset into tuple, if known, or -1 */
+ int16 attcacheoff; /* fixed offset into tuple, if known, or -1 */
int16 attlen; /* attr len in bytes or -1 = varlen, -2 =
* cstring */
bool attbyval; /* as FormData_pg_attribute.attbyval */
- bool attispackable; /* FormData_pg_attribute.attstorage !=
- * TYPSTORAGE_PLAIN */
- bool atthasmissing; /* as FormData_pg_attribute.atthasmissing */
- bool attisdropped; /* as FormData_pg_attribute.attisdropped */
- bool attgenerated; /* FormData_pg_attribute.attgenerated != '\0' */
- char attnullability; /* status of not-null constraint, see below */
uint8 attalignby; /* alignment requirement in bytes */
+ bool attispackable:1; /* FormData_pg_attribute.attstorage !=
+ * TYPSTORAGE_PLAIN */
+ bool atthasmissing:1; /* as FormData_pg_attribute.atthasmissing */
+ bool attisdropped:1; /* as FormData_pg_attribute.attisdropped */
+ bool attgenerated:1; /* FormData_pg_attribute.attgenerated != '\0' */
+ char attnullability; /* status of not-null constraint, see below */
} CompactAttribute;
/* Valid values for CompactAttribute->attnullability */
--
2.51.0
[text/plain] v12-0006-WIP-Introduce-selective-tuple-deforming.patch (42.0K, 7-v12-0006-WIP-Introduce-selective-tuple-deforming.patch)
download | inline diff:
From 894745116658acec4a5a53657af465255cca7068 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Wed, 4 Mar 2026 16:55:09 +1300
Subject: [PATCH v12 6/6] WIP: Introduce selective tuple deforming
Up until now, we have always deformed every attribute of each tuple up
until the last attribute that we require. This did once make sense to
do as we often had to walk the tuple in order to determine the byte
offset to any attribute that we deform. Now, since we proactively
populate the CompactAttribute.attcacheoff, in some cases we might be in
a better position to only deform the attributes that we need to deform
by directly jumping to the cached byte offset and skipping deforming any
attributes which are not required. In some cases the savings can be
very large as it not only allows tts_values and tts_isnull for unneeded
attributes to be left unpopulated, but it might also mean we can skip
loading entire cachelines when the tuple is wide enough to span multiple
cachelines.
We don't want to exclusively always deform tuples this way as doing this
means paying attention to an additional array which states which attnums
we must deform. Looking at that array for a SELECT * query, which
requires us to deform all attributes, would add overhead. To support
this a new expression evaluation operator has been added called
EEOP_SCAN_SELECTSOME and each function which builds an ExprState now
accepts a variant function that allows the caller to specify which attnums
are required from the scan side. This puts it on the caller to decide
which type of deforming should be done. When the caller provides
the attnums, the expression will be built with EEOP_SCAN_SELECTSOME
rather than EEOP_SCAN_FETCHSOME. This currently does not interact well
with the physical tlist optimization. Currently it's the planner's job
to figure out which attributes are actually required.
TODO: JIT support
---
src/backend/executor/execExpr.c | 166 ++++++++++-
src/backend/executor/execExprInterp.c | 13 +
src/backend/executor/execScan.c | 18 ++
src/backend/executor/execTuples.c | 362 +++++++++++++++++++++++-
src/backend/executor/execUtils.c | 47 ++-
src/backend/executor/nodeSeqscan.c | 8 +-
src/backend/optimizer/plan/createplan.c | 43 ++-
src/include/executor/execExpr.h | 24 +-
src/include/executor/executor.h | 19 ++
src/include/executor/tuptable.h | 22 ++
src/include/nodes/plannodes.h | 8 +
11 files changed, 702 insertions(+), 28 deletions(-)
diff --git a/src/backend/executor/execExpr.c b/src/backend/executor/execExpr.c
index 088eca24021..8a2bf04598b 100644
--- a/src/backend/executor/execExpr.c
+++ b/src/backend/executor/execExpr.c
@@ -66,6 +66,14 @@ typedef struct ExprSetupInfo
AttrNumber last_new;
/* MULTIEXPR SubPlan nodes appearing in the expression: */
List *multiexpr_subplans;
+
+ /*
+ * Fetch only these attnums from the scan with EEOP_SCAN_SELECTSOME. Empty
+ * set means use EEOP_SCAN_FETCHSOME (i.e fetch all up until last_scan).
+ * The first user attribute is based at member 0. System attributes not
+ * represented.
+ */
+ Bitmapset *scan_attrs;
} ExprSetupInfo;
static void ExecReadyExpr(ExprState *state);
@@ -77,7 +85,8 @@ static void ExecInitFunc(ExprEvalStep *scratch, Expr *node, List *args,
static void ExecInitSubPlanExpr(SubPlan *subplan,
ExprState *state,
Datum *resv, bool *resnull);
-static void ExecCreateExprSetupSteps(ExprState *state, Node *node);
+static void ExecCreateExprSetupSteps(ExprState *state, Node *node,
+ Bitmapset *scan_attrs);
static void ExecPushExprSetupSteps(ExprState *state, ExprSetupInfo *info);
static bool expr_setup_walker(Node *node, ExprSetupInfo *info);
static bool ExecComputeSlotInfo(ExprState *state, ExprEvalStep *op);
@@ -141,6 +150,19 @@ static void ExecInitJsonCoercion(ExprState *state, JsonReturning *returning,
*/
ExprState *
ExecInitExpr(Expr *node, PlanState *parent)
+{
+ return ExecInitExprWithScanAttrs(node, parent, NULL);
+}
+
+/*
+ * ExecInitExprWithScanAttrs
+ * As ExecInitExpr but when 'scan_attrs' is set, use
+ * EEOP_SCAN_SELECTSOME instead of EEOP_SCAN_FETCHSOME to deform only
+ * the mentioned 'scan_attrs' from the scan tuple.
+ */
+ExprState *
+ExecInitExprWithScanAttrs(Expr *node, PlanState *parent,
+ Bitmapset *scan_attrs)
{
ExprState *state;
ExprEvalStep scratch = {0};
@@ -156,7 +178,7 @@ ExecInitExpr(Expr *node, PlanState *parent)
state->ext_params = NULL;
/* Insert setup steps as needed */
- ExecCreateExprSetupSteps(state, (Node *) node);
+ ExecCreateExprSetupSteps(state, (Node *) node, scan_attrs);
/* Compile the expression proper */
ExecInitExprRec(node, state, &state->resvalue, &state->resnull);
@@ -193,7 +215,7 @@ ExecInitExprWithParams(Expr *node, ParamListInfo ext_params)
state->ext_params = ext_params;
/* Insert setup steps as needed */
- ExecCreateExprSetupSteps(state, (Node *) node);
+ ExecCreateExprSetupSteps(state, (Node *) node, NULL);
/* Compile the expression proper */
ExecInitExprRec(node, state, &state->resvalue, &state->resnull);
@@ -227,6 +249,19 @@ ExecInitExprWithParams(Expr *node, ParamListInfo ext_params)
*/
ExprState *
ExecInitQual(List *qual, PlanState *parent)
+{
+ return ExecInitQualWithScanAttrs(qual, parent, NULL);
+}
+
+/*
+ * ExecInitQualWithScanAttrs
+ * As ExecInitQual but when 'scan_attrs' is set, use
+ * EEOP_SCAN_SELECTSOME instead of EEOP_SCAN_FETCHSOME to deform only
+ * the mentioned 'scan_attrs' from the scan tuple.
+ */
+ExprState *
+ExecInitQualWithScanAttrs(List *qual, PlanState *parent,
+ Bitmapset *scan_attrs)
{
ExprState *state;
ExprEvalStep scratch = {0};
@@ -247,7 +282,7 @@ ExecInitQual(List *qual, PlanState *parent)
state->flags = EEO_FLAG_IS_QUAL;
/* Insert setup steps as needed */
- ExecCreateExprSetupSteps(state, (Node *) qual);
+ ExecCreateExprSetupSteps(state, (Node *) qual, scan_attrs);
/*
* ExecQual() needs to return false for an expression returning NULL. That
@@ -372,6 +407,28 @@ ExecBuildProjectionInfo(List *targetList,
TupleTableSlot *slot,
PlanState *parent,
TupleDesc inputDesc)
+{
+ return ExecBuildProjectionInfoWithScanAttrs(targetList,
+ econtext,
+ slot,
+ parent,
+ inputDesc,
+ NULL);
+}
+
+/*
+ * ExecBuildProjectionInfoWithScanAttrs
+ * As ExecBuildProjectionInfo but when 'scan_attrs' is set, use
+ * EEOP_SCAN_SELECTSOME instead of EEOP_SCAN_FETCHSOME to deform only
+ * the mentioned 'scan_attrs' from the scan tuple.
+ */
+ProjectionInfo *
+ExecBuildProjectionInfoWithScanAttrs(List *targetList,
+ ExprContext *econtext,
+ TupleTableSlot *slot,
+ PlanState *parent,
+ TupleDesc inputDesc,
+ Bitmapset *scan_attrs)
{
ProjectionInfo *projInfo = makeNode(ProjectionInfo);
ExprState *state;
@@ -389,7 +446,7 @@ ExecBuildProjectionInfo(List *targetList,
state->resultslot = slot;
/* Insert setup steps as needed */
- ExecCreateExprSetupSteps(state, (Node *) targetList);
+ ExecCreateExprSetupSteps(state, (Node *) targetList, scan_attrs);
/* Now compile each tlist column */
foreach(lc, targetList)
@@ -2871,11 +2928,19 @@ ExecInitSubPlanExpr(SubPlan *subplan,
/*
* Add expression steps performing setup that's needed before any of the
* main execution of the expression.
+ *
+ * 'scan_attrs' may be given an empty set, in which case deforming the scan
+ * tuple is done via EEOP_SCAN_FETCHSOME, which fetches every attribute from
+ * the scan tuple up until the maximum attribute used by this expression.
+ * When 'scan_attrs' is set, EEOP_SCAN_SELECTSOME is used to only fetch the
+ * attributes mentioned. Callers must create a unioned set of the attributes
+ * needed from the scan for all expressions using the given slot so that we
+ * incrementally fetch the attributes required by all ExprStates.
*/
static void
-ExecCreateExprSetupSteps(ExprState *state, Node *node)
+ExecCreateExprSetupSteps(ExprState *state, Node *node, Bitmapset *scan_attrs)
{
- ExprSetupInfo info = {0, 0, 0, 0, 0, NIL};
+ ExprSetupInfo info = {0, 0, 0, 0, 0, NIL, scan_attrs};
/* Prescan to find out what we need. */
expr_setup_walker(node, &info);
@@ -2923,11 +2988,75 @@ ExecPushExprSetupSteps(ExprState *state, ExprSetupInfo *info)
}
if (info->last_scan > 0)
{
- scratch.opcode = EEOP_SCAN_FETCHSOME;
- scratch.d.fetch.last_var = info->last_scan;
- scratch.d.fetch.fixed = false;
- scratch.d.fetch.kind = NULL;
- scratch.d.fetch.known_desc = NULL;
+ /*
+ * We have two operators for fetching attributes out of a tuple during
+ * scans. EEOP_SCAN_FETCHSOME deforms all attributes in the tuple up
+ * to the 'last_scan' attnum. This isn't ideal in some cases, as we
+ * may only need a few attributes, and those might be deep into the
+ * tuple. EEOP_SCAN_SELECTSOME is an operator that fetches only the
+ * required attributes from the tuple. When the attcacheoff for these
+ * attributes is known and no NULLs exist in the tuple prior to the
+ * required attributes, then this can be a very fast operation.
+ * EEOP_SCAN_FETCHSOME is still supported as many cases require all
+ * attributes, and EEOP_SCAN_FETCHSOME can do this more efficiently.
+ */
+ if (bms_is_empty(info->scan_attrs))
+ {
+ scratch.opcode = EEOP_SCAN_FETCHSOME;
+ scratch.d.fetch.last_var = info->last_scan;
+ scratch.d.fetch.fixed = false;
+ scratch.d.fetch.kind = NULL;
+ scratch.d.fetch.known_desc = NULL;
+ }
+ else
+ {
+ int nattrs = bms_num_members(info->scan_attrs);
+ AttrNumber *atts;
+ int a;
+ int i;
+
+ scratch.opcode = EEOP_SCAN_SELECTSOME;
+ scratch.d.fetch.last_var = info->last_scan;
+ scratch.d.fetch.fixed = false;
+ scratch.d.fetch.natts = nattrs;
+ scratch.d.fetch.kind = NULL;
+ scratch.d.fetch.known_desc = NULL;
+
+ /*
+ * Allocate these two arrays as a single allocation. The
+ * req_attnums array needs 1 element for each attnum that's being
+ * selected, plus a sentinel attnum which we set to the
+ * 'last_scan' attnum so that we correctly terminate each of the
+ * loops during selective deformation before walking off the end
+ * of the array.
+ */
+ atts = palloc_array(AttrNumber, nattrs + 1 + info->last_scan + 1);
+
+ scratch.d.fetch.req_attnums = atts;
+ scratch.d.fetch.next_req_attnums_index = &atts[nattrs + 1];
+
+ /* Store each attnum in the Bitmapset into the req_attnum array */
+ a = -1;
+ i = 0;
+ while ((a = bms_next_member(info->scan_attrs, a)) >= 0)
+ scratch.d.fetch.req_attnums[i++] = a;
+
+ /* install sentinel */
+ scratch.d.fetch.req_attnums[nattrs] = info->last_scan;
+
+ /*
+ * Populate the next_req_attnums_index array. This allows the
+ * deforming function to refind the position in the
+ * next_req_attnums_index array from tts_nvalid.
+ */
+ a = 0;
+ for (i = 0; i <= info->last_scan; i++)
+ {
+ scratch.d.fetch.next_req_attnums_index[i] = a;
+ if (bms_is_member(i, info->scan_attrs))
+ a++;
+ }
+ }
if (ExecComputeSlotInfo(state, &scratch))
ExprEvalPushStep(state, &scratch);
}
@@ -3000,6 +3129,13 @@ expr_setup_walker(Node *node, ExprSetupInfo *info)
switch (variable->varreturningtype)
{
case VAR_RETURNING_DEFAULT:
+
+ /*
+ * scan_attrs must contain a member for this attnum or
+ * be completely empty
+ */
+ Assert(attnum < 0 || bms_is_empty(info->scan_attrs) ||
+ bms_is_member(attnum - 1, info->scan_attrs));
info->last_scan = Max(info->last_scan, attnum);
break;
case VAR_RETURNING_OLD:
@@ -3066,7 +3202,8 @@ ExecComputeSlotInfo(ExprState *state, ExprEvalStep *op)
opcode == EEOP_OUTER_FETCHSOME ||
opcode == EEOP_SCAN_FETCHSOME ||
opcode == EEOP_OLD_FETCHSOME ||
- opcode == EEOP_NEW_FETCHSOME);
+ opcode == EEOP_NEW_FETCHSOME ||
+ opcode == EEOP_SCAN_SELECTSOME);
if (op->d.fetch.known_desc != NULL)
{
@@ -3119,6 +3256,7 @@ ExecComputeSlotInfo(ExprState *state, ExprEvalStep *op)
}
}
else if (opcode == EEOP_SCAN_FETCHSOME ||
+ opcode == EEOP_SCAN_SELECTSOME ||
opcode == EEOP_OLD_FETCHSOME ||
opcode == EEOP_NEW_FETCHSOME)
{
@@ -4311,7 +4449,7 @@ ExecBuildHash32Expr(TupleDesc desc, const TupleTableSlotOps *ops,
state->parent = parent;
/* Insert setup steps as needed. */
- ExecCreateExprSetupSteps(state, (Node *) hash_exprs);
+ ExecCreateExprSetupSteps(state, (Node *) hash_exprs, NULL);
/*
* Make a place to store intermediate hash values between subsequent
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 61ff5ddc74c..76965826f83 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -479,6 +479,7 @@ ExecInterpExpr(ExprState *state, ExprContext *econtext, bool *isnull)
&&CASE_EEOP_SCAN_FETCHSOME,
&&CASE_EEOP_OLD_FETCHSOME,
&&CASE_EEOP_NEW_FETCHSOME,
+ &&CASE_EEOP_SCAN_SELECTSOME,
&&CASE_EEOP_INNER_VAR,
&&CASE_EEOP_OUTER_VAR,
&&CASE_EEOP_SCAN_VAR,
@@ -676,6 +677,18 @@ ExecInterpExpr(ExprState *state, ExprContext *econtext, bool *isnull)
EEO_NEXT();
}
+ EEO_CASE(EEOP_SCAN_SELECTSOME)
+ {
+ CheckOpSlotCompatibility(op, scanslot);
+
+ slot_selectattrs(scanslot,
+ op->d.fetch.last_var,
+ op->d.fetch.req_attnums,
+ op->d.fetch.next_req_attnums_index);
+
+ EEO_NEXT();
+ }
+
EEO_CASE(EEOP_INNER_VAR)
{
int attnum = op->d.var.attnum;
diff --git a/src/backend/executor/execScan.c b/src/backend/executor/execScan.c
index 9f68be17b99..525af11aa08 100644
--- a/src/backend/executor/execScan.c
+++ b/src/backend/executor/execScan.c
@@ -86,6 +86,24 @@ ExecAssignScanProjectionInfo(ScanState *node)
ExecConditionalAssignProjectionInfo(&node->ps, tupdesc, scan->scanrelid);
}
+/*
+ * ExecAssignScanProjectionInfoWithScanAttrs
+ * As ExecAssignScanProjectionInfo but when 'scan_attrs' is set, use
+ * EEOP_SCAN_SELECTSOME instead of EEOP_SCAN_FETCHSOME to deform only
+ * the mentioned 'scan_attrs' from the scan tuple.
+ */
+void
+ExecAssignScanProjectionInfoWithScanAttrs(ScanState *node,
+ Bitmapset *scan_attrs)
+{
+ Scan *scan = (Scan *) node->ps.plan;
+ TupleDesc tupdesc = node->ss_ScanTupleSlot->tts_tupleDescriptor;
+
+ ExecConditionalAssignProjectionInfoWithScanAttrs(&node->ps, tupdesc,
+ scan->scanrelid,
+ scan_attrs);
+}
+
/*
* ExecAssignScanProjectionInfoWithVarno
* As above, but caller can specify varno expected in Vars in the tlist.
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 1b5e1ebd334..5e502bfdbb8 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -74,6 +74,12 @@ static TupleDesc ExecTypeFromTLInternal(List *targetList,
bool skipjunk);
static pg_attribute_always_inline void slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int reqnatts);
+static pg_attribute_always_inline void slot_selectively_deform_heap_tuple(TupleTableSlot *slot,
+ HeapTuple tuple,
+ uint32 *offp,
+ int last_attnum,
+ AttrNumber *attnums,
+ AttrNumber *attnum_map);
static inline void tts_buffer_heap_store_tuple(TupleTableSlot *slot,
HeapTuple tuple,
Buffer buffer,
@@ -129,7 +135,22 @@ tts_virtual_clear(TupleTableSlot *slot)
static void
tts_virtual_getsomeattrs(TupleTableSlot *slot, int natts)
{
- elog(ERROR, "getsomeattrs is not required to be called on a virtual tuple table slot");
+ elog(ERROR,
+ "%s is not required to be called on a virtual tuple table slot",
+ "getsomeattrs");
+}
+
+/*
+ * VirtualTupleTableSlots always have fully populated tts_values and
+ * tts_isnull arrays. So this function should never be called.
+ */
+static void
+tts_virtual_selectattrs(TupleTableSlot *slot, int last_attnum,
+ AttrNumber *attnums, AttrNumber *attnum_map)
+{
+ elog(ERROR,
+ "%s is not required to be called on a virtual tuple table slot",
+ "selectattrs");
}
/*
@@ -352,6 +373,22 @@ tts_heap_getsomeattrs(TupleTableSlot *slot, int natts)
slot_deform_heap_tuple(slot, hslot->tuple, &hslot->off, natts);
}
+static void
+tts_heap_selectattrs(TupleTableSlot *slot, int last_attnum,
+ AttrNumber *attnums, AttrNumber *attnum_map)
+{
+ HeapTupleTableSlot *hslot = (HeapTupleTableSlot *) slot;
+
+ Assert(!TTS_EMPTY(slot));
+
+ slot_selectively_deform_heap_tuple(slot,
+ hslot->tuple,
+ &hslot->off,
+ last_attnum,
+ attnums,
+ attnum_map);
+}
+
static Datum
tts_heap_getsysattr(TupleTableSlot *slot, int attnum, bool *isnull)
{
@@ -550,6 +587,22 @@ tts_minimal_getsomeattrs(TupleTableSlot *slot, int natts)
slot_deform_heap_tuple(slot, mslot->tuple, &mslot->off, natts);
}
+static void
+tts_minimal_selectattrs(TupleTableSlot *slot, int last_attnum,
+ AttrNumber *attnums, AttrNumber *attnum_map)
+{
+ MinimalTupleTableSlot *mslot = (MinimalTupleTableSlot *) slot;
+
+ Assert(!TTS_EMPTY(slot));
+
+ slot_selectively_deform_heap_tuple(slot,
+ mslot->tuple,
+ &mslot->off,
+ last_attnum,
+ attnums,
+ attnum_map);
+}
+
/*
* MinimalTupleTableSlots never provide system attributes. We generally
* shouldn't get here, but provide a user-friendly message if we do.
@@ -757,6 +810,23 @@ tts_buffer_heap_getsomeattrs(TupleTableSlot *slot, int natts)
slot_deform_heap_tuple(slot, bslot->base.tuple, &bslot->base.off, natts);
}
+static void
+tts_buffer_heap_selectattrs(TupleTableSlot *slot, int last_attnum,
+ AttrNumber *attnums, AttrNumber *attnum_map)
+{
+ BufferHeapTupleTableSlot *bslot = (BufferHeapTupleTableSlot *) slot;
+
+ Assert(!TTS_EMPTY(slot));
+
+ slot_selectively_deform_heap_tuple(slot,
+ bslot->base.tuple,
+ &bslot->base.off,
+ last_attnum,
+ attnums,
+ attnum_map);
+}
+
+
static Datum
tts_buffer_heap_getsysattr(TupleTableSlot *slot, int attnum, bool *isnull)
{
@@ -1244,12 +1314,299 @@ done:
*offp = off;
}
+/*
+ * slot_selectively_deform_heap_tuple
+ * Deform attributes of 'tuple' into the Datum/isnull arrays in 'slot'.
+ * Unlike slot_deform_heap_tuple, which deforms every attribute up to the
+ * given attribute number, here we deform only the attribute numbers
+ * mentioned in the 'attnums' array. When only a few attributes are
+ * required, this can be more efficient. When the attributes have a
+ * known attcacheoff and it's valid to use that, then this version can be
+ * much more efficient than slot_deform_heap_tuple when only a small
+ * number of the total attributes are required.
+ */
+static pg_attribute_always_inline void
+slot_selectively_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple,
+ uint32 *offp, int last_attnum,
+ AttrNumber *attnums,
+ AttrNumber *attnum_map)
+{
+ CompactAttribute *cattrs;
+ CompactAttribute *cattr;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+ HeapTupleHeader tup = tuple->t_data;
+ size_t attnum;
+ int attnums_idx;
+ int firstNonCacheOffsetAttr;
+ int firstNonGuaranteedAttr;
+ int firstNullAttr;
+ int natts;
+ Datum *values;
+ bool *isnull;
+ char *tp; /* ptr to tuple data */
+ uint32 off; /* offset in tuple data */
+ int off_attnum; /* the attnum that 'off' points to */
+
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
+
+ isnull = slot->tts_isnull;
+
+ /*
+ * Some callers may form and deform tuples prior to NOT NULL constraints
+ * being checked. Here we'd like to optimize the case where we only need
+ * to fetch attributes before or up to the point where the attribute is
+ * guaranteed to exist in the tuple. We rely on the slot flag being set
+ * correctly to only enable this optimization when it's valid to do so.
+ * This optimization allows us to save fetching the number of attributes
+ * from the tuple and saves the additional cost of handling non-byval
+ * attrs.
+ */
+ firstNonGuaranteedAttr = Min(last_attnum, slot->tts_first_nonguaranteed);
+
+ firstNonCacheOffsetAttr = tupleDesc->firstNonCachedOffsetAttr;
+
+ if (HeapTupleHasNulls(tuple))
+ {
+ natts = HeapTupleHeaderGetNatts(tup);
+ tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits) +
+ BITMAPLEN(natts));
+
+ natts = Min(natts, last_attnum);
+ if (natts > firstNonGuaranteedAttr)
+ {
+ bits8 *bp = tup->t_bits;
+
+ /* Find the first NULL attr */
+ firstNullAttr = first_null_attr(bp, natts);
+
+ /*
+ * And populate the isnull array for all attributes up to the last
+ * attribute we're deforming. When not using attcacheoff, we need
+ * to know if an attribute is NULL even when we're not deforming
+ * it, so that we can skip over it when calculating the offset to
+ * attributes that we are deforming.
+ */
+ populate_isnull_array(bp, natts, isnull);
+
+ /* We can only use any cached offsets until the first NULL attr */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr,
+ firstNullAttr);
+ }
+ else
+ {
+ /* Otherwise all required columns are guaranteed to exist */
+ firstNullAttr = natts;
+ }
+ }
+ else
+ {
+ tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits));
+
+ /*
+ * We only need to look at the tuple's natts if we need more than the
+ * guaranteed number of columns
+ */
+ if (last_attnum > firstNonGuaranteedAttr)
+ natts = Min(HeapTupleHeaderGetNatts(tup), last_attnum);
+ else
+ {
+ /* No need to access the number of attributes in the tuple */
+ natts = last_attnum;
+ }
+
+ /* All attrs can be fetched without checking for NULLs */
+ firstNullAttr = natts;
+ }
+
+ attnums_idx = attnum_map[slot->tts_nvalid];
+ attnum = attnums[attnums_idx];
+ values = slot->tts_values;
+
+ /*
+ * We store the tupleDesc's CompactAttribute array in 'cattrs' as gcc
+ * seems to be unwilling to optimize accessing the CompactAttribute
+ * element efficiently when accessing it via TupleDescCompactAttr().
+ */
+ cattrs = tupleDesc->compact_attrs;
+
+ /* Ensure we calculated tp correctly */
+ Assert(tp == (char *) tup + tup->t_hoff);
+
+ if (attnum < firstNonGuaranteedAttr)
+ {
+ int attlen;
+
+ for (; attnum < firstNonGuaranteedAttr; attnum = attnums[++attnums_idx])
+ {
+ isnull[attnum] = false;
+ cattr = &cattrs[attnum];
+ attlen = cattr->attlen;
+
+ /* We don't expect any non-byval types */
+ pg_assume(attlen > 0);
+ Assert(cattr->attbyval == true);
+
+ off = cattr->attcacheoff;
+ values[attnum] = fetch_att_noerr(tp + off, true, attlen);
+ }
+
+ off += attlen;
+
+ if (attnum == last_attnum)
+ goto done;
+ }
+
+ /* We can only fetch as many attributes as the tuple has. */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, natts);
+
+ /*
+ * Handle the portion of the tuple that we have cached the offset for up
+ * to the first NULL attribute. The offset is effectively fixed for
+ * these, so we can use the CompactAttribute's attcacheoff.
+ */
+ if (attnum < firstNonCacheOffsetAttr)
+ {
+ int attlen;
+
+ for (; attnum < firstNonCacheOffsetAttr; attnum = attnums[++attnums_idx])
+ {
+ isnull[attnum] = false;
+ cattr = &cattrs[attnum];
+ attlen = cattr->attlen;
+
+ off = cattr->attcacheoff;
+ values[attnum] = fetch_att_noerr(tp + off, cattr->attbyval,
+ attlen);
+ }
+
+ off += attlen;
+ Assert(attlen > 0);
+
+ if (attnum == last_attnum)
+ goto done;
+ }
+
+
+ if (slot->tts_nvalid >= firstNonCacheOffsetAttr)
+ {
+ /* Restore state from previous execution */
+ off_attnum = slot->tts_nvalid;
+ off = *offp;
+ }
+ else
+ {
+ off_attnum = firstNonCacheOffsetAttr - 1;
+ off = cattrs[off_attnum].attcacheoff;
+ }
+
+ /*
+ * We no longer have the ability to use attcacheoff, so we must look
+ * through all attributes from this point on. For attributes that we are
+ * not selecting, we only calculate the offset to skip them, and don't do
+ * the actual fetch. Here we loop up to the first NULL attribute.
+ */
+ for (; off_attnum < firstNullAttr; off_attnum++)
+ {
+ int attlen;
+
+ cattr = &cattrs[off_attnum];
+ attlen = cattr->attlen;
+
+ /*
+ * cstrings don't exist in heap tuples. Use pg_assume to instruct the
+ * compiler not to emit the cstring-related code in
+ * align_fetch_then_add().
+ */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ off = att_pointer_alignby(off, cattr->attalignby, attlen, tp + off);
+
+ /*
+ * If this is an attribute we want, do the fetch and then move attnum
+ * to the next attribute we want.
+ */
+ if (off_attnum == attnum)
+ {
+ isnull[off_attnum] = false;
+ values[off_attnum] =
+ fetch_att_noerr(tp + off, cattr->attbyval,
+ attlen);
+ attnum = attnums[++attnums_idx];
+
+ }
+ /* Move offset beyond this attribute */
+ off = att_addlength_pointer(off, attlen, tp + off);
+ }
+
+ /*
+ * Now handle any remaining attributes in the tuple up to the requested
+ * attnum. This time, include NULL checks as we're now at the first NULL
+ * attribute.
+ */
+ for (; off_attnum < natts; off_attnum++)
+ {
+ int attlen;
+
+ cattr = &cattrs[off_attnum];
+ attlen = cattr->attlen;
+
+ /* As above, we don't expect cstrings */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ /* Is this an attribute we're selecting? */
+ if (off_attnum == attnum)
+ {
+ attnum = attnums[++attnums_idx];
+
+ if (isnull[off_attnum])
+ {
+ values[off_attnum] = (Datum) 0;
+ continue;
+ }
+
+ /*
+ * align 'off', fetch the datum, and increment off beyond the
+ * datum
+ */
+ values[off_attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ attlen,
+ cattr->attalignby);
+ }
+ else if (!isnull[off_attnum])
+ {
+ /* We don't want this attribute, move beyond it */
+ off = att_pointer_alignby(off, cattr->attalignby, attlen, tp + off);
+ off = att_addlength_pointer(off, attlen, tp + off);
+ }
+
+ }
+
+ /* Fetch any missing attrs and raise an error if reqnatts is invalid */
+ if (unlikely(attnum < last_attnum))
+ {
+ *offp = off;
+ /* XXX worth doing this selectively too? */
+ slot_getmissingattrs(slot, attnum, last_attnum);
+ slot->tts_nvalid = last_attnum;
+ return;
+ }
+done:
+
+ slot->tts_nvalid = last_attnum;
+ /* Save current offset for next execution */
+ *offp = off;
+}
+
const TupleTableSlotOps TTSOpsVirtual = {
.base_slot_size = sizeof(VirtualTupleTableSlot),
.init = tts_virtual_init,
.release = tts_virtual_release,
.clear = tts_virtual_clear,
.getsomeattrs = tts_virtual_getsomeattrs,
+ .selectattrs = tts_virtual_selectattrs,
.getsysattr = tts_virtual_getsysattr,
.materialize = tts_virtual_materialize,
.is_current_xact_tuple = tts_virtual_is_current_xact_tuple,
@@ -1271,6 +1628,7 @@ const TupleTableSlotOps TTSOpsHeapTuple = {
.release = tts_heap_release,
.clear = tts_heap_clear,
.getsomeattrs = tts_heap_getsomeattrs,
+ .selectattrs = tts_heap_selectattrs,
.getsysattr = tts_heap_getsysattr,
.is_current_xact_tuple = tts_heap_is_current_xact_tuple,
.materialize = tts_heap_materialize,
@@ -1289,6 +1647,7 @@ const TupleTableSlotOps TTSOpsMinimalTuple = {
.release = tts_minimal_release,
.clear = tts_minimal_clear,
.getsomeattrs = tts_minimal_getsomeattrs,
+ .selectattrs = tts_minimal_selectattrs,
.getsysattr = tts_minimal_getsysattr,
.is_current_xact_tuple = tts_minimal_is_current_xact_tuple,
.materialize = tts_minimal_materialize,
@@ -1307,6 +1666,7 @@ const TupleTableSlotOps TTSOpsBufferHeapTuple = {
.release = tts_buffer_heap_release,
.clear = tts_buffer_heap_clear,
.getsomeattrs = tts_buffer_heap_getsomeattrs,
+ .selectattrs = tts_buffer_heap_selectattrs,
.getsysattr = tts_buffer_heap_getsysattr,
.is_current_xact_tuple = tts_buffer_is_current_xact_tuple,
.materialize = tts_buffer_heap_materialize,
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index f62582859f9..252e8306631 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -580,8 +580,7 @@ ExecGetCommonChildSlotOps(PlanState *ps)
* ----------------
*/
void
-ExecAssignProjectionInfo(PlanState *planstate,
- TupleDesc inputDesc)
+ExecAssignProjectionInfo(PlanState *planstate, TupleDesc inputDesc)
{
planstate->ps_ProjInfo =
ExecBuildProjectionInfo(planstate->plan->targetlist,
@@ -591,6 +590,28 @@ ExecAssignProjectionInfo(PlanState *planstate,
inputDesc);
}
+/* ----------------
+ * ExecAssignProjectionInfoWithScanAttrs
+ *
+ * As ExecAssignScanProjectionInfo but when 'scan_attrs' is set, use
+ * EEOP_SCAN_SELECTSOME instead of EEOP_SCAN_FETCHSOME to deform only the
+ * mentioned 'scan_attrs' from the scan tuple.
+ * ----------------
+*/
+void
+ExecAssignProjectionInfoWithScanAttrs(PlanState *planstate,
+ TupleDesc inputDesc,
+ Bitmapset *scan_attrs)
+{
+ planstate->ps_ProjInfo =
+ ExecBuildProjectionInfoWithScanAttrs(planstate->plan->targetlist,
+ planstate->ps_ExprContext,
+ planstate->ps_ResultTupleSlot,
+ planstate,
+ inputDesc,
+ scan_attrs);
+}
+
/* ----------------
* ExecConditionalAssignProjectionInfo
@@ -602,6 +623,26 @@ ExecAssignProjectionInfo(PlanState *planstate,
void
ExecConditionalAssignProjectionInfo(PlanState *planstate, TupleDesc inputDesc,
int varno)
+{
+ ExecConditionalAssignProjectionInfoWithScanAttrs(planstate,
+ inputDesc,
+ varno,
+ NULL);
+}
+
+/* ----------------
+ * ExecConditionalAssignProjectionInfoWithScanAttrs
+ *
+ * As ExecConditionalAssignProjectionInfo but when 'scan_attrs' is set, use
+ * EEOP_SCAN_SELECTSOME instead of EEOP_SCAN_FETCHSOME to deform only the
+ * mentioned 'scan_attrs' from the scan tuple.
+ * ----------------
+*/
+void
+ExecConditionalAssignProjectionInfoWithScanAttrs(PlanState *planstate,
+ TupleDesc inputDesc,
+ int varno,
+ Bitmapset *scan_attrs)
{
if (tlist_matches_tupdesc(planstate,
planstate->plan->targetlist,
@@ -622,7 +663,7 @@ ExecConditionalAssignProjectionInfo(PlanState *planstate, TupleDesc inputDesc,
planstate->resultopsfixed = true;
planstate->resultopsset = true;
}
- ExecAssignProjectionInfo(planstate, inputDesc);
+ ExecAssignProjectionInfoWithScanAttrs(planstate, inputDesc, scan_attrs);
}
}
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 8f219f60a93..41de367832c 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -251,13 +251,15 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags)
* Initialize result type and projection.
*/
ExecInitResultTypeTL(&scanstate->ss.ps);
- ExecAssignScanProjectionInfo(&scanstate->ss);
+ ExecAssignScanProjectionInfoWithScanAttrs(&scanstate->ss,
+ node->scan.scan_varattnos);
/*
* initialize child expressions
*/
- scanstate->ss.ps.qual =
- ExecInitQual(node->scan.plan.qual, (PlanState *) scanstate);
+ scanstate->ss.ps.qual = ExecInitQualWithScanAttrs(node->scan.plan.qual,
+ (PlanState *) scanstate,
+ node->scan.scan_varattnos);
/*
* When EvalPlanQual() is not in use, assign ExecProcNode for this node
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 50b0e10308b..4522ac4d4c0 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -118,7 +118,8 @@ static ModifyTable *create_modifytable_plan(PlannerInfo *root, ModifyTablePath *
static Limit *create_limit_plan(PlannerInfo *root, LimitPath *best_path,
int flags);
static SeqScan *create_seqscan_plan(PlannerInfo *root, Path *best_path,
- List *tlist, List *scan_clauses);
+ List *tlist, List *scan_clauses,
+ Bitmapset *tlist_varattnos);
static SampleScan *create_samplescan_plan(PlannerInfo *root, Path *best_path,
List *tlist, List *scan_clauses);
static Scan *create_indexscan_plan(PlannerInfo *root, IndexPath *best_path,
@@ -178,7 +179,8 @@ static void label_sort_with_costsize(PlannerInfo *root, Sort *plan,
double limit_tuples);
static void label_incrementalsort_with_costsize(PlannerInfo *root, IncrementalSort *plan,
List *pathkeys, double limit_tuples);
-static SeqScan *make_seqscan(List *qptlist, List *qpqual, Index scanrelid);
+static SeqScan *make_seqscan(List *qptlist, List *qpqual, Index scanrelid,
+ Bitmapset *scan_varattnos);
static SampleScan *make_samplescan(List *qptlist, List *qpqual, Index scanrelid,
TableSampleClause *tsc);
static IndexScan *make_indexscan(List *qptlist, List *qpqual, Index scanrelid,
@@ -550,6 +552,7 @@ create_plan_recurse(PlannerInfo *root, Path *best_path, int flags)
static Plan *
create_scan_plan(PlannerInfo *root, Path *best_path, int flags)
{
+ Bitmapset *tlist_varattnos = NULL;
RelOptInfo *rel = best_path->parent;
List *scan_clauses;
List *gating_clauses;
@@ -579,6 +582,14 @@ create_scan_plan(PlannerInfo *root, Path *best_path, int flags)
break;
}
+ /*
+ * Figure out which attributes we need from the scan before applying the
+ * physical tlist optimization.
+ */
+ pull_varattnos((Node *) best_path->pathtarget->exprs,
+ rel->relid,
+ &tlist_varattnos);
+
/*
* If this is a parameterized scan, we also need to enforce all the join
* clauses available from the outer relation(s).
@@ -672,7 +683,8 @@ create_scan_plan(PlannerInfo *root, Path *best_path, int flags)
plan = (Plan *) create_seqscan_plan(root,
best_path,
tlist,
- scan_clauses);
+ scan_clauses,
+ tlist_varattnos);
break;
case T_SampleScan:
@@ -2752,10 +2764,13 @@ create_limit_plan(PlannerInfo *root, LimitPath *best_path, int flags)
*/
static SeqScan *
create_seqscan_plan(PlannerInfo *root, Path *best_path,
- List *tlist, List *scan_clauses)
+ List *tlist, List *scan_clauses, Bitmapset *tlist_varattnos)
{
SeqScan *scan_plan;
Index scan_relid = best_path->parent->relid;
+ Bitmapset *scan_varattnos = tlist_varattnos;
+ Bitmapset *non_sys_attrs = NULL;
+ int i;
/* it should be a base rel... */
Assert(scan_relid > 0);
@@ -2767,6 +2782,19 @@ create_seqscan_plan(PlannerInfo *root, Path *best_path,
/* Reduce RestrictInfo list to bare expressions; ignore pseudoconstants */
scan_clauses = extract_actual_clauses(scan_clauses, false);
+ /* Pull varattnos from WHERE clause Vars */
+ pull_varattnos((Node *) scan_clauses, scan_relid, &scan_varattnos);
+
+ /* Don't set these when whole-row var is present */
+ if (!bms_is_member(0 - FirstLowInvalidHeapAttributeNumber, scan_varattnos))
+ {
+ /* XXX invent bms_right_shift_members()? */
+ i = 0 - FirstLowInvalidHeapAttributeNumber;
+ while ((i = bms_next_member(scan_varattnos, i)) >= 0)
+ non_sys_attrs = bms_add_member(non_sys_attrs,
+ i - 1 + FirstLowInvalidHeapAttributeNumber);
+ }
+
/* Replace any outer-relation variables with nestloop params */
if (best_path->param_info)
{
@@ -2776,7 +2804,8 @@ create_seqscan_plan(PlannerInfo *root, Path *best_path,
scan_plan = make_seqscan(tlist,
scan_clauses,
- scan_relid);
+ scan_relid,
+ non_sys_attrs);
copy_generic_path_info(&scan_plan->scan.plan, best_path);
@@ -5487,7 +5516,8 @@ bitmap_subplan_mark_shared(Plan *plan)
static SeqScan *
make_seqscan(List *qptlist,
List *qpqual,
- Index scanrelid)
+ Index scanrelid,
+ Bitmapset *scan_varattnos)
{
SeqScan *node = makeNode(SeqScan);
Plan *plan = &node->scan.plan;
@@ -5497,6 +5527,7 @@ make_seqscan(List *qptlist,
plan->lefttree = NULL;
plan->righttree = NULL;
node->scan.scanrelid = scanrelid;
+ node->scan.scan_varattnos = scan_varattnos;
return node;
}
diff --git a/src/include/executor/execExpr.h b/src/include/executor/execExpr.h
index aa9b361fa31..f29d9dd799b 100644
--- a/src/include/executor/execExpr.h
+++ b/src/include/executor/execExpr.h
@@ -78,6 +78,9 @@ typedef enum ExprEvalOp
EEOP_OLD_FETCHSOME,
EEOP_NEW_FETCHSOME,
+ /* apply slot_selectattrs on the corresponding tuple slot */
+ EEOP_SCAN_SELECTSOME,
+
/* compute non-system Var value */
EEOP_INNER_VAR,
EEOP_OUTER_VAR,
@@ -318,15 +321,34 @@ typedef struct ExprEvalStep
*/
union
{
- /* for EEOP_INNER/OUTER/SCAN/OLD/NEW_FETCHSOME */
+ /*
+ * for EEOP_INNER/OUTER/SCAN/OLD/NEW_FETCHSOME and
+ * EEOP_SCAN_SELECTSOME
+ */
struct
{
/* attribute number up to which to fetch (inclusive) */
int last_var;
/* will the type of slot be the same for every invocation */
bool fixed;
+ /* Number of elements in req_attnums array. XXX needed? */
+ AttrNumber natts;
+
+ /* One element for each attnum to select, ordered by attnum */
+ AttrNumber *req_attnums;
+
+ /*
+ * Provides mapping of 0-based attnums back to the index of the
+ * req_attnums array that deforming should continue from. This
+ * allows us to re-find the element of req_attnums using the
+ * slot's tts_nvalid so that we can continue deforming from the
+ * last defromed attribute.
+ */
+ AttrNumber *next_req_attnums_index;
+
/* tuple descriptor, if known */
TupleDesc known_desc;
+
/* type of slot, can only be relied upon if fixed is set */
const TupleTableSlotOps *kind;
} fetch;
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index bf239fc156f..69f3c5da6c5 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -324,8 +324,12 @@ ExecProcNode(PlanState *node)
* prototypes from functions in execExpr.c
*/
extern ExprState *ExecInitExpr(Expr *node, PlanState *parent);
+extern ExprState *ExecInitExprWithScanAttrs(Expr *node, PlanState *parent,
+ Bitmapset *scan_attrs);
extern ExprState *ExecInitExprWithParams(Expr *node, ParamListInfo ext_params);
extern ExprState *ExecInitQual(List *qual, PlanState *parent);
+extern ExprState *ExecInitQualWithScanAttrs(List *qual, PlanState *parent,
+ Bitmapset *scan_attrs);
extern ExprState *ExecInitCheck(List *qual, PlanState *parent);
extern List *ExecInitExprList(List *nodes, PlanState *parent);
extern ExprState *ExecBuildAggTrans(AggState *aggstate, struct AggStatePerPhaseData *phase,
@@ -364,6 +368,12 @@ extern ProjectionInfo *ExecBuildProjectionInfo(List *targetList,
TupleTableSlot *slot,
PlanState *parent,
TupleDesc inputDesc);
+extern ProjectionInfo *ExecBuildProjectionInfoWithScanAttrs(List *targetList,
+ ExprContext *econtext,
+ TupleTableSlot *slot,
+ PlanState *parent,
+ TupleDesc inputDesc,
+ Bitmapset *scan_attrs);
extern ProjectionInfo *ExecBuildUpdateProjection(List *targetList,
bool evalTargetList,
List *targetColnos,
@@ -582,6 +592,8 @@ typedef bool (*ExecScanRecheckMtd) (ScanState *node, TupleTableSlot *slot);
extern TupleTableSlot *ExecScan(ScanState *node, ExecScanAccessMtd accessMtd,
ExecScanRecheckMtd recheckMtd);
extern void ExecAssignScanProjectionInfo(ScanState *node);
+extern void ExecAssignScanProjectionInfoWithScanAttrs(ScanState *node,
+ Bitmapset *scan_attrs);
extern void ExecAssignScanProjectionInfoWithVarno(ScanState *node, int varno);
extern void ExecScanReScan(ScanState *node);
@@ -678,8 +690,15 @@ extern const TupleTableSlotOps *ExecGetCommonSlotOps(PlanState **planstates,
extern const TupleTableSlotOps *ExecGetCommonChildSlotOps(PlanState *ps);
extern void ExecAssignProjectionInfo(PlanState *planstate,
TupleDesc inputDesc);
+extern void ExecAssignProjectionInfoWithScanAttrs(PlanState *planstate,
+ TupleDesc inputDesc,
+ Bitmapset *scan_attrs);
extern void ExecConditionalAssignProjectionInfo(PlanState *planstate,
TupleDesc inputDesc, int varno);
+extern void ExecConditionalAssignProjectionInfoWithScanAttrs(PlanState *planstate,
+ TupleDesc inputDesc,
+ int varno,
+ Bitmapset *scan_attrs);
extern void ExecAssignScanType(ScanState *scanstate, TupleDesc tupDesc);
extern void ExecCreateScanSlotFromOuterPlan(EState *estate,
ScanState *scanstate,
diff --git a/src/include/executor/tuptable.h b/src/include/executor/tuptable.h
index 78558098fa3..cc2c5a257d0 100644
--- a/src/include/executor/tuptable.h
+++ b/src/include/executor/tuptable.h
@@ -168,6 +168,16 @@ struct TupleTableSlotOps
*/
void (*getsomeattrs) (TupleTableSlot *slot, int natts);
+ /*
+ * Populate the tts_values and tts_isnull elements of the given slot with
+ * the values of the corresponding attribute from the tuple stored in the
+ * slot. Populate up as far as last_attnum and store each attribute
+ * mentioned in the attnums array. Use attnum_map to determine the
+ * starting element in the attnums array from the slot's tts_nvalid.
+ */
+ void (*selectattrs) (TupleTableSlot *slot, int last_attnum,
+ AttrNumber *attnums, AttrNumber *attnum_map);
+
/*
* Returns value of the given system attribute as a datum and sets isnull
* to false, if it's not NULL. Throws an error if the slot type does not
@@ -374,6 +384,18 @@ slot_getsomeattrs(TupleTableSlot *slot, int attnum)
slot->tts_ops->getsomeattrs(slot, attnum);
}
+static inline void
+slot_selectattrs(TupleTableSlot *slot, int last_attnum, AttrNumber *attnums,
+ AttrNumber *attnum_map)
+{
+ /*
+ * Populate slot only attributes mentioned in the attnums array, up to
+ * 'last_attnum', if it's not already
+ */
+ if (slot->tts_nvalid < last_attnum)
+ slot->tts_ops->selectattrs(slot, last_attnum, attnums, attnum_map);
+}
+
/*
* slot_getallattrs
* This function forces all the entries of the slot's Datum/isnull
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 8c9321aab8c..08dcf02b8bb 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -540,6 +540,14 @@ typedef struct Scan
Plan plan;
/* relid is index into the range table */
Index scanrelid;
+
+ /*
+ * All varattnos that are required from the scanrelid. Does not include
+ * any added due to the physical tlist optimization or system attributes
+ * or whole-row attributes. User attributes are 0 based, i.e attnum==1 is
+ * member 0.
+ */
+ Bitmapset *scan_varattnos;
} Scan;
/* ----------------
--
2.51.0
^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: More speedups for tuple deformation
@ 2026-03-07 16:36 Junwang Zhao <[email protected]>
parent: David Rowley <[email protected]>
0 siblings, 1 reply; 30+ messages in thread
From: Junwang Zhao @ 2026-03-07 16:36 UTC (permalink / raw)
To: David Rowley <[email protected]>; +Cc: Andres Freund <[email protected]>; John Naylor <[email protected]>; Chao Li <[email protected]>; PostgreSQL Developers <[email protected]>
Hi David,
On Fri, Mar 6, 2026 at 12:10 PM David Rowley <[email protected]> wrote:
>
> One of my goals for proactively populating
> CompactAttribute.attcacheoff is to make it so we're able to support
> deforming only a subset of columns. If we only need a small number of
> columns from the tuple and all those columns have a known attcacheoff
> and no NULLs come prior, then we can quite efficiently just go to
> those cached offsets and fetch only the attributes that we need. To
> do this, we'll need an extra array to store which attnums we're
> interested in, rather than deforming all attrs up to the highest
> attnum that we need, as we do today. I expect that looking at this
> new array will slow things down a bit when we're accessing either most
> or all columns in, say, a SELECT * query. So, IMO, it'd be bad to
> *replace* the current deforming code with code which does this.
> Instead, I propose we add an additional deform operator and have some
> heuristic which decides which one is best to use. I expect
> ExecPushExprSetupSteps() could make that choice fairly easily. Perhaps
> something cheap like bms_num_members(scan_attrs) is less than half the
> bms_prev_member(scan_attrs, -1) (the highest member).
>
> There's going to be many cases where the attcacheoff isn't known in
> the attributes being selected. So that we still get some gains when
> that's the case, I've coded it up so that we start walking the tuple
> at the last attribute that has an attcacheoff. In many cases, that'll
> mean we don't need to walk the entire tuple. Often, leading columns
> are fixed-width, so this means that there's likely some benefit to
> most cases. There might need to be a bit more education or
> documentation about best column ordering practises.
>
> There are a few hurdles to make this work, and one is the physical
> tlist optimization. If the planner replaces the targetlist with a
> physical tlist, the executor is going to think we need all columns,
> which would have it likely choose not to do the selective deforming.
> To make this work, I've added some code in createplan.c to extract the
> attnums we need from the qual and tlist before the physical tlist is
> installed. That's recorded in a Bitmapset and passed down to the
> executor and to the code which sets up the ExprStates. Currently,
> mostly to exercise this code as much as possible, I've coded it to
> always do the selective deforming when the Bitmapset isn't empty. So
> far, I've only done this for Seq Scan, but I expect all the scans that
> deform tuples could use this.
>
> I've attached the code which does all this in the 0006 patch.
> Ideally, I'd have had this at least to the current state about 2-3
> months ago, so I don't intend that 0006 is v19 material, but I wanted
> to share to show where I intend this work to go.
>
> Performance:
>
> Using the t_1_40 table from the deform_test_setup.sh script I sent in
> [1], running "select a from t_1_40 where a = 0;" ("a" is the 43rd
> column in that table), on my Zen2 machine, I get the following from
> perf top and pgbench:
>
> master:
> 75.57% postgres [.] tts_buffer_heap_getsomeattrs
> 4.70% postgres [.] ExecInterpExpr
> 2.85% postgres [.] ExecSeqScanWithQualProject
> 1.94% postgres [.] heapgettup_pagemode
> 1.21% postgres [.] UnlockBuffer
> 1.15% postgres [.] slot_getsomeattrs_int
>
> $ for i in {1..3}; do pgbench -n -f bench.sql -M prepared -T 10
> postgres | grep latency; done
> latency average = 154.175 ms
> latency average = 156.780 ms
> latency average = 157.599 ms
>
> 0001-0005:
> 64.24% postgres [.] tts_buffer_heap_getsomeattrs
> 15.01% postgres [.] ExecInterpExpr
> 3.22% postgres [.] ExecSeqScanWithQualProject
> 3.01% postgres [.] heapgettup_pagemode
> 1.57% postgres [.] ExecStoreBufferHeapTuple
> 1.53% postgres [.] heap_prepare_pagescan
>
> $ for i in {1..3}; do pgbench -n -f bench.sql -M prepared -T 10
> postgres | grep latency; done
> latency average = 130.981 ms
> latency average = 134.700 ms
> latency average = 134.898 ms
>
> 0001-0006:
> 42.28% postgres [.] heapgettup_pagemode
> 11.38% postgres [.] ExecInterpExpr
> 7.13% postgres [.] ExecSeqScanWithQualProject
> 5.92% postgres [.] tts_buffer_heap_selectattrs <-- it's down here.
> 5.69% postgres [.] ExecStoreBufferHeapTuple
> 5.11% postgres [.] heap_getnextslot
> 3.87% postgres [.] heap_prepare_pagescan
>
> $ for i in {1..3}; do pgbench -n -f bench.sql -M prepared -T 10
> postgres | grep latency; done
> latency average = 71.689 ms
> latency average = 75.638 ms
> latency average = 75.149 ms
>
> Keep in mind that this is one of the best cases as t_1_40 has no NULLs
> and only has fixed-width columns. The only slightly better case would
> be to add more columns and fetch only the final one. 40 doesn't seem
> excessively unrealistic, to get an idea of the gains that someone
> *could* see.
>
> You can see that perf top report that tts_buffer_heap_getsomeattrs
> dropped from taking 75.57% down to 64.24% with 0001-0005. Adding 0006
> sees that replaced with tts_buffer_heap_selectattrs which takes less
> than 6% of the CPU time. It also highlights the next most interesting
> thing we should probably make faster, heapgettup_pagemode().
>
> I've attached v12 of the patch. There are a few changes in 0001-0005
> that should help make things a bit faster than v11. I've also attached
> the new selective deforming code in 0006. There's no JIT support for
> 0006 yet, I don't need to be told about that :-)
>
> I'm planning on starting to go through 0002-0005 in much more detail
> from mid next week with my committer hat on. If anyone wants to relook
> at any of the 0002-0005 patches, there's still time. I'm also happy to
I have some comments on v12-0004.
1.
+ off += cattr->attlen;
+ firstNonCachedOffsetAttr = i + 1;
+ }
+
+ tupdesc->firstNonCachedOffsetAttr = firstNonCachedOffsetAttr;
+ tupdesc->firstNonGuaranteedAttr = firstNonGuaranteedAttr;
+}
The firstNonCachedOffsetAttr seems to be the first variable width
attribute, but it seems that the offset of this attribute can be cached,
for example, in a table defined as (int, text), the offset of
firstNonCachedOffsetAttr should be 4, is that correct?
If TupleDescFinalize records the offset firstNonCachedOffsetAttr,
it might save one iterator of the deforming loop. For example,
add something like the following after the above mentioned code.
if (firstNonCachedOffsetAttr < tupdesc->natts)
{
cattr = TupleDescCompactAttr(tupdesc, firstNonCachedOffsetAttr);
cattr->attcacheoff = off;
}
2.
in slot_deform_heap_tuple, there are multiple statements setting
firstNonCacheOffsetAttr,
+ firstNonCacheOffsetAttr = tupleDesc->firstNonCachedOffsetAttr;
+ /* We can only use any cached offsets until the first NULL attr */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr,
+ firstNullAttr);
+ /* We can only fetch as many attributes as the tuple has. */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, natts);
Based on the logic, it seems the second one could be moved
to the third position, and the third one could then be safely
removed?
> receive feedback on 0006, but I will address concerns with that at a
> lower priority. One thing that's still left todo in the 0004 patch is
> enable the TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS optimisation for a few
> other scan types.
>
> Thanks for reading
>
> David
>
> [1] https://postgr.es/m/CAApHDvo1i-ycAcWnK3L7ZASTuM8mW46kvRqMaUHD46HSuJmx7A@mail.gmail.com
--
Regards
Junwang Zhao
^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: More speedups for tuple deformation
@ 2026-03-13 12:18 David Rowley <[email protected]>
parent: Junwang Zhao <[email protected]>
0 siblings, 2 replies; 30+ messages in thread
From: David Rowley @ 2026-03-13 12:18 UTC (permalink / raw)
To: Junwang Zhao <[email protected]>; +Cc: Andres Freund <[email protected]>; John Naylor <[email protected]>; Chao Li <[email protected]>; PostgreSQL Developers <[email protected]>
Thanks for having a look.
On Sun, 8 Mar 2026 at 05:36, Junwang Zhao <[email protected]> wrote:
> I have some comments on v12-0004.
>
> 1.
>
> + off += cattr->attlen;
> + firstNonCachedOffsetAttr = i + 1;
> + }
> +
> + tupdesc->firstNonCachedOffsetAttr = firstNonCachedOffsetAttr;
> + tupdesc->firstNonGuaranteedAttr = firstNonGuaranteedAttr;
> +}
>
> The firstNonCachedOffsetAttr seems to be the first variable width
> attribute, but it seems that the offset of this attribute can be cached,
> for example, in a table defined as (int, text), the offset of
> firstNonCachedOffsetAttr should be 4, is that correct?
Yes.
> If TupleDescFinalize records the offset firstNonCachedOffsetAttr,
> it might save one iterator of the deforming loop. For example,
> add something like the following after the above mentioned code.
>
> if (firstNonCachedOffsetAttr < tupdesc->natts)
> {
> cattr = TupleDescCompactAttr(tupdesc, firstNonCachedOffsetAttr);
> cattr->attcacheoff = off;
> }
The problem is that short varlenas have 1 byte alignment and normal
varlenas have 4 byte alignment. It might be possible to do something
if the previous column is 4-byte aligned and has a length of 4 or 8,
since that means the offset must also be 1-byte aligned. The main
reason I don't want to do this is that the only positive is that
*maybe* 1 extra column can be deformed with a fixed offset. The
drawback is that the following code *has* to use
att_addlength_pointer(), *regardless* instead of "off += attlen;".
This means more deforming code and more complexity in
TupleDescFinalize(). I'd rather not do this.
> 2.
>
> in slot_deform_heap_tuple, there are multiple statements setting
> firstNonCacheOffsetAttr,
>
> + firstNonCacheOffsetAttr = tupleDesc->firstNonCachedOffsetAttr;
>
> + /* We can only use any cached offsets until the first NULL attr */
> + firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr,
> + firstNullAttr);
>
> + /* We can only fetch as many attributes as the tuple has. */
> + firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, natts);
>
> Based on the logic, it seems the second one could be moved
> to the third position, and the third one could then be safely
> removed?
Yeah. Well spotted. I've done that in the attached.
I've also modified the 0006 patch to add a new deform_bench_select()
function which allows the benchmark to call the new selective deform
function. See the attached graphs comparing master to v13-0001-0005
and master to v13-0001-0006. It's good to see that there's still quite
a large speedup even from the tests that don't have an attcacheoff for
the column being deformed. Tests 1 and 5 do have a attcacheoff for the
column deformed, so they're a good bit faster again. To get the
0001-0006 results, I used the deform_test_run.sh script from [1] and
modified it to call deform_bench_select() instead of deform_bench().
I also noticed that when building with older gcc versions, I was
getting warnings about attlen and 'off' not being initialised. I ended
up switching back to the do/while loops to fix that rather than adding
needless initialisation, which would add overhead. 1 loop is
guaranteed, and the older compiler is not clever enough to work that
out.
David
[1] https://postgr.es/m/CAApHDvo1i-ycAcWnK3L7ZASTuM8mW46kvRqMaUHD46HSuJmx7A@mail.gmail.com
From cc6d328cf1eb3e25df88afe5733d35817ac4f3e0 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Tue, 27 Jan 2026 15:08:09 +1300
Subject: [PATCH v13 1/6] Introduce deform_bench test module
For benchmarking tuple deformation.
---
src/test/modules/deform_bench/.gitignore | 4 +
src/test/modules/deform_bench/Makefile | 21 ++++
.../deform_bench/deform_bench--1.0.sql | 8 ++
src/test/modules/deform_bench/deform_bench.c | 107 ++++++++++++++++++
.../modules/deform_bench/deform_bench.control | 4 +
src/test/modules/deform_bench/meson.build | 22 ++++
src/test/modules/meson.build | 1 +
7 files changed, 167 insertions(+)
create mode 100644 src/test/modules/deform_bench/.gitignore
create mode 100644 src/test/modules/deform_bench/Makefile
create mode 100644 src/test/modules/deform_bench/deform_bench--1.0.sql
create mode 100644 src/test/modules/deform_bench/deform_bench.c
create mode 100644 src/test/modules/deform_bench/deform_bench.control
create mode 100644 src/test/modules/deform_bench/meson.build
diff --git a/src/test/modules/deform_bench/.gitignore b/src/test/modules/deform_bench/.gitignore
new file mode 100644
index 00000000000..5dcb3ff9723
--- /dev/null
+++ b/src/test/modules/deform_bench/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/src/test/modules/deform_bench/Makefile b/src/test/modules/deform_bench/Makefile
new file mode 100644
index 00000000000..b5fc0f7a583
--- /dev/null
+++ b/src/test/modules/deform_bench/Makefile
@@ -0,0 +1,21 @@
+# src/test/modules/deform_bench/Makefile
+
+MODULE_big = deform_bench
+OBJS = deform_bench.o
+
+EXTENSION = deform_bench
+DATA = deform_bench--1.0.sql
+PGFILEDESC = "deform_bench - tuple deform benchmarking"
+
+REGRESS = deform_bench
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = src/test/modules/deform_bench
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/src/test/modules/deform_bench/deform_bench--1.0.sql b/src/test/modules/deform_bench/deform_bench--1.0.sql
new file mode 100644
index 00000000000..492b71dba3b
--- /dev/null
+++ b/src/test/modules/deform_bench/deform_bench--1.0.sql
@@ -0,0 +1,8 @@
+/* deform_bench--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION deform_bench" to load this file. \quit
+
+CREATE FUNCTION deform_bench(tableoid Oid, attnum int[]) RETURNS FLOAT
+AS 'MODULE_PATHNAME', 'deform_bench'
+LANGUAGE C VOLATILE STRICT;
diff --git a/src/test/modules/deform_bench/deform_bench.c b/src/test/modules/deform_bench/deform_bench.c
new file mode 100644
index 00000000000..7838f639bef
--- /dev/null
+++ b/src/test/modules/deform_bench/deform_bench.c
@@ -0,0 +1,107 @@
+/*-------------------------------------------------------------------------
+ *
+ * deform_bench.c
+ *
+ * for benchmarking tuple deformation routines
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include <time.h>
+#include <sys/time.h>
+
+#include "access/heapam.h"
+#include "access/relscan.h"
+#include "catalog/pg_am_d.h"
+#include "catalog/pg_type_d.h"
+#include "funcapi.h"
+#include "miscadmin.h"
+#include "utils/array.h"
+#include "utils/arrayaccess.h"
+#include "utils/builtins.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(deform_bench);
+
+Datum
+deform_bench(PG_FUNCTION_ARGS)
+{
+ Oid tableoid = PG_GETARG_OID(0);
+ ArrayType *array = PG_GETARG_ARRAYTYPE_P(1);
+ TableScanDesc scan;
+ Relation rel;
+ TupleDesc tupdesc;
+ TupleTableSlot *slot;
+ Datum *elem_datums = NULL;
+ bool *elem_nulls = NULL;
+ int elem_count;
+ int *attnums;
+ clock_t start,
+ end;
+
+ rel = relation_open(tableoid, AccessShareLock);
+
+ if (rel->rd_rel->relam != HEAP_TABLE_AM_OID)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("only heap AM is supported")));
+
+ tupdesc = RelationGetDescr(rel);
+ slot = MakeTupleTableSlot(tupdesc, &TTSOpsBufferHeapTuple);
+ scan = table_beginscan_strat(rel, GetActiveSnapshot(), 0, NULL, true, false);
+
+ /*
+ * The array is used to allow callers to define how many atts to deform.
+ * e.g: '{1,10}'::int[] would deform attnum=1, then in a 2nd pass deform
+ * the remainder up to attnum=10. Passing an element as NULL means all
+ * attnums. This allows simulation of incremental deformation. Generally
+ * if you're passing an array with more than 1 element, then the array
+ * should be in ascending order. Doing something like '{10,1}' would mean
+ * we've already deformed 10 attributes and on the 2nd pass there's
+ * nothing to do since attnum=1 was already deformed in the first pass.
+ *
+ * You'll get an ERROR if you pass a number higher than the number of
+ * attributes in the table.
+ */
+ deconstruct_array(array,
+ INT4OID,
+ sizeof(int32),
+ true,
+ 'i',
+ &elem_datums,
+ &elem_nulls,
+ &elem_count);
+
+ attnums = palloc_array(int, elem_count);
+
+ for (int i = 0; i < elem_count; i++)
+ {
+ /* Make a NULL element mean all attributes */
+ if (elem_nulls[i])
+ attnums[i] = tupdesc->natts;
+ else
+ attnums[i] = DatumGetInt32(elem_datums[i]);
+ }
+
+ start = clock();
+
+ while (heap_getnextslot(scan, ForwardScanDirection, slot))
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ /* Deform in stages according to the attnums array */
+ for (int i = 0; i < elem_count; i++)
+ slot_getsomeattrs(slot, attnums[i]);
+ }
+
+ end = clock();
+
+ ExecDropSingleTupleTableSlot(slot);
+ table_endscan(scan);
+ relation_close(rel, AccessShareLock);
+
+
+ /* Returns the number of milliseconds to run the test */
+ PG_RETURN_FLOAT8((double) (end - start) / (CLOCKS_PER_SEC / 1000));
+}
diff --git a/src/test/modules/deform_bench/deform_bench.control b/src/test/modules/deform_bench/deform_bench.control
new file mode 100644
index 00000000000..a2023f9d738
--- /dev/null
+++ b/src/test/modules/deform_bench/deform_bench.control
@@ -0,0 +1,4 @@
+# deform_bench extension
+comment = 'functions for benchmarking tuple deformation'
+default_version = '1.0'
+module_pathname = '$libdir/deform_bench'
diff --git a/src/test/modules/deform_bench/meson.build b/src/test/modules/deform_bench/meson.build
new file mode 100644
index 00000000000..82049585244
--- /dev/null
+++ b/src/test/modules/deform_bench/meson.build
@@ -0,0 +1,22 @@
+# Copyright (c) 2026, PostgreSQL Global Development Group
+
+deform_bench_sources = files(
+ 'deform_bench.c',
+)
+
+if host_system == 'windows'
+ deform_bench_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+ '--NAME', 'deform_bench',
+ '--FILEDESC', 'deform_bench - benchmarking tuple deformation',])
+endif
+
+deform_bench = shared_module('deform_bench',
+ deform_bench_sources,
+ kwargs: pg_test_mod_args,
+)
+test_install_libs += deform_bench
+
+test_install_data += files(
+ 'deform_bench--1.0.sql',
+ 'deform_bench.control',
+)
diff --git a/src/test/modules/meson.build b/src/test/modules/meson.build
index e2b3eef4136..1312f3b4d7b 100644
--- a/src/test/modules/meson.build
+++ b/src/test/modules/meson.build
@@ -2,6 +2,7 @@
subdir('brin')
subdir('commit_ts')
+subdir('deform_bench')
subdir('delay_execution')
subdir('dummy_index_am')
subdir('dummy_seclabel')
--
2.51.0
From 1fef8685b5ea019086f68b41fa2bfc995da4353c Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Mon, 16 Feb 2026 14:20:19 +1300
Subject: [PATCH v13 2/6] Allow sibling call optimization in
slot_getsomeattrs_int()
This changes the TupleTableSlotOps contract to make it so the
getsomeattrs() function is in charge of calling
slot_getmissingattrs().
Since this removes all code from slot_getsomeattrs_int() aside from the
getsomeattrs() call itself, we may as well adjust slot_getsomeattrs() so
that it calls getsomeattrs() directly. We leave slot_getsomeattrs_int()
intact as this is still called from the JIT code.
---
src/backend/executor/execTuples.c | 58 ++++++++++++++++---------------
src/include/executor/tuptable.h | 13 ++++---
2 files changed, 38 insertions(+), 33 deletions(-)
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index b768eae9e53..7effe954286 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -73,7 +73,7 @@
static TupleDesc ExecTypeFromTLInternal(List *targetList,
bool skipjunk);
static pg_attribute_always_inline void slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
- int natts);
+ int reqnatts);
static inline void tts_buffer_heap_store_tuple(TupleTableSlot *slot,
HeapTuple tuple,
Buffer buffer,
@@ -1108,7 +1108,10 @@ slot_deform_heap_tuple_internal(TupleTableSlot *slot, HeapTuple tuple,
* slot_deform_heap_tuple
* Given a TupleTableSlot, extract data from the slot's physical tuple
* into its Datum/isnull arrays. Data is extracted up through the
- * natts'th column (caller must ensure this is a legal column number).
+ * reqnatts'th column. If there are insufficient attributes in the given
+ * tuple, then slot_getmissingattrs() is called to populate the
+ * remainder. If reqnatts is above the number of attributes in the
+ * slot's TupleDesc, an error is raised.
*
* This is essentially an incremental version of heap_deform_tuple:
* on each call we extract attributes up to the one needed, without
@@ -1120,21 +1123,23 @@ slot_deform_heap_tuple_internal(TupleTableSlot *slot, HeapTuple tuple,
*/
static pg_attribute_always_inline void
slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
- int natts)
+ int reqnatts)
{
bool hasnulls = HeapTupleHasNulls(tuple);
int attnum;
+ int natts;
uint32 off; /* offset in tuple data */
bool slow; /* can we use/set attcacheoff? */
/* We can only fetch as many attributes as the tuple has. */
- natts = Min(HeapTupleHeaderGetNatts(tuple->t_data), natts);
+ natts = Min(HeapTupleHeaderGetNatts(tuple->t_data), reqnatts);
/*
* Check whether the first call for this tuple, and initialize or restore
* loop state.
*/
attnum = slot->tts_nvalid;
+ slot->tts_nvalid = reqnatts;
if (attnum == 0)
{
/* Start from the first attribute */
@@ -1199,12 +1204,15 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
/*
* Save state for next execution
*/
- slot->tts_nvalid = attnum;
*offp = off;
if (slow)
slot->tts_flags |= TTS_FLAG_SLOW;
else
slot->tts_flags &= ~TTS_FLAG_SLOW;
+
+ /* Fetch any missing attrs and raise an error if reqnatts is invalid. */
+ if (unlikely(attnum < reqnatts))
+ slot_getmissingattrs(slot, attnum, reqnatts);
}
const TupleTableSlotOps TTSOpsVirtual = {
@@ -2058,34 +2066,36 @@ slot_getmissingattrs(TupleTableSlot *slot, int startAttNum, int lastAttNum)
{
AttrMissing *attrmiss = NULL;
+ /* Check for invalid attnums */
+ if (unlikely(lastAttNum > slot->tts_tupleDescriptor->natts))
+ elog(ERROR, "invalid attribute number %d", lastAttNum);
+
if (slot->tts_tupleDescriptor->constr)
attrmiss = slot->tts_tupleDescriptor->constr->missing;
if (!attrmiss)
{
/* no missing values array at all, so just fill everything in as NULL */
- memset(slot->tts_values + startAttNum, 0,
- (lastAttNum - startAttNum) * sizeof(Datum));
- memset(slot->tts_isnull + startAttNum, 1,
- (lastAttNum - startAttNum) * sizeof(bool));
+ for (int attnum = startAttNum; attnum < lastAttNum; attnum++)
+ {
+ slot->tts_values[attnum] = (Datum) 0;
+ slot->tts_isnull[attnum] = true;
+ }
}
else
{
- int missattnum;
-
- /* if there is a missing values array we must process them one by one */
- for (missattnum = startAttNum;
- missattnum < lastAttNum;
- missattnum++)
+ /* use attrmiss to set the missing values */
+ for (int attnum = startAttNum; attnum < lastAttNum; attnum++)
{
- slot->tts_values[missattnum] = attrmiss[missattnum].am_value;
- slot->tts_isnull[missattnum] = !attrmiss[missattnum].am_present;
+ slot->tts_values[attnum] = attrmiss[attnum].am_value;
+ slot->tts_isnull[attnum] = !attrmiss[attnum].am_present;
}
}
}
/*
- * slot_getsomeattrs_int - workhorse for slot_getsomeattrs()
+ * slot_getsomeattrs_int
+ * external function to call getsomeattrs() for use in JIT
*/
void
slot_getsomeattrs_int(TupleTableSlot *slot, int attnum)
@@ -2094,21 +2104,13 @@ slot_getsomeattrs_int(TupleTableSlot *slot, int attnum)
Assert(slot->tts_nvalid < attnum); /* checked in slot_getsomeattrs */
Assert(attnum > 0);
- if (unlikely(attnum > slot->tts_tupleDescriptor->natts))
- elog(ERROR, "invalid attribute number %d", attnum);
-
/* Fetch as many attributes as possible from the underlying tuple. */
slot->tts_ops->getsomeattrs(slot, attnum);
/*
- * If the underlying tuple doesn't have enough attributes, tuple
- * descriptor must have the missing attributes.
+ * Avoid putting new code here as that would prevent the compiler from
+ * using the sibling call optimization for the above function.
*/
- if (unlikely(slot->tts_nvalid < attnum))
- {
- slot_getmissingattrs(slot, slot->tts_nvalid, attnum);
- slot->tts_nvalid = attnum;
- }
}
/* ----------------------------------------------------------------
diff --git a/src/include/executor/tuptable.h b/src/include/executor/tuptable.h
index a2dfd707e78..3b09abbf99f 100644
--- a/src/include/executor/tuptable.h
+++ b/src/include/executor/tuptable.h
@@ -151,10 +151,12 @@ struct TupleTableSlotOps
/*
* Fill up first natts entries of tts_values and tts_isnull arrays with
- * values from the tuple contained in the slot. The function may be called
- * with natts more than the number of attributes available in the tuple,
- * in which case it should set tts_nvalid to the number of returned
- * columns.
+ * values from the tuple contained in the slot and set the slot's
+ * tts_nvalid to natts. The function may be called with an natts value
+ * more than the number of attributes available in the tuple, in which
+ * case the function must call slot_getmissingattrs() to populate the
+ * remaining attributes. The function must raise an ERROR if 'natts' is
+ * higher than the number of attributes in the slot's TupleDesc.
*/
void (*getsomeattrs) (TupleTableSlot *slot, int natts);
@@ -357,8 +359,9 @@ extern void slot_getsomeattrs_int(TupleTableSlot *slot, int attnum);
static inline void
slot_getsomeattrs(TupleTableSlot *slot, int attnum)
{
+ /* Populate slot with attributes up to 'attnum', if it's not already */
if (slot->tts_nvalid < attnum)
- slot_getsomeattrs_int(slot, attnum);
+ slot->tts_ops->getsomeattrs(slot, attnum);
}
/*
--
2.51.0
From a0d67378e4b1d30b4afead1a37d7cf93490dc656 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Wed, 21 Jan 2026 15:41:37 +1300
Subject: [PATCH v13 3/6] Add empty TupleDescFinalize() function
Currently does nothing, but will in a future commit.
---
contrib/dblink/dblink.c | 4 ++++
contrib/pg_buffercache/pg_buffercache_pages.c | 2 ++
contrib/pg_visibility/pg_visibility.c | 2 ++
src/backend/access/brin/brin_tuple.c | 1 +
src/backend/access/common/tupdesc.c | 13 +++++++++++++
src/backend/access/gin/ginutil.c | 1 +
src/backend/access/gist/gistscan.c | 1 +
src/backend/access/spgist/spgutils.c | 1 +
src/backend/access/transam/twophase.c | 1 +
src/backend/access/transam/xlogfuncs.c | 1 +
src/backend/backup/basebackup_copy.c | 3 +++
src/backend/catalog/index.c | 2 ++
src/backend/catalog/pg_publication.c | 1 +
src/backend/catalog/toasting.c | 6 ++++++
src/backend/commands/explain.c | 1 +
src/backend/commands/functioncmds.c | 1 +
src/backend/commands/sequence.c | 1 +
src/backend/commands/tablecmds.c | 4 ++++
src/backend/commands/wait.c | 1 +
src/backend/executor/execSRF.c | 2 ++
src/backend/executor/execTuples.c | 4 ++++
src/backend/executor/nodeFunctionscan.c | 2 ++
src/backend/parser/parse_relation.c | 4 +++-
src/backend/parser/parse_target.c | 2 ++
.../replication/libpqwalreceiver/libpqwalreceiver.c | 1 +
src/backend/replication/walsender.c | 5 +++++
src/backend/utils/adt/acl.c | 1 +
src/backend/utils/adt/genfile.c | 1 +
src/backend/utils/adt/lockfuncs.c | 1 +
src/backend/utils/adt/orderedsetaggs.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 5 +++++
src/backend/utils/adt/tsvector_op.c | 1 +
src/backend/utils/cache/relcache.c | 8 ++++++++
src/backend/utils/fmgr/funcapi.c | 6 ++++++
src/backend/utils/misc/guc_funcs.c | 5 +++++
src/include/access/tupdesc.h | 1 +
src/pl/plpgsql/src/pl_comp.c | 2 ++
.../test_custom_stats/test_custom_fixed_stats.c | 1 +
src/test/modules/test_predtest/test_predtest.c | 1 +
39 files changed, 100 insertions(+), 1 deletion(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 2498d80c8e7..4038950a6ef 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -881,6 +881,7 @@ materializeResult(FunctionCallInfo fcinfo, PGconn *conn, PGresult *res)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "status",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
ntuples = 1;
nfields = 1;
}
@@ -1044,6 +1045,7 @@ materializeQueryResult(FunctionCallInfo fcinfo,
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "status",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
attinmeta = TupleDescGetAttInMetadata(tupdesc);
oldcontext = MemoryContextSwitchTo(rsinfo->econtext->ecxt_per_query_memory);
@@ -1529,6 +1531,8 @@ dblink_get_pkey(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 2, "colname",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+
/*
* Generate attribute metadata needed later to produce tuples from raw
* C strings
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 89b86855243..a6b4fb5252b 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -174,6 +174,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
INT4OID, -1, 0);
+ TupleDescFinalize(tupledesc);
fctx->tupdesc = BlessTupleDesc(tupledesc);
/* Allocate NBuffers worth of BufferCachePagesRec records. */
@@ -442,6 +443,7 @@ pg_buffercache_os_pages_internal(FunctionCallInfo fcinfo, bool include_numa)
TupleDescInitEntry(tupledesc, (AttrNumber) 3, "numa_node",
INT4OID, -1, 0);
+ TupleDescFinalize(tupledesc);
fctx->tupdesc = BlessTupleDesc(tupledesc);
fctx->include_numa = include_numa;
diff --git a/contrib/pg_visibility/pg_visibility.c b/contrib/pg_visibility/pg_visibility.c
index 9bc3a784bf7..dfab0b64cf5 100644
--- a/contrib/pg_visibility/pg_visibility.c
+++ b/contrib/pg_visibility/pg_visibility.c
@@ -469,6 +469,8 @@ pg_visibility_tupdesc(bool include_blkno, bool include_pd)
TupleDescInitEntry(tupdesc, ++a, "pd_all_visible", BOOLOID, -1, 0);
Assert(a == maxattr);
+ TupleDescFinalize(tupdesc);
+
return BlessTupleDesc(tupdesc);
}
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 69c233c62eb..742ac089a28 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -84,6 +84,7 @@ brtuple_disk_tupdesc(BrinDesc *brdesc)
MemoryContextSwitchTo(oldcxt);
+ TupleDescFinalize(tupdesc);
brdesc->bd_disktdesc = tupdesc;
}
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index b69d10f0a45..2137385a833 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -221,6 +221,9 @@ CreateTupleDesc(int natts, Form_pg_attribute *attrs)
memcpy(TupleDescAttr(desc, i), attrs[i], ATTRIBUTE_FIXED_PART_SIZE);
populate_compact_attribute(desc, i);
}
+
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -265,6 +268,8 @@ CreateTupleDescCopy(TupleDesc tupdesc)
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -311,6 +316,8 @@ CreateTupleDescTruncatedCopy(TupleDesc tupdesc, int natts)
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -396,6 +403,8 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -438,6 +447,8 @@ TupleDescCopy(TupleDesc dst, TupleDesc src)
* source's refcount would be wrong in any case.)
*/
dst->tdrefcount = -1;
+
+ TupleDescFinalize(dst);
}
/*
@@ -1065,6 +1076,8 @@ BuildDescFromLists(const List *names, const List *types, const List *typmods, co
TupleDescInitEntryCollation(desc, attnum, attcollation);
}
+ TupleDescFinalize(desc);
+
return desc;
}
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index ff927279cc3..fe7b984ff32 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -129,6 +129,7 @@ initGinState(GinState *state, Relation index)
attr->attndims);
TupleDescInitEntryCollation(state->tupdesc[i], (AttrNumber) 2,
attr->attcollation);
+ TupleDescFinalize(state->tupdesc[i]);
}
/*
diff --git a/src/backend/access/gist/gistscan.c b/src/backend/access/gist/gistscan.c
index f23bc4a6757..c65f93abdae 100644
--- a/src/backend/access/gist/gistscan.c
+++ b/src/backend/access/gist/gistscan.c
@@ -201,6 +201,7 @@ gistrescan(IndexScanDesc scan, ScanKey key, int nkeys,
attno - 1)->atttypid,
-1, 0);
}
+ TupleDescFinalize(so->giststate->fetchTupdesc);
scan->xs_hitupdesc = so->giststate->fetchTupdesc;
/* Also create a memory context that will hold the returned tuples */
diff --git a/src/backend/access/spgist/spgutils.c b/src/backend/access/spgist/spgutils.c
index 9f5379b87ac..b246e8127db 100644
--- a/src/backend/access/spgist/spgutils.c
+++ b/src/backend/access/spgist/spgutils.c
@@ -340,6 +340,7 @@ getSpGistTupleDesc(Relation index, SpGistTypeDesc *keyType)
TupleDescCompactAttr(outTupDesc, i)->attcacheoff = -1;
populate_compact_attribute(outTupDesc, spgKeyColumn);
+ TupleDescFinalize(outTupDesc);
}
return outTupDesc;
}
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 55b9f38927d..d468c9774b3 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -745,6 +745,7 @@ pg_prepared_xact(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 5, "dbid",
OIDOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
/*
diff --git a/src/backend/access/transam/xlogfuncs.c b/src/backend/access/transam/xlogfuncs.c
index ecb3c8c0820..4e35311b2f3 100644
--- a/src/backend/access/transam/xlogfuncs.c
+++ b/src/backend/access/transam/xlogfuncs.c
@@ -430,6 +430,7 @@ pg_walfile_name_offset(PG_FUNCTION_ARGS)
TupleDescInitEntry(resultTupleDesc, (AttrNumber) 2, "file_offset",
INT4OID, -1, 0);
+ TupleDescFinalize(resultTupleDesc);
resultTupleDesc = BlessTupleDesc(resultTupleDesc);
/*
diff --git a/src/backend/backup/basebackup_copy.c b/src/backend/backup/basebackup_copy.c
index 07f58b39d8c..6c3453efd80 100644
--- a/src/backend/backup/basebackup_copy.c
+++ b/src/backend/backup/basebackup_copy.c
@@ -357,6 +357,8 @@ SendXlogRecPtrResult(XLogRecPtr ptr, TimeLineID tli)
*/
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "tli", INT8OID, -1, 0);
+ TupleDescFinalize(tupdesc);
+
/* send RowDescription */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -388,6 +390,7 @@ SendTablespaceList(List *tablespaces)
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 1, "spcoid", OIDOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "spclocation", TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 3, "size", INT8OID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* send RowDescription */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 5ee6389d39c..0e93ababa87 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -481,6 +481,8 @@ ConstructTupleDescriptor(Relation heapRelation,
populate_compact_attribute(indexTupDesc, i);
}
+ TupleDescFinalize(indexTupDesc);
+
return indexTupDesc;
}
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index aadc7c202c6..a79157c43bf 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -1322,6 +1322,7 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 4, "qual",
PG_NODE_TREEOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
funcctx->user_fctx = table_infos;
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index c78dcea98c1..078a1cf5127 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -229,6 +229,12 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attcompression = InvalidCompressionMethod;
TupleDescAttr(tupdesc, 2)->attcompression = InvalidCompressionMethod;
+ populate_compact_attribute(tupdesc, 0);
+ populate_compact_attribute(tupdesc, 1);
+ populate_compact_attribute(tupdesc, 2);
+
+ TupleDescFinalize(tupdesc);
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 93918a223b8..5f922c3f5c2 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -281,6 +281,7 @@ ExplainResultDesc(ExplainStmt *stmt)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "QUERY PLAN",
result_type, -1, 0);
+ TupleDescFinalize(tupdesc);
return tupdesc;
}
diff --git a/src/backend/commands/functioncmds.c b/src/backend/commands/functioncmds.c
index 242372b1e68..3afd762e9dc 100644
--- a/src/backend/commands/functioncmds.c
+++ b/src/backend/commands/functioncmds.c
@@ -2424,6 +2424,7 @@ CallStmtResultDesc(CallStmt *stmt)
-1,
0);
}
+ TupleDescFinalize(tupdesc);
}
return tupdesc;
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index e1b808bbb60..551667650ba 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -1808,6 +1808,7 @@ pg_get_sequence_data(PG_FUNCTION_ARGS)
BOOLOID, -1, 0);
TupleDescInitEntry(resultTupleDesc, (AttrNumber) 3, "page_lsn",
LSNOID, -1, 0);
+ TupleDescFinalize(resultTupleDesc);
resultTupleDesc = BlessTupleDesc(resultTupleDesc);
seqrel = try_relation_open(relid, AccessShareLock);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index cd6d720386f..a2e3b72f156 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -1038,6 +1038,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
}
}
+ TupleDescFinalize(descriptor);
+
/*
* For relations with table AM and partitioned tables, select access
* method to use: an explicitly indicated one, or (in the case of a
@@ -1466,6 +1468,8 @@ BuildDescForRelation(const List *columns)
populate_compact_attribute(desc, attnum - 1);
}
+ TupleDescFinalize(desc);
+
return desc;
}
diff --git a/src/backend/commands/wait.c b/src/backend/commands/wait.c
index 1290df10c6f..8e920a72372 100644
--- a/src/backend/commands/wait.c
+++ b/src/backend/commands/wait.c
@@ -338,5 +338,6 @@ WaitStmtResultDesc(WaitStmt *stmt)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "status",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
return tupdesc;
}
diff --git a/src/backend/executor/execSRF.c b/src/backend/executor/execSRF.c
index a0b111dc0e4..b481e50acfb 100644
--- a/src/backend/executor/execSRF.c
+++ b/src/backend/executor/execSRF.c
@@ -272,6 +272,7 @@ ExecMakeTableFunctionResult(SetExprState *setexpr,
funcrettype,
-1,
0);
+ TupleDescFinalize(tupdesc);
rsinfo.setDesc = tupdesc;
}
MemoryContextSwitchTo(oldcontext);
@@ -776,6 +777,7 @@ init_sexpr(Oid foid, Oid input_collation, Expr *node,
funcrettype,
-1,
0);
+ TupleDescFinalize(tupdesc);
sexpr->funcResultDesc = tupdesc;
sexpr->funcReturnsTuple = false;
}
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 7effe954286..07b248aa5f3 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -2175,6 +2175,8 @@ ExecTypeFromTLInternal(List *targetList, bool skipjunk)
cur_resno++;
}
+ TupleDescFinalize(typeInfo);
+
return typeInfo;
}
@@ -2209,6 +2211,8 @@ ExecTypeFromExprList(List *exprList)
cur_resno++;
}
+ TupleDescFinalize(typeInfo);
+
return typeInfo;
}
diff --git a/src/backend/executor/nodeFunctionscan.c b/src/backend/executor/nodeFunctionscan.c
index 63e605e1f81..feb82d64967 100644
--- a/src/backend/executor/nodeFunctionscan.c
+++ b/src/backend/executor/nodeFunctionscan.c
@@ -414,6 +414,7 @@ ExecInitFunctionScan(FunctionScan *node, EState *estate, int eflags)
TupleDescInitEntryCollation(tupdesc,
(AttrNumber) 1,
exprCollation(funcexpr));
+ TupleDescFinalize(tupdesc);
}
else
{
@@ -485,6 +486,7 @@ ExecInitFunctionScan(FunctionScan *node, EState *estate, int eflags)
0);
}
+ TupleDescFinalize(scan_tupdesc);
Assert(attno == natts);
}
diff --git a/src/backend/parser/parse_relation.c b/src/backend/parser/parse_relation.c
index e003db520de..9c415e166ee 100644
--- a/src/backend/parser/parse_relation.c
+++ b/src/backend/parser/parse_relation.c
@@ -1883,6 +1883,7 @@ addRangeTableEntryForFunction(ParseState *pstate,
TupleDescInitEntryCollation(tupdesc,
(AttrNumber) 1,
exprCollation(funcexpr));
+ TupleDescFinalize(tupdesc);
}
else if (functypclass == TYPEFUNC_RECORD)
{
@@ -1940,6 +1941,7 @@ addRangeTableEntryForFunction(ParseState *pstate,
i++;
}
+ TupleDescFinalize(tupdesc);
/*
* Ensure that the coldeflist defines a legal set of names (no
@@ -2008,7 +2010,7 @@ addRangeTableEntryForFunction(ParseState *pstate,
0);
/* no need to set collation */
}
-
+ TupleDescFinalize(tupdesc);
Assert(natts == totalatts);
}
else
diff --git a/src/backend/parser/parse_target.c b/src/backend/parser/parse_target.c
index 3bcfc1f5e3d..f57c4d41080 100644
--- a/src/backend/parser/parse_target.c
+++ b/src/backend/parser/parse_target.c
@@ -1572,6 +1572,8 @@ expandRecordVariable(ParseState *pstate, Var *var, int levelsup)
}
Assert(lname == NULL && lvar == NULL); /* lists same length? */
+ TupleDescFinalize(tupleDesc);
+
return tupleDesc;
}
diff --git a/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c b/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
index 7c8639b32e9..9f04c9ed25d 100644
--- a/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
+++ b/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
@@ -1073,6 +1073,7 @@ libpqrcv_processTuples(PGresult *pgres, WalRcvExecResult *walres,
for (coln = 0; coln < nRetTypes; coln++)
TupleDescInitEntry(walres->tupledesc, (AttrNumber) coln + 1,
PQfname(pgres, coln), retTypes[coln], -1, 0);
+ TupleDescFinalize(walres->tupledesc);
attinmeta = TupleDescGetAttInMetadata(walres->tupledesc);
/* No point in doing more here if there were no tuples returned. */
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c
index 79fc192b171..376ff46340d 100644
--- a/src/backend/replication/walsender.c
+++ b/src/backend/replication/walsender.c
@@ -452,6 +452,7 @@ IdentifySystem(void)
TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 4, "dbname",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -497,6 +498,7 @@ ReadReplicationSlot(ReadReplicationSlotCmd *cmd)
/* TimeLineID is unsigned, so int4 is not wide enough. */
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 3, "restart_tli",
INT8OID, -1, 0);
+ TupleDescFinalize(tupdesc);
memset(nulls, true, READ_REPLICATION_SLOT_COLS * sizeof(bool));
@@ -599,6 +601,7 @@ SendTimeLineHistory(TimeLineHistoryCmd *cmd)
tupdesc = CreateTemplateTupleDesc(2);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 1, "filename", TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "content", TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
TLHistoryFileName(histfname, cmd->timeline);
TLHistoryFilePath(path, cmd->timeline);
@@ -1016,6 +1019,7 @@ StartReplication(StartReplicationCmd *cmd)
INT8OID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "next_tli_startpos",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuple */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -1370,6 +1374,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd)
TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 4, "output_plugin",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
diff --git a/src/backend/utils/adt/acl.c b/src/backend/utils/adt/acl.c
index 071e3f2c49e..e210d6472be 100644
--- a/src/backend/utils/adt/acl.c
+++ b/src/backend/utils/adt/acl.c
@@ -1831,6 +1831,7 @@ aclexplode(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 4, "is_grantable",
BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
/* allocate memory for user context */
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index c083608b1d5..bfb949401d0 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -454,6 +454,7 @@ pg_stat_file(PG_FUNCTION_ARGS)
"creation", TIMESTAMPTZOID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 6,
"isdir", BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
memset(isnull, false, sizeof(isnull));
diff --git a/src/backend/utils/adt/lockfuncs.c b/src/backend/utils/adt/lockfuncs.c
index 9dadd6da672..4481c354fd6 100644
--- a/src/backend/utils/adt/lockfuncs.c
+++ b/src/backend/utils/adt/lockfuncs.c
@@ -146,6 +146,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 16, "waitstart",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
/*
diff --git a/src/backend/utils/adt/orderedsetaggs.c b/src/backend/utils/adt/orderedsetaggs.c
index 3b6da8e36ac..fd8b8676470 100644
--- a/src/backend/utils/adt/orderedsetaggs.c
+++ b/src/backend/utils/adt/orderedsetaggs.c
@@ -233,6 +233,7 @@ ordered_set_startup(FunctionCallInfo fcinfo, bool use_tuples)
-1,
0);
+ TupleDescFinalize(newdesc);
FreeTupleDesc(qstate->tupdesc);
qstate->tupdesc = newdesc;
}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 5ac022274a7..bad5642d9c9 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -770,6 +770,7 @@ pg_stat_get_backend_subxact(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 2, "subxact_overflow",
BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
if ((local_beentry = pgstat_get_local_beentry_by_proc_number(procNumber)) != NULL)
@@ -1671,6 +1672,7 @@ pg_stat_wal_build_tuple(PgStat_WalCounters wal_counters,
TupleDescInitEntry(tupdesc, (AttrNumber) 6, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
/* Fill values and NULLs */
@@ -2098,6 +2100,7 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 7, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
/* Get statistics about the archiver process */
@@ -2179,6 +2182,7 @@ pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
TIMESTAMPTZOID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 13, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
namestrcpy(&slotname, text_to_cstring(slotname_text));
@@ -2266,6 +2270,7 @@ pg_stat_get_subscription_stats(PG_FUNCTION_ARGS)
INT8OID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 13, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
if (!subentry)
diff --git a/src/backend/utils/adt/tsvector_op.c b/src/backend/utils/adt/tsvector_op.c
index 71c7c7d3b3c..d8dece42b9b 100644
--- a/src/backend/utils/adt/tsvector_op.c
+++ b/src/backend/utils/adt/tsvector_op.c
@@ -651,6 +651,7 @@ tsvector_unnest(PG_FUNCTION_ARGS)
TEXTARRAYOID, -1, 0);
if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
elog(ERROR, "return type must be a row type");
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = tupdesc;
funcctx->user_fctx = PG_GETARG_TSVECTOR_COPY(0);
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a1c88c6b1b6..d27ac216e6d 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -729,6 +729,8 @@ RelationBuildTupleDesc(Relation relation)
pfree(constr);
relation->rd_att->constr = NULL;
}
+
+ TupleDescFinalize(relation->rd_att);
}
/*
@@ -1985,6 +1987,7 @@ formrdesc(const char *relationName, Oid relationReltype,
/* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
TupleDescCompactAttr(relation->rd_att, 0)->attcacheoff = 0;
+ TupleDescFinalize(relation->rd_att);
/* mark not-null status */
if (has_not_null)
@@ -3688,6 +3691,8 @@ RelationBuildLocalRelation(const char *relname,
for (i = 0; i < natts; i++)
TupleDescAttr(rel->rd_att, i)->attrelid = relid;
+ TupleDescFinalize(rel->rd_att);
+
rel->rd_rel->reltablespace = reltablespace;
if (mapped_relation)
@@ -4443,6 +4448,7 @@ BuildHardcodedDescriptor(int natts, const FormData_pg_attribute *attrs)
/* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
TupleDescCompactAttr(result, 0)->attcacheoff = 0;
+ TupleDescFinalize(result);
/* Note: we don't bother to set up a TupleConstr entry */
@@ -6291,6 +6297,8 @@ load_relcache_init_file(bool shared)
populate_compact_attribute(rel->rd_att, i);
}
+ TupleDescFinalize(rel->rd_att);
+
/* next read the access method specific field */
if (fread(&len, 1, sizeof(len), fp) != sizeof(len))
goto read_failed;
diff --git a/src/backend/utils/fmgr/funcapi.c b/src/backend/utils/fmgr/funcapi.c
index 8a934ea8dca..516d02cfb82 100644
--- a/src/backend/utils/fmgr/funcapi.c
+++ b/src/backend/utils/fmgr/funcapi.c
@@ -340,6 +340,8 @@ get_expr_result_type(Node *expr,
exprCollation(col));
i++;
}
+ TupleDescFinalize(tupdesc);
+
if (resultTypeId)
*resultTypeId = rexpr->row_typeid;
if (resultTupleDesc)
@@ -1044,6 +1046,7 @@ resolve_polymorphic_tupdesc(TupleDesc tupdesc, oidvector *declared_args,
}
}
+ TupleDescFinalize(tupdesc);
return true;
}
@@ -1853,6 +1856,8 @@ build_function_result_tupdesc_d(char prokind,
0);
}
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -1970,6 +1975,7 @@ TypeGetTupleDesc(Oid typeoid, List *colaliases)
typeoid,
-1,
0);
+ TupleDescFinalize(tupdesc);
}
else if (functypclass == TYPEFUNC_RECORD)
{
diff --git a/src/backend/utils/misc/guc_funcs.c b/src/backend/utils/misc/guc_funcs.c
index 8524dd3a981..472cb5393ce 100644
--- a/src/backend/utils/misc/guc_funcs.c
+++ b/src/backend/utils/misc/guc_funcs.c
@@ -444,6 +444,7 @@ GetPGVariableResultDesc(const char *name)
TupleDescInitEntry(tupdesc, (AttrNumber) 1, varname,
TEXTOID, -1, 0);
}
+ TupleDescFinalize(tupdesc);
return tupdesc;
}
@@ -465,6 +466,7 @@ ShowGUCConfigOption(const char *name, DestReceiver *dest)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 1, varname,
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -499,6 +501,7 @@ ShowAllGUCConfig(DestReceiver *dest)
TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 3, "description",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -934,6 +937,8 @@ show_all_settings(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 17, "pending_restart",
BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+
/*
* Generate attribute metadata needed later to produce tuples from raw
* C strings
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index d46cdbf7a3c..595413dbbc5 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -195,6 +195,7 @@ extern TupleDesc CreateTupleDescTruncatedCopy(TupleDesc tupdesc, int natts);
extern TupleDesc CreateTupleDescCopyConstr(TupleDesc tupdesc);
+#define TupleDescFinalize(d) ((void) 0)
#define TupleDescSize(src) \
(offsetof(struct TupleDescData, compact_attrs) + \
(src)->natts * sizeof(CompactAttribute) + \
diff --git a/src/pl/plpgsql/src/pl_comp.c b/src/pl/plpgsql/src/pl_comp.c
index 5ecc7766757..b72c963b3be 100644
--- a/src/pl/plpgsql/src/pl_comp.c
+++ b/src/pl/plpgsql/src/pl_comp.c
@@ -1912,6 +1912,8 @@ build_row_from_vars(PLpgSQL_variable **vars, int numvars)
TupleDescInitEntryCollation(row->rowtupdesc, i + 1, typcoll);
}
+ TupleDescFinalize(row->rowtupdesc);
+
return row;
}
diff --git a/src/test/modules/test_custom_stats/test_custom_fixed_stats.c b/src/test/modules/test_custom_stats/test_custom_fixed_stats.c
index 485e08e5c19..f9e7c717280 100644
--- a/src/test/modules/test_custom_stats/test_custom_fixed_stats.c
+++ b/src/test/modules/test_custom_stats/test_custom_fixed_stats.c
@@ -206,6 +206,7 @@ test_custom_stats_fixed_report(PG_FUNCTION_ARGS)
INT8OID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 2, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
values[0] = Int64GetDatum(stats->numcalls);
diff --git a/src/test/modules/test_predtest/test_predtest.c b/src/test/modules/test_predtest/test_predtest.c
index 679a5de456d..48ca2a4ea70 100644
--- a/src/test/modules/test_predtest/test_predtest.c
+++ b/src/test/modules/test_predtest/test_predtest.c
@@ -230,6 +230,7 @@ test_predtest(PG_FUNCTION_ARGS)
"s_r_holds", BOOLOID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 8,
"w_r_holds", BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
tupdesc = BlessTupleDesc(tupdesc);
values[0] = BoolGetDatum(strong_implied_by);
--
2.51.0
From ab5fc2c7de7c6661d568e677eaad37c43977bef8 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Tue, 31 Dec 2024 09:19:24 +1300
Subject: [PATCH v13 4/6] Optimize tuple deformation
This commit includes various optimizations to improve the performance of
tuple deformation.
We now precalculate CompactAttribute's attcacheoff, which allows us to
remove the code from the deform routines which was setting the
attcacheoff. Setting the attcacheoff is handled by TupleDescFinalize(),
which must be called before the TupleDesc is used for anything. Having
this TupleDescFinalize() function means we can store the first
attribute in the TupleDesc which does not have an offset cached. That
allows us to add a dedicated deforming loop to deform all attributes up
to the final one with an attcacheoff set, or up to the first NULL
attribute, whichever comes first.
We also record the maximum attribute number which is guaranteed to exist
in the tuple, that is, has a NOT NULL constraint and isn't an
atthasmissing attribute. When deforming only attributes prior to the
guaranteed attnum, we've no need to access the tuple's natt count. As an
additional optimization, we only count fixed-width columns when
calculating the maximum guaranteed column as this eliminates the need to
emit code to fetch byref types in the deformation loop for guaranteed
attributes.
Some locations in the code deform tuples that have yet to go through NOT
NULL constraint validation. We're unable to perform the guaranteed
attribute optimization when that's the case. The optimization is opt-in
via the TupleTableSlot using the TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS
flag.
This commit also adds a more efficient way of populating the isnull
array by using a bit-wise trick which performs multiplication on the
inverse of the tuple's bitmap byte and masking out all but the lower bit
of each of the boolean's byte. This results in much more optimal code
when compared to determining the NULLness via att_isnull(). 8 isnull
elements are processed at once using this method, which means we need to
round the tts_isnull array size up to the next 8 bytes. The palloc code
does this anyway, but the round-up needed to be formalized so as not to
overwrite the sentinel byte in debug builds.
---
src/backend/access/common/heaptuple.c | 360 +++++++--------
src/backend/access/common/indextuple.c | 363 +++++++--------
src/backend/access/common/tupdesc.c | 51 +++
src/backend/access/spgist/spgutils.c | 3 -
src/backend/executor/execMain.c | 8 +-
src/backend/executor/execTuples.c | 415 ++++++++++--------
src/backend/executor/execUtils.c | 2 +-
src/backend/executor/nodeAgg.c | 2 +-
src/backend/executor/nodeBitmapHeapscan.c | 5 +-
src/backend/executor/nodeCtescan.c | 2 +-
src/backend/executor/nodeCustom.c | 4 +-
src/backend/executor/nodeForeignscan.c | 4 +-
src/backend/executor/nodeFunctionscan.c | 2 +-
src/backend/executor/nodeIndexonlyscan.c | 7 +-
src/backend/executor/nodeIndexscan.c | 5 +-
.../executor/nodeNamedtuplestorescan.c | 2 +-
src/backend/executor/nodeSamplescan.c | 6 +-
src/backend/executor/nodeSeqscan.c | 3 +-
src/backend/executor/nodeSubqueryscan.c | 3 +-
src/backend/executor/nodeTableFuncscan.c | 2 +-
src/backend/executor/nodeTidrangescan.c | 5 +-
src/backend/executor/nodeTidscan.c | 5 +-
src/backend/executor/nodeValuesscan.c | 2 +-
src/backend/executor/nodeWorktablescan.c | 2 +-
src/backend/jit/llvm/llvmjit_deform.c | 6 -
src/backend/replication/pgoutput/pgoutput.c | 4 +-
src/backend/utils/cache/relcache.c | 12 -
src/include/access/tupdesc.h | 20 +-
src/include/access/tupmacs.h | 224 +++++++++-
src/include/executor/executor.h | 3 +-
src/include/executor/tuptable.h | 28 +-
src/test/modules/deform_bench/deform_bench.c | 4 +-
32 files changed, 900 insertions(+), 664 deletions(-)
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 11bec20e82e..b2ac7fef35b 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -498,19 +498,7 @@ heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc)
* nocachegetattr
*
* This only gets called from fastgetattr(), in cases where we
- * can't use a cacheoffset and the value is not null.
- *
- * This caches attribute offsets in the attribute descriptor.
- *
- * An alternative way to speed things up would be to cache offsets
- * with the tuple, but that seems more difficult unless you take
- * the storage hit of actually putting those offsets into the
- * tuple you send to disk. Yuck.
- *
- * This scheme will be slightly slower than that, but should
- * perform well for queries which hit large #'s of tuples. After
- * you cache the offsets once, examining all the other tuples using
- * the same attribute descriptor will go much quicker. -cim 5/4/91
+ * can't use the attcacheoff and the value is not null.
*
* NOTE: if you need to change this code, see also heap_deform_tuple.
* Also see nocache_index_getattr, which is the same code for index
@@ -522,194 +510,125 @@ nocachegetattr(HeapTuple tup,
int attnum,
TupleDesc tupleDesc)
{
+ CompactAttribute *cattr;
HeapTupleHeader td = tup->t_data;
char *tp; /* ptr to data part of tuple */
bits8 *bp = td->t_bits; /* ptr to null bitmap in tuple */
- bool slow = false; /* do we have to walk attrs? */
int off; /* current offset within data */
+ int startAttr;
+ int firstNullAttr;
+ int i;
+ bool hasnulls = HeapTupleHasNulls(tup);
- /* ----------------
- * Three cases:
- *
- * 1: No nulls and no variable-width attributes.
- * 2: Has a null or a var-width AFTER att.
- * 3: Has nulls or var-widths BEFORE att.
- * ----------------
- */
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
attnum--;
- if (!HeapTupleNoNulls(tup))
+ /*
+ * To minimize the number of attributes we need to look at, start walking
+ * the tuple at the attribute with the highest attcacheoff prior to attnum
+ * or the first NULL attribute prior to attnum, whichever comes first.
+ */
+ if (hasnulls)
+ firstNullAttr = first_null_attr(bp, attnum);
+ else
+ firstNullAttr = attnum;
+
+ if (tupleDesc->firstNonCachedOffsetAttr > 0)
{
/*
- * there's a null somewhere in the tuple
- *
- * check to see if any preceding bits are null...
+ * Start at the highest attcacheoff attribute with no NULLs in prior
+ * attributes.
*/
- int byte = attnum >> 3;
- int finalbit = attnum & 0x07;
-
- /* check for nulls "before" final bit of last byte */
- if ((~bp[byte]) & ((1 << finalbit) - 1))
- slow = true;
- else
- {
- /* check for nulls in any "earlier" bytes */
- int i;
-
- for (i = 0; i < byte; i++)
- {
- if (bp[i] != 0xFF)
- {
- slow = true;
- break;
- }
- }
- }
+ startAttr = Min(tupleDesc->firstNonCachedOffsetAttr - 1, firstNullAttr);
+ off = TupleDescCompactAttr(tupleDesc, startAttr)->attcacheoff;
+ }
+ else
+ {
+ /* Otherwise, start at the beginning... */
+ startAttr = 0;
+ off = 0;
}
tp = (char *) td + td->t_hoff;
- if (!slow)
+ /*
+ * Calculate 'off' up to the first NULL attr. We use two cheaper loops
+ * when the tuple has no variable-width columns. When variable-width
+ * columns exists, we use att_addlength_pointer() to move the offset
+ * beyond the current attribute.
+ */
+ if (!HeapTupleHasVarWidth(tup))
{
- CompactAttribute *att;
-
- /*
- * If we get here, there are no nulls up to and including the target
- * attribute. If we have a cached offset, we can use it.
- */
- att = TupleDescCompactAttr(tupleDesc, attnum);
- if (att->attcacheoff >= 0)
- return fetchatt(att, tp + att->attcacheoff);
-
- /*
- * Otherwise, check for non-fixed-length attrs up to and including
- * target. If there aren't any, it's safe to cheaply initialize the
- * cached offsets for these attrs.
- */
- if (HeapTupleHasVarWidth(tup))
+ for (i = startAttr; i < firstNullAttr; i++)
{
- int j;
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- for (j = 0; j <= attnum; j++)
- {
- if (TupleDescCompactAttr(tupleDesc, j)->attlen <= 0)
- {
- slow = true;
- break;
- }
- }
+ off = att_nominal_alignby(off, cattr->attalignby);
+ off += cattr->attlen;
}
- }
-
- if (!slow)
- {
- int natts = tupleDesc->natts;
- int j = 1;
-
- /*
- * If we get here, we have a tuple with no nulls or var-widths up to
- * and including the target attribute, so we can use the cached offset
- * ... only we don't have it yet, or we'd not have got here. Since
- * it's cheap to compute offsets for fixed-width columns, we take the
- * opportunity to initialize the cached offsets for *all* the leading
- * fixed-width columns, in hope of avoiding future visits to this
- * routine.
- */
- TupleDescCompactAttr(tupleDesc, 0)->attcacheoff = 0;
-
- /* we might have set some offsets in the slow path previously */
- while (j < natts && TupleDescCompactAttr(tupleDesc, j)->attcacheoff > 0)
- j++;
-
- off = TupleDescCompactAttr(tupleDesc, j - 1)->attcacheoff +
- TupleDescCompactAttr(tupleDesc, j - 1)->attlen;
- for (; j < natts; j++)
+ for (; i < attnum; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, j);
+ if (att_isnull(i, bp))
+ continue;
- if (att->attlen <= 0)
- break;
-
- off = att_nominal_alignby(off, att->attalignby);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- att->attcacheoff = off;
-
- off += att->attlen;
+ off = att_nominal_alignby(off, cattr->attalignby);
+ off += cattr->attlen;
}
-
- Assert(j > attnum);
-
- off = TupleDescCompactAttr(tupleDesc, attnum)->attcacheoff;
}
else
{
- bool usecache = true;
- int i;
-
- /*
- * Now we know that we have to walk the tuple CAREFULLY. But we still
- * might be able to cache some offsets for next time.
- *
- * Note - This loop is a little tricky. For each non-null attribute,
- * we have to first account for alignment padding before the attr,
- * then advance over the attr based on its length. Nulls have no
- * storage and no alignment padding either. We can use/set
- * attcacheoff until we reach either a null or a var-width attribute.
- */
- off = 0;
- for (i = 0;; i++) /* loop exit is at "break" */
+ for (i = startAttr; i < firstNullAttr; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, i);
+ int attlen;
- if (HeapTupleHasNulls(tup) && att_isnull(i, bp))
- {
- usecache = false;
- continue; /* this cannot be the target att */
- }
+ cattr = TupleDescCompactAttr(tupleDesc, i);
+ attlen = cattr->attlen;
- /* If we know the next offset, we can skip the rest */
- if (usecache && att->attcacheoff >= 0)
- off = att->attcacheoff;
- else if (att->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be
- * no pad bytes in any case: then the offset will be valid for
- * either an aligned or unaligned value.
- */
- if (usecache &&
- off == att_nominal_alignby(off, att->attalignby))
- att->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, att->attalignby, -1,
- tp + off);
- usecache = false;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, att->attalignby);
+ /*
+ * cstrings don't exist in heap tuples. Use pg_assume to instruct
+ * the compiler not to emit the cstring-related code in
+ * att_addlength_pointer().
+ */
+ pg_assume(attlen > 0 || attlen == -1);
- if (usecache)
- att->attcacheoff = off;
- }
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ attlen,
+ tp + off);
+ off = att_addlength_pointer(off, attlen, tp + off);
+ }
- if (i == attnum)
- break;
+ for (; i < attnum; i++)
+ {
+ int attlen;
- off = att_addlength_pointer(off, att->attlen, tp + off);
+ if (att_isnull(i, bp))
+ continue;
- if (usecache && att->attlen <= 0)
- usecache = false;
+ cattr = TupleDescCompactAttr(tupleDesc, i);
+ attlen = cattr->attlen;
+
+ /* As above, heaptuples have no cstrings */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ off = att_pointer_alignby(off, cattr->attalignby, attlen,
+ tp + off);
+ off = att_addlength_pointer(off, attlen, tp + off);
}
}
- return fetchatt(TupleDescCompactAttr(tupleDesc, attnum), tp + off);
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+
+ return fetchatt(cattr, tp + off);
}
/* ----------------
@@ -1347,6 +1266,7 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,
Datum *values, bool *isnull)
{
HeapTupleHeader tup = tuple->t_data;
+ CompactAttribute *cattr;
bool hasnulls = HeapTupleHasNulls(tuple);
int tdesc_natts = tupleDesc->natts;
int natts; /* number of atts to extract */
@@ -1354,70 +1274,98 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,
char *tp; /* ptr to tuple data */
uint32 off; /* offset in tuple data */
bits8 *bp = tup->t_bits; /* ptr to null bitmap in tuple */
- bool slow = false; /* can we use/set attcacheoff? */
+ int firstNonCacheOffsetAttr;
+ int firstNullAttr;
natts = HeapTupleHeaderGetNatts(tup);
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
+
/*
* In inheritance situations, it is possible that the given tuple actually
* has more fields than the caller is expecting. Don't run off the end of
* the caller's arrays.
*/
natts = Min(natts, tdesc_natts);
+ firstNonCacheOffsetAttr = Min(tupleDesc->firstNonCachedOffsetAttr, natts);
+
+ if (hasnulls)
+ {
+ firstNullAttr = first_null_attr(bp, natts);
+
+ /*
+ * XXX: it'd be nice to use populate_isnull_array() here, but that
+ * requires that the isnull array's size is rounded up to the next
+ * multiple of 8. Doing that would require adjusting many locations
+ * that allocate the array.
+ */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, firstNullAttr);
+ }
+ else
+ firstNullAttr = natts;
tp = (char *) tup + tup->t_hoff;
+ attnum = 0;
- off = 0;
+ if (firstNonCacheOffsetAttr > 0)
+ {
+#ifdef USE_ASSERT_CHECKING
+ /* In Assert enabled builds, verify attcacheoff is correct */
+ int offcheck = 0;
+#endif
+ do
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ off = cattr->attcacheoff;
- for (attnum = 0; attnum < natts; attnum++)
+#ifdef USE_ASSERT_CHECKING
+ offcheck = att_nominal_alignby(offcheck, cattr->attalignby);
+ Assert(offcheck == cattr->attcacheoff);
+ offcheck += cattr->attlen;
+#endif
+
+ values[attnum] = fetch_att_noerr(tp + off,
+ cattr->attbyval,
+ cattr->attlen);
+ } while (++attnum < firstNonCacheOffsetAttr);
+ off += cattr->attlen;
+ }
+ else
+ off = 0;
+
+ for (; attnum < firstNullAttr; attnum++)
{
- CompactAttribute *thisatt = TupleDescCompactAttr(tupleDesc, attnum);
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
+ }
+
+ for (; attnum < natts; attnum++)
+ {
+ Assert(hasnulls);
- if (hasnulls && att_isnull(attnum, bp))
+ if (att_isnull(attnum, bp))
{
values[attnum] = (Datum) 0;
isnull[attnum] = true;
- slow = true; /* can't use attcacheoff anymore */
continue;
}
isnull[attnum] = false;
-
- if (!slow && thisatt->attcacheoff >= 0)
- off = thisatt->attcacheoff;
- else if (thisatt->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be no
- * pad bytes in any case: then the offset will be valid for either
- * an aligned or unaligned value.
- */
- if (!slow &&
- off == att_nominal_alignby(off, thisatt->attalignby))
- thisatt->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, thisatt->attalignby, -1,
- tp + off);
- slow = true;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, thisatt->attalignby);
-
- if (!slow)
- thisatt->attcacheoff = off;
- }
-
- values[attnum] = fetchatt(thisatt, tp + off);
-
- off = att_addlength_pointer(off, thisatt->attlen, tp + off);
-
- if (thisatt->attlen <= 0)
- slow = true; /* can't use attcacheoff anymore */
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
}
/*
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index d6350201e01..8c410853191 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -223,18 +223,6 @@ index_form_tuple_context(TupleDesc tupleDescriptor,
*
* This gets called from index_getattr() macro, and only in cases
* where we can't use cacheoffset and the value is not null.
- *
- * This caches attribute offsets in the attribute descriptor.
- *
- * An alternative way to speed things up would be to cache offsets
- * with the tuple, but that seems more difficult unless you take
- * the storage hit of actually putting those offsets into the
- * tuple you send to disk. Yuck.
- *
- * This scheme will be slightly slower than that, but should
- * perform well for queries which hit large #'s of tuples. After
- * you cache the offsets once, examining all the other tuples using
- * the same attribute descriptor will go much quicker. -cim 5/4/91
* ----------------
*/
Datum
@@ -242,205 +230,124 @@ nocache_index_getattr(IndexTuple tup,
int attnum,
TupleDesc tupleDesc)
{
+ CompactAttribute *cattr;
char *tp; /* ptr to data part of tuple */
bits8 *bp = NULL; /* ptr to null bitmap in tuple */
- bool slow = false; /* do we have to walk attrs? */
int data_off; /* tuple data offset */
int off; /* current offset within data */
+ int startAttr;
+ int firstNullAttr;
+ bool hasnulls = IndexTupleHasNulls(tup);
+ int i;
- /* ----------------
- * Three cases:
- *
- * 1: No nulls and no variable-width attributes.
- * 2: Has a null or a var-width AFTER att.
- * 3: Has nulls or var-widths BEFORE att.
- * ----------------
- */
-
- data_off = IndexInfoFindDataOffset(tup->t_info);
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
attnum--;
- if (IndexTupleHasNulls(tup))
- {
- /*
- * there's a null somewhere in the tuple
- *
- * check to see if desired att is null
- */
+ data_off = IndexInfoFindDataOffset(tup->t_info);
+ tp = (char *) tup + data_off;
- /* XXX "knows" t_bits are just after fixed tuple header! */
+ /*
+ * To minimize the number of attributes we need to look at, start walking
+ * the tuple at the attribute with the highest attcacheoff prior to attnum
+ * or the first NULL attribute prior to attnum, whichever comes first.
+ */
+ if (hasnulls)
+ {
bp = (bits8 *) ((char *) tup + sizeof(IndexTupleData));
-
- /*
- * Now check to see if any preceding bits are null...
- */
- {
- int byte = attnum >> 3;
- int finalbit = attnum & 0x07;
-
- /* check for nulls "before" final bit of last byte */
- if ((~bp[byte]) & ((1 << finalbit) - 1))
- slow = true;
- else
- {
- /* check for nulls in any "earlier" bytes */
- int i;
-
- for (i = 0; i < byte; i++)
- {
- if (bp[i] != 0xFF)
- {
- slow = true;
- break;
- }
- }
- }
- }
+ firstNullAttr = first_null_attr(bp, attnum);
}
+ else
+ firstNullAttr = attnum;
- tp = (char *) tup + data_off;
-
- if (!slow)
+ if (tupleDesc->firstNonCachedOffsetAttr > 0)
{
- CompactAttribute *att;
-
- /*
- * If we get here, there are no nulls up to and including the target
- * attribute. If we have a cached offset, we can use it.
- */
- att = TupleDescCompactAttr(tupleDesc, attnum);
- if (att->attcacheoff >= 0)
- return fetchatt(att, tp + att->attcacheoff);
-
/*
- * Otherwise, check for non-fixed-length attrs up to and including
- * target. If there aren't any, it's safe to cheaply initialize the
- * cached offsets for these attrs.
+ * Start at the highest attcacheoff attribute with no NULLs in prior
+ * attributes.
*/
- if (IndexTupleHasVarwidths(tup))
- {
- int j;
-
- for (j = 0; j <= attnum; j++)
- {
- if (TupleDescCompactAttr(tupleDesc, j)->attlen <= 0)
- {
- slow = true;
- break;
- }
- }
- }
+ startAttr = Min(tupleDesc->firstNonCachedOffsetAttr - 1, firstNullAttr);
+ off = TupleDescCompactAttr(tupleDesc, startAttr)->attcacheoff;
}
-
- if (!slow)
+ else
{
- int natts = tupleDesc->natts;
- int j = 1;
-
- /*
- * If we get here, we have a tuple with no nulls or var-widths up to
- * and including the target attribute, so we can use the cached offset
- * ... only we don't have it yet, or we'd not have got here. Since
- * it's cheap to compute offsets for fixed-width columns, we take the
- * opportunity to initialize the cached offsets for *all* the leading
- * fixed-width columns, in hope of avoiding future visits to this
- * routine.
- */
- TupleDescCompactAttr(tupleDesc, 0)->attcacheoff = 0;
+ /* Otherwise, start at the beginning... */
+ startAttr = 0;
+ off = 0;
+ }
- /* we might have set some offsets in the slow path previously */
- while (j < natts && TupleDescCompactAttr(tupleDesc, j)->attcacheoff > 0)
- j++;
+ /*
+ * Calculate 'off' up to the first NULL attr. We use two cheaper loops
+ * when the tuple has no variable-width columns. When variable-width
+ * columns exists, we use att_addlength_pointer() to move the offset
+ * beyond the current attribute.
+ */
+ if (IndexTupleHasVarwidths(tup))
+ {
+ /* Calculate the offset up until the first NULL */
+ for (i = startAttr; i < firstNullAttr; i++)
+ {
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- off = TupleDescCompactAttr(tupleDesc, j - 1)->attcacheoff +
- TupleDescCompactAttr(tupleDesc, j - 1)->attlen;
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+ off = att_addlength_pointer(off, cattr->attlen, tp + off);
+ }
- for (; j < natts; j++)
+ /* Calculate the offset for any remaining columns. */
+ for (; i < attnum; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, j);
+ Assert(hasnulls);
- if (att->attlen <= 0)
- break;
+ if (att_isnull(i, bp))
+ continue;
- off = att_nominal_alignby(off, att->attalignby);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- att->attcacheoff = off;
-
- off += att->attlen;
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+ off = att_addlength_pointer(off, cattr->attlen, tp + off);
}
-
- Assert(j > attnum);
-
- off = TupleDescCompactAttr(tupleDesc, attnum)->attcacheoff;
}
else
{
- bool usecache = true;
- int i;
+ /* Handle tuples with only fixed-width attributes */
- /*
- * Now we know that we have to walk the tuple CAREFULLY. But we still
- * might be able to cache some offsets for next time.
- *
- * Note - This loop is a little tricky. For each non-null attribute,
- * we have to first account for alignment padding before the attr,
- * then advance over the attr based on its length. Nulls have no
- * storage and no alignment padding either. We can use/set
- * attcacheoff until we reach either a null or a var-width attribute.
- */
- off = 0;
- for (i = 0;; i++) /* loop exit is at "break" */
+ /* Calculate the offset up until the first NULL */
+ for (i = startAttr; i < firstNullAttr; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, i);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- if (IndexTupleHasNulls(tup) && att_isnull(i, bp))
- {
- usecache = false;
- continue; /* this cannot be the target att */
- }
-
- /* If we know the next offset, we can skip the rest */
- if (usecache && att->attcacheoff >= 0)
- off = att->attcacheoff;
- else if (att->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be
- * no pad bytes in any case: then the offset will be valid for
- * either an aligned or unaligned value.
- */
- if (usecache &&
- off == att_nominal_alignby(off, att->attalignby))
- att->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, att->attalignby, -1,
- tp + off);
- usecache = false;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, att->attalignby);
+ Assert(cattr->attlen > 0);
+ off = att_nominal_alignby(off, cattr->attalignby);
+ off += cattr->attlen;
+ }
- if (usecache)
- att->attcacheoff = off;
- }
+ /* Calculate the offset for any remaining columns. */
+ for (; i < attnum; i++)
+ {
+ Assert(hasnulls);
- if (i == attnum)
- break;
+ if (att_isnull(i, bp))
+ continue;
- off = att_addlength_pointer(off, att->attlen, tp + off);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- if (usecache && att->attlen <= 0)
- usecache = false;
+ Assert(cattr->attlen > 0);
+ off = att_nominal_alignby(off, cattr->attalignby);
+ off += cattr->attlen;
}
}
- return fetchatt(TupleDescCompactAttr(tupleDesc, attnum), tp + off);
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ off = att_pointer_alignby(off, cattr->attalignby,
+ cattr->attlen, tp + off);
+ return fetchatt(cattr, tp + off);
}
/*
@@ -480,63 +387,87 @@ index_deform_tuple_internal(TupleDesc tupleDescriptor,
Datum *values, bool *isnull,
char *tp, bits8 *bp, int hasnulls)
{
+ CompactAttribute *cattr;
int natts = tupleDescriptor->natts; /* number of atts to extract */
- int attnum;
- int off = 0; /* offset in tuple data */
- bool slow = false; /* can we use/set attcacheoff? */
+ int attnum = 0;
+ uint32 off = 0; /* offset in tuple data */
+ int firstNonCacheOffsetAttr;
+ int firstNullAttr;
/* Assert to protect callers who allocate fixed-size arrays */
Assert(natts <= INDEX_MAX_KEYS);
- for (attnum = 0; attnum < natts; attnum++)
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDescriptor->firstNonCachedOffsetAttr >= 0);
+
+ firstNonCacheOffsetAttr = Min(tupleDescriptor->firstNonCachedOffsetAttr, natts);
+
+ if (hasnulls)
+ {
+ firstNullAttr = first_null_attr(bp, natts);
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, firstNullAttr);
+ }
+ else
+ firstNullAttr = natts;
+
+ if (firstNonCacheOffsetAttr > 0)
{
- CompactAttribute *thisatt = TupleDescCompactAttr(tupleDescriptor, attnum);
+#ifdef USE_ASSERT_CHECKING
+ /* In Assert enabled builds, verify attcacheoff is correct */
+ off = 0;
+#endif
- if (hasnulls && att_isnull(attnum, bp))
+ do
{
- values[attnum] = (Datum) 0;
- isnull[attnum] = true;
- slow = true; /* can't use attcacheoff anymore */
- continue;
- }
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDescriptor, attnum);
- isnull[attnum] = false;
+#ifdef USE_ASSERT_CHECKING
+ off = att_nominal_alignby(off, cattr->attalignby);
+ Assert(off == cattr->attcacheoff);
+ off += cattr->attlen;
+#endif
- if (!slow && thisatt->attcacheoff >= 0)
- off = thisatt->attcacheoff;
- else if (thisatt->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be no
- * pad bytes in any case: then the offset will be valid for either
- * an aligned or unaligned value.
- */
- if (!slow &&
- off == att_nominal_alignby(off, thisatt->attalignby))
- thisatt->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, thisatt->attalignby, -1,
- tp + off);
- slow = true;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, thisatt->attalignby);
+ values[attnum] = fetch_att_noerr(tp + cattr->attcacheoff, cattr->attbyval,
+ cattr->attlen);
+ } while (++attnum < firstNonCacheOffsetAttr);
- if (!slow)
- thisatt->attcacheoff = off;
- }
+ off = cattr->attcacheoff + cattr->attlen;
+ }
- values[attnum] = fetchatt(thisatt, tp + off);
+ for (; attnum < firstNullAttr; attnum++)
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDescriptor, attnum);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
+ }
- off = att_addlength_pointer(off, thisatt->attlen, tp + off);
+ for (; attnum < natts; attnum++)
+ {
+ Assert(hasnulls);
- if (thisatt->attlen <= 0)
- slow = true; /* can't use attcacheoff anymore */
+ if (att_isnull(attnum, bp))
+ {
+ values[attnum] = (Datum) 0;
+ isnull[attnum] = true;
+ continue;
+ }
+
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDescriptor, attnum);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
}
}
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 2137385a833..c68561337d7 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -197,6 +197,10 @@ CreateTemplateTupleDesc(int natts)
desc->tdtypmod = -1;
desc->tdrefcount = -1; /* assume not reference-counted */
+ /* This will be set to the correct value by TupleDescFinalize() */
+ desc->firstNonCachedOffsetAttr = -1;
+ desc->firstNonGuaranteedAttr = -1;
+
return desc;
}
@@ -457,6 +461,9 @@ TupleDescCopy(TupleDesc dst, TupleDesc src)
* descriptor to another.
*
* !!! Constraints and defaults are not copied !!!
+ *
+ * The caller must take care of calling TupleDescFinalize() on 'dst' once all
+ * TupleDesc changes have been made.
*/
void
TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
@@ -489,6 +496,50 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
populate_compact_attribute(dst, dstAttno - 1);
}
+/*
+ * TupleDescFinalize
+ * Finalize the given TupleDesc. This must be called after the
+ * attributes arrays have been populated or adjusted by any code.
+ *
+ * Must be called after populate_compact_attribute() and before
+ * BlessTupleDesc().
+ */
+void
+TupleDescFinalize(TupleDesc tupdesc)
+{
+ int firstNonCachedOffsetAttr = 0;
+ int firstNonGuaranteedAttr = tupdesc->natts;
+ int off = 0;
+
+ for (int i = 0; i < tupdesc->natts; i++)
+ {
+ CompactAttribute *cattr = TupleDescCompactAttr(tupdesc, i);
+
+ /*
+ * Find the highest attnum which is guaranteed to exist in all tuples
+ * in the table. We currently only pay attention to byval attributes
+ * to allow additional optimizations during tuple deformation.
+ */
+ if (firstNonGuaranteedAttr == tupdesc->natts &&
+ (cattr->attnullability != ATTNULLABLE_VALID || !cattr->attbyval ||
+ cattr->atthasmissing || cattr->attisdropped || cattr->attlen <= 0))
+ firstNonGuaranteedAttr = i;
+
+ if (cattr->attlen <= 0)
+ break;
+
+ off = att_nominal_alignby(off, cattr->attalignby);
+
+ cattr->attcacheoff = off;
+
+ off += cattr->attlen;
+ firstNonCachedOffsetAttr = i + 1;
+ }
+
+ tupdesc->firstNonCachedOffsetAttr = firstNonCachedOffsetAttr;
+ tupdesc->firstNonGuaranteedAttr = firstNonGuaranteedAttr;
+}
+
/*
* Free a TupleDesc including all substructure
*/
diff --git a/src/backend/access/spgist/spgutils.c b/src/backend/access/spgist/spgutils.c
index b246e8127db..a4694bd8065 100644
--- a/src/backend/access/spgist/spgutils.c
+++ b/src/backend/access/spgist/spgutils.c
@@ -335,9 +335,6 @@ getSpGistTupleDesc(Relation index, SpGistTypeDesc *keyType)
/* We shouldn't need to bother with making these valid: */
att->attcompression = InvalidCompressionMethod;
att->attcollation = InvalidOid;
- /* In case we changed typlen, we'd better reset following offsets */
- for (int i = spgFirstIncludeColumn; i < outTupDesc->natts; i++)
- TupleDescCompactAttr(outTupDesc, i)->attcacheoff = -1;
populate_compact_attribute(outTupDesc, spgKeyColumn);
TupleDescFinalize(outTupDesc);
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index bfd3ebc601e..0b635486993 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -1944,7 +1944,7 @@ ExecPartitionCheckEmitError(ResultRelInfo *resultRelInfo,
*/
if (map != NULL)
slot = execute_attr_map_slot(map, slot,
- MakeTupleTableSlot(tupdesc, &TTSOpsVirtual));
+ MakeTupleTableSlot(tupdesc, &TTSOpsVirtual, 0));
modifiedCols = bms_union(ExecGetInsertedCols(rootrel, estate),
ExecGetUpdatedCols(rootrel, estate));
}
@@ -2060,7 +2060,7 @@ ExecConstraints(ResultRelInfo *resultRelInfo,
*/
if (map != NULL)
slot = execute_attr_map_slot(map, slot,
- MakeTupleTableSlot(tupdesc, &TTSOpsVirtual));
+ MakeTupleTableSlot(tupdesc, &TTSOpsVirtual, 0));
modifiedCols = bms_union(ExecGetInsertedCols(rootrel, estate),
ExecGetUpdatedCols(rootrel, estate));
rel = rootrel->ri_RelationDesc;
@@ -2196,7 +2196,7 @@ ReportNotNullViolationError(ResultRelInfo *resultRelInfo, TupleTableSlot *slot,
*/
if (map != NULL)
slot = execute_attr_map_slot(map, slot,
- MakeTupleTableSlot(tupdesc, &TTSOpsVirtual));
+ MakeTupleTableSlot(tupdesc, &TTSOpsVirtual, 0));
modifiedCols = bms_union(ExecGetInsertedCols(rootrel, estate),
ExecGetUpdatedCols(rootrel, estate));
rel = rootrel->ri_RelationDesc;
@@ -2304,7 +2304,7 @@ ExecWithCheckOptions(WCOKind kind, ResultRelInfo *resultRelInfo,
*/
if (map != NULL)
slot = execute_attr_map_slot(map, slot,
- MakeTupleTableSlot(tupdesc, &TTSOpsVirtual));
+ MakeTupleTableSlot(tupdesc, &TTSOpsVirtual, 0));
modifiedCols = bms_union(ExecGetInsertedCols(rootrel, estate),
ExecGetUpdatedCols(rootrel, estate));
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 07b248aa5f3..bfd2286ec2b 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -992,118 +992,6 @@ tts_buffer_heap_store_tuple(TupleTableSlot *slot, HeapTuple tuple,
}
}
-/*
- * slot_deform_heap_tuple_internal
- * An always inline helper function for use in slot_deform_heap_tuple to
- * allow the compiler to emit specialized versions of this function for
- * various combinations of "slow" and "hasnulls". For example, if a
- * given tuple has no nulls, then we needn't check "hasnulls" for every
- * attribute that we're deforming. The caller can just call this
- * function with hasnulls set to constant-false and have the compiler
- * remove the constant-false branches and emit more optimal code.
- *
- * Returns the next attnum to deform, which can be equal to natts when the
- * function manages to deform all requested attributes. *offp is an input and
- * output parameter which is the byte offset within the tuple to start deforming
- * from which, on return, gets set to the offset where the next attribute
- * should be deformed from. *slowp is set to true when subsequent deforming
- * of this tuple must use a version of this function with "slow" passed as
- * true.
- *
- * Callers cannot assume when we return "attnum" (i.e. all requested
- * attributes have been deformed) that slow mode isn't required for any
- * additional deforming as the final attribute may have caused a switch to
- * slow mode.
- */
-static pg_attribute_always_inline int
-slot_deform_heap_tuple_internal(TupleTableSlot *slot, HeapTuple tuple,
- int attnum, int natts, bool slow,
- bool hasnulls, uint32 *offp, bool *slowp)
-{
- TupleDesc tupleDesc = slot->tts_tupleDescriptor;
- Datum *values = slot->tts_values;
- bool *isnull = slot->tts_isnull;
- HeapTupleHeader tup = tuple->t_data;
- char *tp; /* ptr to tuple data */
- bits8 *bp = tup->t_bits; /* ptr to null bitmap in tuple */
- bool slownext = false;
-
- tp = (char *) tup + tup->t_hoff;
-
- for (; attnum < natts; attnum++)
- {
- CompactAttribute *thisatt = TupleDescCompactAttr(tupleDesc, attnum);
-
- if (hasnulls && att_isnull(attnum, bp))
- {
- values[attnum] = (Datum) 0;
- isnull[attnum] = true;
- if (!slow)
- {
- *slowp = true;
- return attnum + 1;
- }
- else
- continue;
- }
-
- isnull[attnum] = false;
-
- /* calculate the offset of this attribute */
- if (!slow && thisatt->attcacheoff >= 0)
- *offp = thisatt->attcacheoff;
- else if (thisatt->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be no
- * pad bytes in any case: then the offset will be valid for either
- * an aligned or unaligned value.
- */
- if (!slow && *offp == att_nominal_alignby(*offp, thisatt->attalignby))
- thisatt->attcacheoff = *offp;
- else
- {
- *offp = att_pointer_alignby(*offp,
- thisatt->attalignby,
- -1,
- tp + *offp);
-
- if (!slow)
- slownext = true;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- *offp = att_nominal_alignby(*offp, thisatt->attalignby);
-
- if (!slow)
- thisatt->attcacheoff = *offp;
- }
-
- values[attnum] = fetchatt(thisatt, tp + *offp);
-
- *offp = att_addlength_pointer(*offp, thisatt->attlen, tp + *offp);
-
- /* check if we need to switch to slow mode */
- if (!slow)
- {
- /*
- * We're unable to deform any further if the above code set
- * 'slownext', or if this isn't a fixed-width attribute.
- */
- if (slownext || thisatt->attlen <= 0)
- {
- *slowp = true;
- return attnum + 1;
- }
- }
- }
-
- return natts;
-}
-
/*
* slot_deform_heap_tuple
* Given a TupleTableSlot, extract data from the slot's physical tuple
@@ -1125,94 +1013,226 @@ static pg_attribute_always_inline void
slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int reqnatts)
{
- bool hasnulls = HeapTupleHasNulls(tuple);
- int attnum;
+ CompactAttribute *cattr;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+ HeapTupleHeader tup = tuple->t_data;
+ size_t attnum;
+ int firstNonCacheOffsetAttr;
+ int firstNonGuaranteedAttr;
+ int firstNullAttr;
int natts;
+ Datum *values;
+ bool *isnull;
+ char *tp; /* ptr to tuple data */
uint32 off; /* offset in tuple data */
- bool slow; /* can we use/set attcacheoff? */
- /* We can only fetch as many attributes as the tuple has. */
- natts = Min(HeapTupleHeaderGetNatts(tuple->t_data), reqnatts);
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
+
+ isnull = slot->tts_isnull;
/*
- * Check whether the first call for this tuple, and initialize or restore
- * loop state.
+ * Some callers may form and deform tuples prior to NOT NULL constraints
+ * being checked. Here we'd like to optimize the case where we only need
+ * to fetch attributes before or up to the point where the attribute is
+ * guaranteed to exist in the tuple. We rely on the slot flag being set
+ * correctly to only enable this optimization when it's valid to do so.
+ * This optimization allows us to save fetching the number of attributes
+ * from the tuple and saves the additional cost of handling non-byval
+ * attrs.
*/
+ firstNonGuaranteedAttr = Min(reqnatts, slot->tts_first_nonguaranteed);
+
+ firstNonCacheOffsetAttr = tupleDesc->firstNonCachedOffsetAttr;
+
+ if (HeapTupleHasNulls(tuple))
+ {
+ natts = HeapTupleHeaderGetNatts(tup);
+ tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits) +
+ BITMAPLEN(natts));
+
+ natts = Min(natts, reqnatts);
+ if (natts > firstNonGuaranteedAttr)
+ {
+ bits8 *bp = tup->t_bits;
+
+ /* Find the first NULL attr */
+ firstNullAttr = first_null_attr(bp, natts);
+
+ /*
+ * And populate the isnull array for all attributes being fetched
+ * from the tuple.
+ */
+ populate_isnull_array(bp, natts, isnull);
+ }
+ else
+ {
+ /* Otherwise all required columns are guaranteed to exist */
+ firstNullAttr = natts;
+ }
+ }
+ else
+ {
+ tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits));
+
+ /*
+ * We only need to look at the tuple's natts if we need more than the
+ * guaranteed number of columns
+ */
+ if (reqnatts > firstNonGuaranteedAttr)
+ natts = Min(HeapTupleHeaderGetNatts(tup), reqnatts);
+ else
+ {
+ /* No need to access the number of attributes in the tuple */
+ natts = reqnatts;
+ }
+
+ /* All attrs can be fetched without checking for NULLs */
+ firstNullAttr = natts;
+ }
+
attnum = slot->tts_nvalid;
+ values = slot->tts_values;
slot->tts_nvalid = reqnatts;
- if (attnum == 0)
+
+ /* Ensure we calculated tp correctly */
+ Assert(tp == (char *) tup + tup->t_hoff);
+
+ if (attnum < firstNonGuaranteedAttr)
{
- /* Start from the first attribute */
- off = 0;
- slow = false;
+ int attlen;
+
+ do
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ attlen = cattr->attlen;
+
+ /* We don't expect any non-byval types */
+ pg_assume(attlen > 0);
+ Assert(cattr->attbyval == true);
+
+ off = cattr->attcacheoff;
+ values[attnum] = fetch_att_noerr(tp + off, true, attlen);
+ attnum++;
+ } while (attnum < firstNonGuaranteedAttr);
+
+ off += attlen;
+
+ if (attnum == reqnatts)
+ goto done;
}
else
{
/* Restore state from previous execution */
off = *offp;
- slow = TTS_SLOW(slot);
+
+ /* We expect *offp to be set to 0 when attnum == 0 */
+ Assert(off == 0 || attnum > 0);
}
+ /* We can use attcacheoff up until the first NULL */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, firstNullAttr);
+
/*
- * If 'slow' isn't set, try deforming using deforming code that does not
- * contain any of the extra checks required for non-fixed offset
- * deforming. During deforming, if or when we find a NULL or a variable
- * length attribute, we'll switch to a deforming method which includes the
- * extra code required for non-fixed offset deforming, a.k.a slow mode.
- * Because this is performance critical, we inline
- * slot_deform_heap_tuple_internal passing the 'slow' and 'hasnull'
- * parameters as constants to allow the compiler to emit specialized code
- * with the known-const false comparisons and subsequent branches removed.
+ * Handle the portion of the tuple that we have cached the offset for up
+ * to the first NULL attribute. The offset is effectively fixed for
+ * these, so we can use the CompactAttribute's attcacheoff.
*/
- if (!slow)
+ if (attnum < firstNonCacheOffsetAttr)
{
- /* Tuple without any NULLs? We can skip doing any NULL checking */
- if (!hasnulls)
- attnum = slot_deform_heap_tuple_internal(slot,
- tuple,
- attnum,
- natts,
- false, /* slow */
- false, /* hasnulls */
- &off,
- &slow);
- else
- attnum = slot_deform_heap_tuple_internal(slot,
- tuple,
- attnum,
- natts,
- false, /* slow */
- true, /* hasnulls */
- &off,
- &slow);
+ int attlen;
+
+ do
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ attlen = cattr->attlen;
+
+ off = cattr->attcacheoff;
+ values[attnum] = fetch_att_noerr(tp + off,
+ cattr->attbyval,
+ attlen);
+ attnum++;
+ } while (attnum < firstNonCacheOffsetAttr);
+
+ /*
+ * Point the offset after the end of the last attribute with a cached
+ * offset. We expect the final cached offset attribute to have a
+ * fixed width, so just add the attlen to the attcacheoff
+ */
+ Assert(attlen > 0);
+ off += attlen;
}
- /* If there's still work to do then we must be in slow mode */
- if (attnum < natts)
+ /*
+ * Handle any portion of the tuple that doesn't have a fixed offset up
+ * until the first NULL attribute. This loop only differs from the one
+ * after it by the NULL checks.
+ */
+ for (; attnum < firstNullAttr; attnum++)
{
- /* XXX is it worth adding a separate call when hasnulls is false? */
- attnum = slot_deform_heap_tuple_internal(slot,
- tuple,
- attnum,
- natts,
- true, /* slow */
- hasnulls,
- &off,
- &slow);
+ int attlen;
+
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ attlen = cattr->attlen;
+
+ /*
+ * cstrings don't exist in heap tuples. Use pg_assume to instruct the
+ * compiler not to emit the cstring-related code in
+ * align_fetch_then_add().
+ */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ attlen,
+ cattr->attalignby);
}
/*
- * Save state for next execution
+ * Now handle any remaining attributes in the tuple up to the requested
+ * attnum. This time, include NULL checks as we're now at the first NULL
+ * attribute.
*/
- *offp = off;
- if (slow)
- slot->tts_flags |= TTS_FLAG_SLOW;
- else
- slot->tts_flags &= ~TTS_FLAG_SLOW;
+ for (; attnum < natts; attnum++)
+ {
+ int attlen;
+
+ if (isnull[attnum])
+ {
+ values[attnum] = (Datum) 0;
+ continue;
+ }
+
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ attlen = cattr->attlen;
- /* Fetch any missing attrs and raise an error if reqnatts is invalid. */
+ /* As above, we don't expect cstrings */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ attlen,
+ cattr->attalignby);
+ }
+
+ /* Fetch any missing attrs and raise an error if reqnatts is invalid */
if (unlikely(attnum < reqnatts))
+ {
+ *offp = off;
slot_getmissingattrs(slot, attnum, reqnatts);
+ return;
+ }
+done:
+
+ /* Save current offset for next execution */
+ *offp = off;
}
const TupleTableSlotOps TTSOpsVirtual = {
@@ -1307,7 +1327,7 @@ const TupleTableSlotOps TTSOpsBufferHeapTuple = {
*/
TupleTableSlot *
MakeTupleTableSlot(TupleDesc tupleDesc,
- const TupleTableSlotOps *tts_ops)
+ const TupleTableSlotOps *tts_ops, uint16 flags)
{
Size basesz,
allocsz;
@@ -1331,6 +1351,7 @@ MakeTupleTableSlot(TupleDesc tupleDesc,
*((const TupleTableSlotOps **) &slot->tts_ops) = tts_ops;
slot->type = T_TupleTableSlot;
slot->tts_flags |= TTS_FLAG_EMPTY;
+ slot->tts_flags |= flags;
if (tupleDesc != NULL)
slot->tts_flags |= TTS_FLAG_FIXED;
slot->tts_tupleDescriptor = tupleDesc;
@@ -1342,12 +1363,31 @@ MakeTupleTableSlot(TupleDesc tupleDesc,
slot->tts_values = (Datum *)
(((char *) slot)
+ MAXALIGN(basesz));
+
+ /*
+ * We round the size of tts_isnull up to the next highest multiple of
+ * 8. This is needed as populate_isnull_array() operates on 8
+ * elements at a time when converting a tuple's NULL bitmap into a
+ * boolean array.
+ */
slot->tts_isnull = (bool *)
(((char *) slot)
+ MAXALIGN(basesz)
- + MAXALIGN(tupleDesc->natts * sizeof(Datum)));
+ + TYPEALIGN(8, tupleDesc->natts * sizeof(Datum)));
PinTupleDesc(tupleDesc);
+
+ /*
+ * Precalculate the maximum guaranteed attribute that has to exist in
+ * every tuple which gets deformed into this slot. When the
+ * TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS flag is enabled, we simply take
+ * the precalculated value from the tupleDesc, otherwise the
+ * optimization is disabled, and we set the value to 0.
+ */
+ if ((flags & TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS) != 0)
+ slot->tts_first_nonguaranteed = tupleDesc->firstNonGuaranteedAttr;
+ else
+ slot->tts_first_nonguaranteed = 0;
}
/*
@@ -1366,9 +1406,9 @@ MakeTupleTableSlot(TupleDesc tupleDesc,
*/
TupleTableSlot *
ExecAllocTableSlot(List **tupleTable, TupleDesc desc,
- const TupleTableSlotOps *tts_ops)
+ const TupleTableSlotOps *tts_ops, uint16 flags)
{
- TupleTableSlot *slot = MakeTupleTableSlot(desc, tts_ops);
+ TupleTableSlot *slot = MakeTupleTableSlot(desc, tts_ops, flags);
*tupleTable = lappend(*tupleTable, slot);
@@ -1435,7 +1475,7 @@ TupleTableSlot *
MakeSingleTupleTableSlot(TupleDesc tupdesc,
const TupleTableSlotOps *tts_ops)
{
- TupleTableSlot *slot = MakeTupleTableSlot(tupdesc, tts_ops);
+ TupleTableSlot *slot = MakeTupleTableSlot(tupdesc, tts_ops, 0);
return slot;
}
@@ -1515,8 +1555,14 @@ ExecSetSlotDescriptor(TupleTableSlot *slot, /* slot to change */
*/
slot->tts_values = (Datum *)
MemoryContextAlloc(slot->tts_mcxt, tupdesc->natts * sizeof(Datum));
+
+ /*
+ * We round the size of tts_isnull up to the next highest multiple of 8.
+ * This is needed as populate_isnull_array() operates on 8 elements at a
+ * time when converting a tuple's NULL bitmap into a boolean array.
+ */
slot->tts_isnull = (bool *)
- MemoryContextAlloc(slot->tts_mcxt, tupdesc->natts * sizeof(bool));
+ MemoryContextAlloc(slot->tts_mcxt, TYPEALIGN(8, tupdesc->natts * sizeof(bool)));
}
/* --------------------------------
@@ -1978,7 +2024,7 @@ ExecInitResultSlot(PlanState *planstate, const TupleTableSlotOps *tts_ops)
TupleTableSlot *slot;
slot = ExecAllocTableSlot(&planstate->state->es_tupleTable,
- planstate->ps_ResultTupleDesc, tts_ops);
+ planstate->ps_ResultTupleDesc, tts_ops, 0);
planstate->ps_ResultTupleSlot = slot;
planstate->resultopsfixed = planstate->ps_ResultTupleDesc != NULL;
@@ -2006,10 +2052,11 @@ ExecInitResultTupleSlotTL(PlanState *planstate,
*/
void
ExecInitScanTupleSlot(EState *estate, ScanState *scanstate,
- TupleDesc tupledesc, const TupleTableSlotOps *tts_ops)
+ TupleDesc tupledesc, const TupleTableSlotOps *tts_ops,
+ uint16 flags)
{
scanstate->ss_ScanTupleSlot = ExecAllocTableSlot(&estate->es_tupleTable,
- tupledesc, tts_ops);
+ tupledesc, tts_ops, flags);
scanstate->ps.scandesc = tupledesc;
scanstate->ps.scanopsfixed = tupledesc != NULL;
scanstate->ps.scanops = tts_ops;
@@ -2029,7 +2076,7 @@ ExecInitExtraTupleSlot(EState *estate,
TupleDesc tupledesc,
const TupleTableSlotOps *tts_ops)
{
- return ExecAllocTableSlot(&estate->es_tupleTable, tupledesc, tts_ops);
+ return ExecAllocTableSlot(&estate->es_tupleTable, tupledesc, tts_ops, 0);
}
/* ----------------
@@ -2261,10 +2308,16 @@ ExecTypeSetColNames(TupleDesc typeInfo, List *namesList)
* This happens "for free" if the tupdesc came from a relcache entry, but
* not if we have manufactured a tupdesc for a transient RECORD datatype.
* In that case we have to notify typcache.c of the existence of the type.
+ *
+ * TupleDescFinalize() must be called on the TupleDesc before calling this
+ * function.
*/
TupleDesc
BlessTupleDesc(TupleDesc tupdesc)
{
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupdesc->firstNonCachedOffsetAttr >= 0);
+
if (tupdesc->tdtypeid == RECORDOID &&
tupdesc->tdtypmod < 0)
assign_record_type_typmod(tupdesc);
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index a7955e476f9..f62582859f9 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -711,7 +711,7 @@ ExecCreateScanSlotFromOuterPlan(EState *estate,
outerPlan = outerPlanState(scanstate);
tupDesc = ExecGetResultType(outerPlan);
- ExecInitScanTupleSlot(estate, scanstate, tupDesc, tts_ops);
+ ExecInitScanTupleSlot(estate, scanstate, tupDesc, tts_ops, 0);
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
index 7d487a165fa..c5c321b4f42 100644
--- a/src/backend/executor/nodeAgg.c
+++ b/src/backend/executor/nodeAgg.c
@@ -1682,7 +1682,7 @@ find_hash_columns(AggState *aggstate)
&perhash->hashfunctions);
perhash->hashslot =
ExecAllocTableSlot(&estate->es_tupleTable, hashDesc,
- &TTSOpsMinimalTuple);
+ &TTSOpsMinimalTuple, 0);
list_free(hashTlist);
bms_free(colnos);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index e0b6df64767..6f29954e84f 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -382,7 +382,10 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
*/
ExecInitScanTupleSlot(estate, &scanstate->ss,
RelationGetDescr(currentRelation),
- table_slot_callbacks(currentRelation));
+ table_slot_callbacks(currentRelation), 0);
+
+ scanstate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeCtescan.c b/src/backend/executor/nodeCtescan.c
index e6e476388e5..45b09d93b93 100644
--- a/src/backend/executor/nodeCtescan.c
+++ b/src/backend/executor/nodeCtescan.c
@@ -261,7 +261,7 @@ ExecInitCteScan(CteScan *node, EState *estate, int eflags)
*/
ExecInitScanTupleSlot(estate, &scanstate->ss,
ExecGetResultType(scanstate->cteplanstate),
- &TTSOpsMinimalTuple);
+ &TTSOpsMinimalTuple, 0);
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeCustom.c b/src/backend/executor/nodeCustom.c
index a9ad5af6a98..b7cc890cd20 100644
--- a/src/backend/executor/nodeCustom.c
+++ b/src/backend/executor/nodeCustom.c
@@ -79,14 +79,14 @@ ExecInitCustomScan(CustomScan *cscan, EState *estate, int eflags)
TupleDesc scan_tupdesc;
scan_tupdesc = ExecTypeFromTL(cscan->custom_scan_tlist);
- ExecInitScanTupleSlot(estate, &css->ss, scan_tupdesc, slotOps);
+ ExecInitScanTupleSlot(estate, &css->ss, scan_tupdesc, slotOps, 0);
/* Node's targetlist will contain Vars with varno = INDEX_VAR */
tlistvarno = INDEX_VAR;
}
else
{
ExecInitScanTupleSlot(estate, &css->ss, RelationGetDescr(scan_rel),
- slotOps);
+ slotOps, 0);
/* Node's targetlist will contain Vars with varno = scanrelid */
tlistvarno = scanrelid;
}
diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c
index 8721b67b7cc..6f0daddce07 100644
--- a/src/backend/executor/nodeForeignscan.c
+++ b/src/backend/executor/nodeForeignscan.c
@@ -191,7 +191,7 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
scan_tupdesc = ExecTypeFromTL(node->fdw_scan_tlist);
ExecInitScanTupleSlot(estate, &scanstate->ss, scan_tupdesc,
- &TTSOpsHeapTuple);
+ &TTSOpsHeapTuple, 0);
/* Node's targetlist will contain Vars with varno = INDEX_VAR */
tlistvarno = INDEX_VAR;
}
@@ -202,7 +202,7 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
/* don't trust FDWs to return tuples fulfilling NOT NULL constraints */
scan_tupdesc = CreateTupleDescCopy(RelationGetDescr(currentRelation));
ExecInitScanTupleSlot(estate, &scanstate->ss, scan_tupdesc,
- &TTSOpsHeapTuple);
+ &TTSOpsHeapTuple, 0);
/* Node's targetlist will contain Vars with varno = scanrelid */
tlistvarno = scanrelid;
}
diff --git a/src/backend/executor/nodeFunctionscan.c b/src/backend/executor/nodeFunctionscan.c
index feb82d64967..222741adf3b 100644
--- a/src/backend/executor/nodeFunctionscan.c
+++ b/src/backend/executor/nodeFunctionscan.c
@@ -494,7 +494,7 @@ ExecInitFunctionScan(FunctionScan *node, EState *estate, int eflags)
* Initialize scan slot and type.
*/
ExecInitScanTupleSlot(estate, &scanstate->ss, scan_tupdesc,
- &TTSOpsMinimalTuple);
+ &TTSOpsMinimalTuple, 0);
/*
* Initialize result slot, type and projection.
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index c2d09374517..144a57fde95 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -567,7 +567,10 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
*/
tupDesc = ExecTypeFromTL(node->indextlist);
ExecInitScanTupleSlot(estate, &indexstate->ss, tupDesc,
- &TTSOpsVirtual);
+ &TTSOpsVirtual, 0);
+
+ indexstate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
/*
* We need another slot, in a format that's suitable for the table AM, for
@@ -576,7 +579,7 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
indexstate->ioss_TableSlot =
ExecAllocTableSlot(&estate->es_tupleTable,
RelationGetDescr(currentRelation),
- table_slot_callbacks(currentRelation));
+ table_slot_callbacks(currentRelation), 0);
/*
* Initialize result type and projection info. The node's targetlist will
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index a616abff04c..e7bebb89517 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -938,7 +938,10 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
*/
ExecInitScanTupleSlot(estate, &indexstate->ss,
RelationGetDescr(currentRelation),
- table_slot_callbacks(currentRelation));
+ table_slot_callbacks(currentRelation), 0);
+
+ indexstate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeNamedtuplestorescan.c b/src/backend/executor/nodeNamedtuplestorescan.c
index fdfccc8169f..29d862a4001 100644
--- a/src/backend/executor/nodeNamedtuplestorescan.c
+++ b/src/backend/executor/nodeNamedtuplestorescan.c
@@ -137,7 +137,7 @@ ExecInitNamedTuplestoreScan(NamedTuplestoreScan *node, EState *estate, int eflag
* The scan tuple type is specified for the tuplestore.
*/
ExecInitScanTupleSlot(estate, &scanstate->ss, scanstate->tupdesc,
- &TTSOpsMinimalTuple);
+ &TTSOpsMinimalTuple, 0);
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c
index 1b0af70fd7a..a1da05ecc18 100644
--- a/src/backend/executor/nodeSamplescan.c
+++ b/src/backend/executor/nodeSamplescan.c
@@ -128,7 +128,11 @@ ExecInitSampleScan(SampleScan *node, EState *estate, int eflags)
/* and create slot with appropriate rowtype */
ExecInitScanTupleSlot(estate, &scanstate->ss,
RelationGetDescr(scanstate->ss.ss_currentRelation),
- table_slot_callbacks(scanstate->ss.ss_currentRelation));
+ table_slot_callbacks(scanstate->ss.ss_currentRelation),
+ 0);
+
+ scanstate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index af3c788ce8b..8f219f60a93 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -244,7 +244,8 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags)
/* and create slot with the appropriate rowtype */
ExecInitScanTupleSlot(estate, &scanstate->ss,
RelationGetDescr(scanstate->ss.ss_currentRelation),
- table_slot_callbacks(scanstate->ss.ss_currentRelation));
+ table_slot_callbacks(scanstate->ss.ss_currentRelation),
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS);
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeSubqueryscan.c b/src/backend/executor/nodeSubqueryscan.c
index 4fd6f6fb4a5..70914e8189c 100644
--- a/src/backend/executor/nodeSubqueryscan.c
+++ b/src/backend/executor/nodeSubqueryscan.c
@@ -130,7 +130,8 @@ ExecInitSubqueryScan(SubqueryScan *node, EState *estate, int eflags)
*/
ExecInitScanTupleSlot(estate, &subquerystate->ss,
ExecGetResultType(subquerystate->subplan),
- ExecGetResultSlotOps(subquerystate->subplan, NULL));
+ ExecGetResultSlotOps(subquerystate->subplan, NULL),
+ 0);
/*
* The slot used as the scantuple isn't the slot above (outside of EPQ),
diff --git a/src/backend/executor/nodeTableFuncscan.c b/src/backend/executor/nodeTableFuncscan.c
index 52070d147a4..769b9766542 100644
--- a/src/backend/executor/nodeTableFuncscan.c
+++ b/src/backend/executor/nodeTableFuncscan.c
@@ -148,7 +148,7 @@ ExecInitTableFuncScan(TableFuncScan *node, EState *estate, int eflags)
tf->colcollations);
/* and the corresponding scan slot */
ExecInitScanTupleSlot(estate, &scanstate->ss, tupdesc,
- &TTSOpsMinimalTuple);
+ &TTSOpsMinimalTuple, 0);
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index 503817da65b..decc3167ba7 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -394,7 +394,10 @@ ExecInitTidRangeScan(TidRangeScan *node, EState *estate, int eflags)
*/
ExecInitScanTupleSlot(estate, &tidrangestate->ss,
RelationGetDescr(currentRelation),
- table_slot_callbacks(currentRelation));
+ table_slot_callbacks(currentRelation), 0);
+
+ tidrangestate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeTidscan.c b/src/backend/executor/nodeTidscan.c
index 4eddb0828b5..26593b930af 100644
--- a/src/backend/executor/nodeTidscan.c
+++ b/src/backend/executor/nodeTidscan.c
@@ -536,7 +536,10 @@ ExecInitTidScan(TidScan *node, EState *estate, int eflags)
*/
ExecInitScanTupleSlot(estate, &tidstate->ss,
RelationGetDescr(currentRelation),
- table_slot_callbacks(currentRelation));
+ table_slot_callbacks(currentRelation), 0);
+
+ tidstate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeValuesscan.c b/src/backend/executor/nodeValuesscan.c
index e663fb68cfc..effc896ea1c 100644
--- a/src/backend/executor/nodeValuesscan.c
+++ b/src/backend/executor/nodeValuesscan.c
@@ -247,7 +247,7 @@ ExecInitValuesScan(ValuesScan *node, EState *estate, int eflags)
* Get info about values list, initialize scan slot with it.
*/
tupdesc = ExecTypeFromExprList((List *) linitial(node->values_lists));
- ExecInitScanTupleSlot(estate, &scanstate->ss, tupdesc, &TTSOpsVirtual);
+ ExecInitScanTupleSlot(estate, &scanstate->ss, tupdesc, &TTSOpsVirtual, 0);
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeWorktablescan.c b/src/backend/executor/nodeWorktablescan.c
index 210cc44f911..66e904c7636 100644
--- a/src/backend/executor/nodeWorktablescan.c
+++ b/src/backend/executor/nodeWorktablescan.c
@@ -165,7 +165,7 @@ ExecInitWorkTableScan(WorkTableScan *node, EState *estate, int eflags)
scanstate->ss.ps.resultopsset = true;
scanstate->ss.ps.resultopsfixed = false;
- ExecInitScanTupleSlot(estate, &scanstate->ss, NULL, &TTSOpsMinimalTuple);
+ ExecInitScanTupleSlot(estate, &scanstate->ss, NULL, &TTSOpsMinimalTuple, 0);
/*
* initialize child expressions
diff --git a/src/backend/jit/llvm/llvmjit_deform.c b/src/backend/jit/llvm/llvmjit_deform.c
index 3eb087eb56b..12521e3e46a 100644
--- a/src/backend/jit/llvm/llvmjit_deform.c
+++ b/src/backend/jit/llvm/llvmjit_deform.c
@@ -62,7 +62,6 @@ slot_compile_deform(LLVMJitContext *context, TupleDesc desc,
LLVMValueRef v_tts_values;
LLVMValueRef v_tts_nulls;
LLVMValueRef v_slotoffp;
- LLVMValueRef v_flagsp;
LLVMValueRef v_nvalidp;
LLVMValueRef v_nvalid;
LLVMValueRef v_maxatt;
@@ -178,7 +177,6 @@ slot_compile_deform(LLVMJitContext *context, TupleDesc desc,
v_tts_nulls =
l_load_struct_gep(b, StructTupleTableSlot, v_slot, FIELDNO_TUPLETABLESLOT_ISNULL,
"tts_ISNULL");
- v_flagsp = l_struct_gep(b, StructTupleTableSlot, v_slot, FIELDNO_TUPLETABLESLOT_FLAGS, "");
v_nvalidp = l_struct_gep(b, StructTupleTableSlot, v_slot, FIELDNO_TUPLETABLESLOT_NVALID, "");
if (ops == &TTSOpsHeapTuple || ops == &TTSOpsBufferHeapTuple)
@@ -747,14 +745,10 @@ slot_compile_deform(LLVMJitContext *context, TupleDesc desc,
{
LLVMValueRef v_off = l_load(b, TypeSizeT, v_offp, "");
- LLVMValueRef v_flags;
LLVMBuildStore(b, l_int16_const(lc, natts), v_nvalidp);
v_off = LLVMBuildTrunc(b, v_off, LLVMInt32TypeInContext(lc), "");
LLVMBuildStore(b, v_off, v_slotoffp);
- v_flags = l_load(b, LLVMInt16TypeInContext(lc), v_flagsp, "tts_flags");
- v_flags = LLVMBuildOr(b, v_flags, l_int16_const(lc, TTS_FLAG_SLOW), "");
- LLVMBuildStore(b, v_flags, v_flagsp);
LLVMBuildRetVoid(b);
}
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 857ebf7d6fb..4ecfcbff7ab 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -1559,7 +1559,7 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
if (relentry->attrmap)
{
TupleTableSlot *slot = MakeTupleTableSlot(RelationGetDescr(targetrel),
- &TTSOpsVirtual);
+ &TTSOpsVirtual, 0);
old_slot = execute_attr_map_slot(relentry->attrmap, old_slot, slot);
}
@@ -1574,7 +1574,7 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
if (relentry->attrmap)
{
TupleTableSlot *slot = MakeTupleTableSlot(RelationGetDescr(targetrel),
- &TTSOpsVirtual);
+ &TTSOpsVirtual, 0);
new_slot = execute_attr_map_slot(relentry->attrmap, new_slot, slot);
}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index d27ac216e6d..597de687b45 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -666,14 +666,6 @@ RelationBuildTupleDesc(Relation relation)
elog(ERROR, "pg_attribute catalog is missing %d attribute(s) for relation OID %u",
need, RelationGetRelid(relation));
- /*
- * We can easily set the attcacheoff value for the first attribute: it
- * must be zero. This eliminates the need for special cases for attnum=1
- * that used to exist in fastgetattr() and index_getattr().
- */
- if (RelationGetNumberOfAttributes(relation) > 0)
- TupleDescCompactAttr(relation->rd_att, 0)->attcacheoff = 0;
-
/*
* Set up constraint/default info
*/
@@ -1985,8 +1977,6 @@ formrdesc(const char *relationName, Oid relationReltype,
populate_compact_attribute(relation->rd_att, i);
}
- /* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
- TupleDescCompactAttr(relation->rd_att, 0)->attcacheoff = 0;
TupleDescFinalize(relation->rd_att);
/* mark not-null status */
@@ -4446,8 +4436,6 @@ BuildHardcodedDescriptor(int natts, const FormData_pg_attribute *attrs)
populate_compact_attribute(result, i);
}
- /* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
- TupleDescCompactAttr(result, 0)->attcacheoff = 0;
TupleDescFinalize(result);
/* Note: we don't bother to set up a TupleConstr entry */
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 595413dbbc5..ad7bc013812 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -131,6 +131,19 @@ typedef struct CompactAttribute
* Any code making changes manually to and fields in the FormData_pg_attribute
* array must subsequently call populate_compact_attribute() to flush the
* changes out to the corresponding 'compact_attrs' element.
+ *
+ * firstNonCachedOffsetAttr stores the index into the compact_attrs array for
+ * the first attribute that we don't have a known attcacheoff for.
+ *
+ * firstNonGuaranteedAttr stores the index to into the compact_attrs array for
+ * the first attribute that is either NULLable, missing, or !attbyval. This
+ * can be used in locations as a guarantee that attributes before this will
+ * always exist in tuples. The !attbyval part isn't required for this, but
+ * including this allows various tuple deforming routines to forego any checks
+ * for !attbyval.
+ *
+ * Once a TupleDesc has been populated, before it is used for any purpose,
+ * TupleDescFinalize() must be called on it.
*/
typedef struct TupleDescData
{
@@ -138,6 +151,11 @@ typedef struct TupleDescData
Oid tdtypeid; /* composite type ID for tuple type */
int32 tdtypmod; /* typmod for tuple type */
int tdrefcount; /* reference count, or -1 if not counting */
+ int firstNonCachedOffsetAttr; /* index of the first att without
+ * an attcacheoff */
+ int firstNonGuaranteedAttr; /* index of the first nullable,
+ * missing, dropped, or !attbyval
+ * attribute. */
TupleConstr *constr; /* constraints, or NULL if none */
/* compact_attrs[N] is the compact metadata of Attribute Number N+1 */
CompactAttribute compact_attrs[FLEXIBLE_ARRAY_MEMBER];
@@ -195,7 +213,6 @@ extern TupleDesc CreateTupleDescTruncatedCopy(TupleDesc tupdesc, int natts);
extern TupleDesc CreateTupleDescCopyConstr(TupleDesc tupdesc);
-#define TupleDescFinalize(d) ((void) 0)
#define TupleDescSize(src) \
(offsetof(struct TupleDescData, compact_attrs) + \
(src)->natts * sizeof(CompactAttribute) + \
@@ -206,6 +223,7 @@ extern void TupleDescCopy(TupleDesc dst, TupleDesc src);
extern void TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
TupleDesc src, AttrNumber srcAttno);
+extern void TupleDescFinalize(TupleDesc tupdesc);
extern void FreeTupleDesc(TupleDesc tupdesc);
extern void IncrTupleDescRefCount(TupleDesc tupdesc);
diff --git a/src/include/access/tupmacs.h b/src/include/access/tupmacs.h
index d64c18b950b..87dbeb76618 100644
--- a/src/include/access/tupmacs.h
+++ b/src/include/access/tupmacs.h
@@ -15,7 +15,9 @@
#define TUPMACS_H
#include "catalog/pg_type_d.h" /* for TYPALIGN macros */
-
+#include "port/pg_bitutils.h"
+#include "port/pg_bswap.h"
+#include "varatt.h"
/*
* Check a tuple's null bitmap to determine whether the attribute is null.
@@ -28,6 +30,62 @@ att_isnull(int ATT, const bits8 *BITS)
return !(BITS[ATT >> 3] & (1 << (ATT & 0x07)));
}
+/*
+ * populate_isnull_array
+ * Transform a tuple's null bitmap into a boolean array.
+ *
+ * Caller must ensure that the isnull array is sized so it contains
+ * at least as many elements as there are bits in the 'bits' array.
+ * Callers should be aware that isnull is populated 8 elements at a time,
+ * effectively as if natts is rounded up to the next multiple of 8.
+ */
+static inline void
+populate_isnull_array(const bits8 *bits, int natts, bool *isnull)
+{
+ int nbytes = (natts + 7) >> 3;
+
+ /*
+ * Multiplying the inverted NULL bitmap byte by this value results in the
+ * lowest bit in each byte being set the same as each bit of the inverted
+ * byte. We perform this as 2 32-bit operations rather than a single
+ * 64-bit operation as multiplying by the required value to do this in
+ * 64-bits would result in overflowing a uint64 in some cases.
+ *
+ * XXX if we ever require BMI2 (-march=x86-64-v3), then this could be done
+ * more efficiently on most X86-64 CPUs with the PDEP instruction. Beware
+ * that some chips (e.g. AMD's Zen2) are horribly inefficient at PDEP.
+ */
+#define SPREAD_BITS_MULTIPLIER_32 0x204081U
+
+ for (int i = 0; i < nbytes; i++, isnull += 8)
+ {
+ uint64 isnull_8;
+ bits8 nullbyte = ~bits[i];
+
+ /* Convert the lower 4 bits of NULL bitmap word into a 64 bit int */
+ isnull_8 = (nullbyte & 0xf) * SPREAD_BITS_MULTIPLIER_32;
+
+ /*
+ * Convert the upper 4 bits of null bitmap word into a 64 bit int,
+ * shift into the upper 32 bit and bitwise-OR with the result of the
+ * lower 4 bits.
+ */
+ isnull_8 |= ((uint64) ((nullbyte >> 4) * SPREAD_BITS_MULTIPLIER_32)) << 32;
+
+ /* Mask out all other bits apart from the lowest bit of each byte. */
+ isnull_8 &= UINT64CONST(0x0101010101010101);
+
+#ifdef WORDS_BIGENDIAN
+
+ /*
+ * Fix byte order on big-endian machines before copying to the array.
+ */
+ isnull_8 = pg_bswap64(isnull_8);
+#endif
+ memcpy(isnull, &isnull_8, sizeof(uint64));
+ }
+}
+
#ifndef FRONTEND
/*
* Given an attbyval and an attlen from either a Form_pg_attribute or
@@ -69,6 +127,170 @@ fetch_att(const void *T, bool attbyval, int attlen)
else
return PointerGetDatum(T);
}
+
+/*
+ * Same, but no error checking for invalid attlens for byval types. This
+ * is safe to use when attlen comes from CompactAttribute as we validate the
+ * length when populating that struct.
+ */
+static inline Datum
+fetch_att_noerr(const void *T, bool attbyval, int attlen)
+{
+ if (attbyval)
+ {
+ switch (attlen)
+ {
+ case sizeof(int32):
+ return Int32GetDatum(*((const int32 *) T));
+ case sizeof(int16):
+ return Int16GetDatum(*((const int16 *) T));
+ case sizeof(char):
+ return CharGetDatum(*((const char *) T));
+ default:
+ Assert(attlen == sizeof(int64));
+ return Int64GetDatum(*((const int64 *) T));
+ }
+ }
+ else
+ return PointerGetDatum(T);
+}
+
+
+/*
+ * align_fetch_then_add
+ * Applies all the functionality of att_pointer_alignby(),
+ * fetch_att_noerr() and att_addlength_pointer(), resulting in the *off
+ * pointer to the perhaps unaligned number of bytes into 'tupptr', ready
+ * to deform the next attribute.
+ *
+ * tupptr: pointer to the beginning of the tuple, after the header and any
+ * NULL bitmask.
+ * off: offset in bytes for reading tuple data, possibly unaligned.
+ * attbyval, attlen and attalignby are values from CompactAttribute.
+ */
+static inline Datum
+align_fetch_then_add(const char *tupptr, uint32 *off, bool attbyval, int attlen,
+ uint8 attalignby)
+{
+ Datum res;
+
+ if (attlen > 0)
+ {
+ const char *offset_ptr;
+
+ *off = TYPEALIGN(attalignby, *off);
+ offset_ptr = tupptr + *off;
+ *off += attlen;
+ if (attbyval)
+ {
+ switch (attlen)
+ {
+ case sizeof(char):
+ return CharGetDatum(*((const char *) offset_ptr));
+ case sizeof(int16):
+ return Int16GetDatum(*((const int16 *) offset_ptr));
+ case sizeof(int32):
+ return Int32GetDatum(*((const int32 *) offset_ptr));
+ default:
+
+ /*
+ * populate_compact_attribute_internal() should have
+ * checked
+ */
+ Assert(attlen == sizeof(int64));
+ return Int64GetDatum(*((const int64 *) offset_ptr));
+ }
+ }
+ return PointerGetDatum(offset_ptr);
+ }
+ else if (attlen == -1)
+ {
+ if (!VARATT_IS_SHORT(tupptr + *off))
+ *off = TYPEALIGN(attalignby, *off);
+
+ res = PointerGetDatum(tupptr + *off);
+ *off += VARSIZE_ANY(DatumGetPointer(res));
+ return res;
+ }
+ else
+ {
+ Assert(attlen == -2);
+ *off = TYPEALIGN(attalignby, *off);
+ res = PointerGetDatum(tupptr + *off);
+ *off += strlen(tupptr + *off) + 1;
+ return res;
+ }
+}
+
+/*
+ * first_null_attr
+ * Inspect a NULL bitmap from a tuple and return the 0-based attnum of the
+ * first NULL attribute. Returns natts if no NULLs were found.
+ *
+ * This is coded to expect that 'bits' contains at least one 0 bit somewhere
+ * in the array, but not necessarily < natts. Note that natts may be passed
+ * as a value lower than the number of bits physically stored in the tuple's
+ * NULL bitmap, in which case we may not find a NULL and return natts.
+ *
+ * The reason we require at least one 0 bit somewhere in the NULL bitmap is
+ * that the for loop that checks 0xFF bytes would loop to the last byte in
+ * the array if all bytes were 0xFF, and the subsequent code that finds the
+ * right-most 0 bit would access the first byte beyond the bitmap. Provided
+ * we find a 0 bit before then, that won't happen. Since tuples which have no
+ * NULLs don't have a NULL bitmap, this function won't get called for that
+ * case.
+ */
+static inline int
+first_null_attr(const bits8 *bits, int natts)
+{
+ int nattByte = natts >> 3;
+ int bytenum;
+ int res;
+
+#ifdef USE_ASSERT_CHECKING
+ int firstnull_check = natts;
+
+ /* Do it the slow way and check we get the same answer. */
+ for (int i = 0; i < natts; i++)
+ {
+ if (att_isnull(i, bits))
+ {
+ firstnull_check = i;
+ break;
+ }
+ }
+#endif
+
+ /* Process all bytes up to just before the byte for the natts attribute */
+ for (bytenum = 0; bytenum < nattByte; bytenum++)
+ {
+ /* break if there's any NULL attrs (a 0 bit) */
+ if (bits[bytenum] != 0xFF)
+ break;
+ }
+
+ /*
+ * Look for the highest 0-bit in the 'bytenum' element. To do this, we
+ * promote the uint8 to uint32 before performing the bitwise NOT and
+ * looking for the first 1-bit. This works even when the byte is 0xFF, as
+ * the bitwise NOT of 0xFF in 32 bits is 0xFFFFFF00, in which case
+ * pg_rightmost_one_pos32() will return 8. We may end up with a value
+ * higher than natts here, but we'll fix that with the Min() below.
+ */
+ res = bytenum << 3;
+ res += pg_rightmost_one_pos32(~((uint32) bits[bytenum]));
+
+ /*
+ * Since we did no masking to mask out bits beyond the natt'th bit, we may
+ * have found a bit higher than natts, so we must cap res to natts
+ */
+ res = Min(res, natts);
+
+ /* Ensure we got the same answer as the att_isnull() loop got */
+ Assert(res == firstnull_check);
+
+ return res;
+}
#endif /* FRONTEND */
/*
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 82c442d23f8..b1820653506 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -597,7 +597,8 @@ extern void ExecInitResultTupleSlotTL(PlanState *planstate,
const TupleTableSlotOps *tts_ops);
extern void ExecInitScanTupleSlot(EState *estate, ScanState *scanstate,
TupleDesc tupledesc,
- const TupleTableSlotOps *tts_ops);
+ const TupleTableSlotOps *tts_ops,
+ uint16 flags);
extern TupleTableSlot *ExecInitExtraTupleSlot(EState *estate,
TupleDesc tupledesc,
const TupleTableSlotOps *tts_ops);
diff --git a/src/include/executor/tuptable.h b/src/include/executor/tuptable.h
index 3b09abbf99f..78558098fa3 100644
--- a/src/include/executor/tuptable.h
+++ b/src/include/executor/tuptable.h
@@ -84,9 +84,6 @@
* tts_values/tts_isnull are allocated either when the slot is created (when
* the descriptor is provided), or when a descriptor is assigned to the slot;
* they are of length equal to the descriptor's natts.
- *
- * The TTS_FLAG_SLOW flag is saved state for
- * slot_deform_heap_tuple, and should not be touched by any other code.
*----------
*/
@@ -98,9 +95,13 @@
#define TTS_FLAG_SHOULDFREE (1 << 2)
#define TTS_SHOULDFREE(slot) (((slot)->tts_flags & TTS_FLAG_SHOULDFREE) != 0)
-/* saved state for slot_deform_heap_tuple */
-#define TTS_FLAG_SLOW (1 << 3)
-#define TTS_SLOW(slot) (((slot)->tts_flags & TTS_FLAG_SLOW) != 0)
+/*
+ * true = slot's formed tuple guaranteed to not have NULLs in NOT NULLable
+ * columns.
+ */
+#define TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS (1 << 3)
+#define TTS_OBEYS_NOT_NULL_CONSTRAINTS(slot) \
+ (((slot)->tts_flags & TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS) != 0)
/* fixed tuple descriptor */
#define TTS_FLAG_FIXED (1 << 4)
@@ -123,7 +124,14 @@ typedef struct TupleTableSlot
#define FIELDNO_TUPLETABLESLOT_VALUES 5
Datum *tts_values; /* current per-attribute values */
#define FIELDNO_TUPLETABLESLOT_ISNULL 6
- bool *tts_isnull; /* current per-attribute isnull flags */
+ bool *tts_isnull; /* current per-attribute isnull flags. Array
+ * size must always be rounded up to the next
+ * multiple of 8 elements. */
+ int tts_first_nonguaranteed; /* The value from the TupleDesc's
+ * firstNonGuaranteedAttr, or 0
+ * when tts_flags does not contain
+ * TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS */
+
MemoryContext tts_mcxt; /* slot itself is in this context */
ItemPointerData tts_tid; /* stored tuple's tid */
Oid tts_tableOid; /* table oid of tuple */
@@ -313,9 +321,11 @@ typedef struct MinimalTupleTableSlot
/* in executor/execTuples.c */
extern TupleTableSlot *MakeTupleTableSlot(TupleDesc tupleDesc,
- const TupleTableSlotOps *tts_ops);
+ const TupleTableSlotOps *tts_ops,
+ uint16 flags);
extern TupleTableSlot *ExecAllocTableSlot(List **tupleTable, TupleDesc desc,
- const TupleTableSlotOps *tts_ops);
+ const TupleTableSlotOps *tts_ops,
+ uint16 flags);
extern void ExecResetTupleTable(List *tupleTable, bool shouldFree);
extern TupleTableSlot *MakeSingleTupleTableSlot(TupleDesc tupdesc,
const TupleTableSlotOps *tts_ops);
diff --git a/src/test/modules/deform_bench/deform_bench.c b/src/test/modules/deform_bench/deform_bench.c
index 7838f639bef..4f104989297 100644
--- a/src/test/modules/deform_bench/deform_bench.c
+++ b/src/test/modules/deform_bench/deform_bench.c
@@ -48,7 +48,9 @@ deform_bench(PG_FUNCTION_ARGS)
errmsg("only heap AM is supported")));
tupdesc = RelationGetDescr(rel);
- slot = MakeTupleTableSlot(tupdesc, &TTSOpsBufferHeapTuple);
+ slot = MakeTupleTableSlot(tupdesc,
+ &TTSOpsBufferHeapTuple,
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS);
scan = table_beginscan_strat(rel, GetActiveSnapshot(), 0, NULL, true, false);
/*
--
2.51.0
From a66fa85a01d4c03c570b66afe16ab9b49044b448 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Mon, 23 Feb 2026 09:39:37 +1300
Subject: [PATCH v13 5/6] Reduce size of CompactAttribute struct to 8 bytes
Previously, this was 16 bytes. With the use of some bitflags and by
reducing the attcacheoff field size to a 16-bit type, we can halve the
size of the struct.
It's unlikely that caching the offsets for offsets larger than what will
fit in a 16-bit int will help much as the tuple is very likely to have
some non-fixed-width types anyway, the offsets of which we cannot cache.
---
src/backend/access/common/tupdesc.c | 10 ++++++++++
src/backend/executor/execTuples.c | 17 ++++++++++++-----
src/include/access/tupdesc.h | 16 ++++++++--------
3 files changed, 30 insertions(+), 13 deletions(-)
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index c68561337d7..71461ba6096 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -530,6 +530,16 @@ TupleDescFinalize(TupleDesc tupdesc)
off = att_nominal_alignby(off, cattr->attalignby);
+ /*
+ * attcacheoff is an int16, so don't try to cache any offsets larger
+ * than will fit in that type. Any attributes which are offset more
+ * than 2^15 are likely due to variable-length attributes. Since we
+ * don't cache offsets for or beyond variable-length attributes, using
+ * an int16 rather than an int32 here is unlikely to cost us anything.
+ */
+ if (off > PG_INT16_MAX)
+ break;
+
cattr->attcacheoff = off;
off += cattr->attlen;
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index bfd2286ec2b..d24c07dca9f 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -1013,6 +1013,7 @@ static pg_attribute_always_inline void
slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int reqnatts)
{
+ CompactAttribute *cattrs;
CompactAttribute *cattr;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
HeapTupleHeader tup = tuple->t_data;
@@ -1095,6 +1096,13 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
values = slot->tts_values;
slot->tts_nvalid = reqnatts;
+ /*
+ * We store the tupleDesc's CompactAttribute array in 'cattrs' as gcc
+ * seems to be unwilling to optimize accessing the CompactAttribute
+ * element efficiently when accessing it via TupleDescCompactAttr().
+ */
+ cattrs = tupleDesc->compact_attrs;
+
/* Ensure we calculated tp correctly */
Assert(tp == (char *) tup + tup->t_hoff);
@@ -1105,7 +1113,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
do
{
isnull[attnum] = false;
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
attlen = cattr->attlen;
/* We don't expect any non-byval types */
@@ -1146,9 +1154,8 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
do
{
isnull[attnum] = false;
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
attlen = cattr->attlen;
-
off = cattr->attcacheoff;
values[attnum] = fetch_att_noerr(tp + off,
cattr->attbyval,
@@ -1175,7 +1182,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int attlen;
isnull[attnum] = false;
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
attlen = cattr->attlen;
/*
@@ -1208,7 +1215,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
continue;
}
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
attlen = cattr->attlen;
/* As above, we don't expect cstrings */
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index ad7bc013812..e98036b58bf 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -55,7 +55,7 @@ typedef struct TupleConstr
* directly after the FormData_pg_attribute struct is populated or
* altered in any way.
*
- * Currently, this struct is 16 bytes. Any code changes which enlarge this
+ * Currently, this struct is 8 bytes. Any code changes which enlarge this
* struct should be considered very carefully.
*
* Code which must access a TupleDesc's attribute data should always make use
@@ -67,17 +67,17 @@ typedef struct TupleConstr
*/
typedef struct CompactAttribute
{
- int32 attcacheoff; /* fixed offset into tuple, if known, or -1 */
+ int16 attcacheoff; /* fixed offset into tuple, if known, or -1 */
int16 attlen; /* attr len in bytes or -1 = varlen, -2 =
* cstring */
bool attbyval; /* as FormData_pg_attribute.attbyval */
- bool attispackable; /* FormData_pg_attribute.attstorage !=
- * TYPSTORAGE_PLAIN */
- bool atthasmissing; /* as FormData_pg_attribute.atthasmissing */
- bool attisdropped; /* as FormData_pg_attribute.attisdropped */
- bool attgenerated; /* FormData_pg_attribute.attgenerated != '\0' */
- char attnullability; /* status of not-null constraint, see below */
uint8 attalignby; /* alignment requirement in bytes */
+ bool attispackable:1; /* FormData_pg_attribute.attstorage !=
+ * TYPSTORAGE_PLAIN */
+ bool atthasmissing:1; /* as FormData_pg_attribute.atthasmissing */
+ bool attisdropped:1; /* as FormData_pg_attribute.attisdropped */
+ bool attgenerated:1; /* FormData_pg_attribute.attgenerated != '\0' */
+ char attnullability; /* status of not-null constraint, see below */
} CompactAttribute;
/* Valid values for CompactAttribute->attnullability */
--
2.51.0
From ca630369d8f299c4425817479a164a6e9b37cd72 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Wed, 4 Mar 2026 16:55:09 +1300
Subject: [PATCH v13 6/6] WIP: Introduce selective tuple deforming
Up until now, we have always deformed every attribute of each tuple up
until the last attribute that we require. This did once make sense to
do as we often had to walk the tuple in order to determine the byte
offset to any attribute that we deform. Now, since we proactively
populate the CompactAttribute.attcacheoff, in some cases we might be in
a better position to only deform the attributes that we need to deform
by directly jumping to the cached byte offset and skipping deforming any
attributes which are not required. In some cases the savings can be
very large as it not only allows tts_values and tts_isnull for unneeded
attributes to be left unpopulated, but it might also mean we can skip
loading entire cachelines when the tuple is wide enough to span multiple
cachelines.
We don't want to exclusively always deform tuples this way as doing this
means paying attention to an additional array which states which attnums
we must deform. Looking at that array for a SELECT * query, which
requires us to deform all attributes, would add overhead. To support
this a new expression evaluation operator has been added called
EEOP_SCAN_SELECTSOME and each function which builds an ExprState now
accepts a variant function that allows the caller to specify which attnums
are required from the scan side. This puts it on the caller to decide
which type of deforming should be done. When the caller provides
the attnums, the expression will be built with EEOP_SCAN_SELECTSOME
rather than EEOP_SCAN_FETCHSOME. This currently does not interact well
with the physical tlist optimization. Currently it's the planner's job
to figure out which attributes are actually required.
TODO: JIT support
---
src/backend/executor/execExpr.c | 182 ++++++++-
src/backend/executor/execExprInterp.c | 13 +
src/backend/executor/execScan.c | 18 +
src/backend/executor/execTuples.c | 364 +++++++++++++++++-
src/backend/executor/execUtils.c | 47 ++-
src/backend/executor/nodeSeqscan.c | 8 +-
src/backend/optimizer/plan/createplan.c | 43 ++-
src/include/executor/execExpr.h | 24 +-
src/include/executor/executor.h | 19 +
src/include/executor/tuptable.h | 22 ++
src/include/nodes/plannodes.h | 8 +
.../deform_bench/deform_bench--1.0.sql | 4 +
src/test/modules/deform_bench/deform_bench.c | 98 +++++
13 files changed, 821 insertions(+), 29 deletions(-)
diff --git a/src/backend/executor/execExpr.c b/src/backend/executor/execExpr.c
index bd46b75e498..a9dd2842ea5 100644
--- a/src/backend/executor/execExpr.c
+++ b/src/backend/executor/execExpr.c
@@ -66,8 +66,18 @@ typedef struct ExprSetupInfo
AttrNumber last_new;
/* MULTIEXPR SubPlan nodes appearing in the expression: */
List *multiexpr_subplans;
+
+ /*
+ * Fetch only these attnums from the scan with EEOP_SCAN_SELECTSOME. Empty
+ * set means use EEOP_SCAN_FETCHSOME (i.e fetch all up until last_scan).
+ * The first user attribute is based at member 0. System attributes not
+ * represented.
+ */
+ Bitmapset *scan_attrs;
} ExprSetupInfo;
+static ExprState *ExecInitExprInternal(Expr *node, PlanState *parent,
+ Node *escontext, Bitmapset *scan_attrs);
static void ExecReadyExpr(ExprState *state);
static void ExecInitExprRec(Expr *node, ExprState *state,
Datum *resv, bool *resnull);
@@ -77,7 +87,8 @@ static void ExecInitFunc(ExprEvalStep *scratch, Expr *node, List *args,
static void ExecInitSubPlanExpr(SubPlan *subplan,
ExprState *state,
Datum *resv, bool *resnull);
-static void ExecCreateExprSetupSteps(ExprState *state, Node *node);
+static void ExecCreateExprSetupSteps(ExprState *state, Node *node,
+ Bitmapset *scan_attrs);
static void ExecPushExprSetupSteps(ExprState *state, ExprSetupInfo *info);
static bool expr_setup_walker(Node *node, ExprSetupInfo *info);
static bool ExecComputeSlotInfo(ExprState *state, ExprEvalStep *op);
@@ -142,7 +153,7 @@ static void ExecInitJsonCoercion(ExprState *state, JsonReturning *returning,
ExprState *
ExecInitExpr(Expr *node, PlanState *parent)
{
- return ExecInitExprWithContext(node, parent, NULL);
+ return ExecInitExprInternal(node, parent, NULL, NULL);
}
/*
@@ -161,6 +172,31 @@ ExecInitExpr(Expr *node, PlanState *parent)
*/
ExprState *
ExecInitExprWithContext(Expr *node, PlanState *parent, Node *escontext)
+{
+ return ExecInitExprInternal(node, parent, escontext, NULL);
+}
+
+/*
+ * ExecInitExprWithScanAttrs
+ * As ExecInitExpr but when 'scan_attrs' is set, use
+ * EEOP_SCAN_SELECTSOME instead of EEOP_SCAN_FETCHSOME to deform only
+ * the mentioned 'scan_attrs' from the scan tuple.
+ */
+ExprState *
+ExecInitExprWithScanAttrs(Expr *node, PlanState *parent,
+ Bitmapset *scan_attrs)
+{
+ return ExecInitExprInternal(node, parent, NULL, scan_attrs);
+}
+
+/*
+ * ExecInitExprInteral
+ * Internal version to implement ExecInitExpr, ExecInitExprWithContext
+ * and ExecInitExprWithScanAttrs.
+ */
+static ExprState *
+ExecInitExprInternal(Expr *node, PlanState *parent, Node *escontext,
+ Bitmapset *scan_attrs)
{
ExprState *state;
ExprEvalStep scratch = {0};
@@ -177,7 +213,7 @@ ExecInitExprWithContext(Expr *node, PlanState *parent, Node *escontext)
state->escontext = (ErrorSaveContext *) escontext;
/* Insert setup steps as needed */
- ExecCreateExprSetupSteps(state, (Node *) node);
+ ExecCreateExprSetupSteps(state, (Node *) node, scan_attrs);
/* Compile the expression proper */
ExecInitExprRec(node, state, &state->resvalue, &state->resnull);
@@ -214,7 +250,7 @@ ExecInitExprWithParams(Expr *node, ParamListInfo ext_params)
state->ext_params = ext_params;
/* Insert setup steps as needed */
- ExecCreateExprSetupSteps(state, (Node *) node);
+ ExecCreateExprSetupSteps(state, (Node *) node, NULL);
/* Compile the expression proper */
ExecInitExprRec(node, state, &state->resvalue, &state->resnull);
@@ -248,6 +284,19 @@ ExecInitExprWithParams(Expr *node, ParamListInfo ext_params)
*/
ExprState *
ExecInitQual(List *qual, PlanState *parent)
+{
+ return ExecInitQualWithScanAttrs(qual, parent, NULL);
+}
+
+/*
+ * ExecInitQualWithScanAttrs
+ * As ExecInitQual but when 'scan_attrs' is set, use
+ * EEOP_SCAN_SELECTSOME instead of EEOP_SCAN_FETCHSOME to deform only
+ * the mentioned 'scan_attrs' from the scan tuple.
+ */
+ExprState *
+ExecInitQualWithScanAttrs(List *qual, PlanState *parent,
+ Bitmapset *scan_attrs)
{
ExprState *state;
ExprEvalStep scratch = {0};
@@ -268,7 +317,7 @@ ExecInitQual(List *qual, PlanState *parent)
state->flags = EEO_FLAG_IS_QUAL;
/* Insert setup steps as needed */
- ExecCreateExprSetupSteps(state, (Node *) qual);
+ ExecCreateExprSetupSteps(state, (Node *) qual, scan_attrs);
/*
* ExecQual() needs to return false for an expression returning NULL. That
@@ -393,6 +442,28 @@ ExecBuildProjectionInfo(List *targetList,
TupleTableSlot *slot,
PlanState *parent,
TupleDesc inputDesc)
+{
+ return ExecBuildProjectionInfoWithScanAttrs(targetList,
+ econtext,
+ slot,
+ parent,
+ inputDesc,
+ NULL);
+}
+
+/*
+ * ExecBuildProjectionInfoWithScanAttrs
+ * As ExecBuildProjectionInfo but when 'scan_attrs' is set, use
+ * EEOP_SCAN_SELECTSOME instead of EEOP_SCAN_FETCHSOME to deform only
+ * the mentioned 'scan_attrs' from the scan tuple.
+ */
+ProjectionInfo *
+ExecBuildProjectionInfoWithScanAttrs(List *targetList,
+ ExprContext *econtext,
+ TupleTableSlot *slot,
+ PlanState *parent,
+ TupleDesc inputDesc,
+ Bitmapset *scan_attrs)
{
ProjectionInfo *projInfo = makeNode(ProjectionInfo);
ExprState *state;
@@ -410,7 +481,7 @@ ExecBuildProjectionInfo(List *targetList,
state->resultslot = slot;
/* Insert setup steps as needed */
- ExecCreateExprSetupSteps(state, (Node *) targetList);
+ ExecCreateExprSetupSteps(state, (Node *) targetList, scan_attrs);
/* Now compile each tlist column */
foreach(lc, targetList)
@@ -2904,11 +2975,19 @@ ExecInitSubPlanExpr(SubPlan *subplan,
/*
* Add expression steps performing setup that's needed before any of the
* main execution of the expression.
+ *
+ * 'scan_attrs' may be given an empty set, in which case deforming the scan
+ * tuple is done via EEOP_SCAN_FETCHSOME, which fetches every attribute from
+ * the scan tuple up until the maximum attribute used by this expression.
+ * When 'scan_attrs' is set, EEOP_SCAN_SELECTSOME is used to only fetch the
+ * attributes mentioned. Callers must create a unioned set of the attributes
+ * needed from the scan for all expressions using the given slot so that we
+ * incrementally fetch the attributes required by all ExprStates.
*/
static void
-ExecCreateExprSetupSteps(ExprState *state, Node *node)
+ExecCreateExprSetupSteps(ExprState *state, Node *node, Bitmapset *scan_attrs)
{
- ExprSetupInfo info = {0, 0, 0, 0, 0, NIL};
+ ExprSetupInfo info = {0, 0, 0, 0, 0, NIL, scan_attrs};
/* Prescan to find out what we need. */
expr_setup_walker(node, &info);
@@ -2956,11 +3035,75 @@ ExecPushExprSetupSteps(ExprState *state, ExprSetupInfo *info)
}
if (info->last_scan > 0)
{
- scratch.opcode = EEOP_SCAN_FETCHSOME;
- scratch.d.fetch.last_var = info->last_scan;
- scratch.d.fetch.fixed = false;
- scratch.d.fetch.kind = NULL;
- scratch.d.fetch.known_desc = NULL;
+ /*
+ * We have two operators for fetching attributes out of a tuple during
+ * scans. EEOP_SCAN_FETCHSOME deforms all attributes in the tuple up
+ * to the 'last_scan' attnum. This isn't ideal in some cases, as we
+ * may only need a few attributes, and those might be deep into the
+ * tuple. EEOP_SCAN_SELECTSOME is an operator that fetches only the
+ * required attributes from the tuple. When the attcacheoff for these
+ * attributes is known and no NULLs exist in the tuple prior to the
+ * required attributes, then this can be a very fast operation.
+ * EEOP_SCAN_FETCHSOME is still supported as many cases require all
+ * attributes, and EEOP_SCAN_FETCHSOME can do this more efficiently.
+ */
+ if (bms_is_empty(info->scan_attrs))
+ {
+ scratch.opcode = EEOP_SCAN_FETCHSOME;
+ scratch.d.fetch.last_var = info->last_scan;
+ scratch.d.fetch.fixed = false;
+ scratch.d.fetch.kind = NULL;
+ scratch.d.fetch.known_desc = NULL;
+ }
+ else
+ {
+ int nattrs = bms_num_members(info->scan_attrs);
+ AttrNumber *atts;
+ int a;
+ int i;
+
+ scratch.opcode = EEOP_SCAN_SELECTSOME;
+ scratch.d.fetch.last_var = info->last_scan;
+ scratch.d.fetch.fixed = false;
+ scratch.d.fetch.natts = nattrs;
+ scratch.d.fetch.kind = NULL;
+ scratch.d.fetch.known_desc = NULL;
+
+ /*
+ * Allocate these two arrays as a single allocation. The
+ * req_attnums array needs 1 element for each attnum that's being
+ * selected, plus a sentinel attnum which we set to the
+ * 'last_scan' attnum so that we correctly terminate each of the
+ * loops during selective deformation before walking off the end
+ * of the array.
+ */
+ atts = palloc_array(AttrNumber, nattrs + 1 + info->last_scan + 1);
+
+ scratch.d.fetch.req_attnums = atts;
+ scratch.d.fetch.next_req_attnums_index = &atts[nattrs + 1];
+
+ /* Store each attnum in the Bitmapset into the req_attnum array */
+ a = -1;
+ i = 0;
+ while ((a = bms_next_member(info->scan_attrs, a)) >= 0)
+ scratch.d.fetch.req_attnums[i++] = a;
+
+ /* install sentinel */
+ scratch.d.fetch.req_attnums[nattrs] = info->last_scan;
+
+ /*
+ * Populate the next_req_attnums_index array. This allows the
+ * deforming function to refind the position in the
+ * next_req_attnums_index array from tts_nvalid.
+ */
+ a = 0;
+ for (i = 0; i <= info->last_scan; i++)
+ {
+ scratch.d.fetch.next_req_attnums_index[i] = a;
+ if (bms_is_member(i, info->scan_attrs))
+ a++;
+ }
+ }
if (ExecComputeSlotInfo(state, &scratch))
ExprEvalPushStep(state, &scratch);
}
@@ -3033,6 +3176,13 @@ expr_setup_walker(Node *node, ExprSetupInfo *info)
switch (variable->varreturningtype)
{
case VAR_RETURNING_DEFAULT:
+
+ /*
+ * scan_attrs must contain a member for this attnum or
+ * be completely empty
+ */
+ Assert(attnum < 0 || bms_is_empty(info->scan_attrs) ||
+ bms_is_member(attnum - 1, info->scan_attrs));
info->last_scan = Max(info->last_scan, attnum);
break;
case VAR_RETURNING_OLD:
@@ -3099,7 +3249,8 @@ ExecComputeSlotInfo(ExprState *state, ExprEvalStep *op)
opcode == EEOP_OUTER_FETCHSOME ||
opcode == EEOP_SCAN_FETCHSOME ||
opcode == EEOP_OLD_FETCHSOME ||
- opcode == EEOP_NEW_FETCHSOME);
+ opcode == EEOP_NEW_FETCHSOME ||
+ opcode == EEOP_SCAN_SELECTSOME);
if (op->d.fetch.known_desc != NULL)
{
@@ -3152,6 +3303,7 @@ ExecComputeSlotInfo(ExprState *state, ExprEvalStep *op)
}
}
else if (opcode == EEOP_SCAN_FETCHSOME ||
+ opcode == EEOP_SCAN_SELECTSOME ||
opcode == EEOP_OLD_FETCHSOME ||
opcode == EEOP_NEW_FETCHSOME)
{
@@ -4344,7 +4496,7 @@ ExecBuildHash32Expr(TupleDesc desc, const TupleTableSlotOps *ops,
state->parent = parent;
/* Insert setup steps as needed. */
- ExecCreateExprSetupSteps(state, (Node *) hash_exprs);
+ ExecCreateExprSetupSteps(state, (Node *) hash_exprs, NULL);
/*
* Make a place to store intermediate hash values between subsequent
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 61ff5ddc74c..76965826f83 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -479,6 +479,7 @@ ExecInterpExpr(ExprState *state, ExprContext *econtext, bool *isnull)
&&CASE_EEOP_SCAN_FETCHSOME,
&&CASE_EEOP_OLD_FETCHSOME,
&&CASE_EEOP_NEW_FETCHSOME,
+ &&CASE_EEOP_SCAN_SELECTSOME,
&&CASE_EEOP_INNER_VAR,
&&CASE_EEOP_OUTER_VAR,
&&CASE_EEOP_SCAN_VAR,
@@ -676,6 +677,18 @@ ExecInterpExpr(ExprState *state, ExprContext *econtext, bool *isnull)
EEO_NEXT();
}
+ EEO_CASE(EEOP_SCAN_SELECTSOME)
+ {
+ CheckOpSlotCompatibility(op, scanslot);
+
+ slot_selectattrs(scanslot,
+ op->d.fetch.last_var,
+ op->d.fetch.req_attnums,
+ op->d.fetch.next_req_attnums_index);
+
+ EEO_NEXT();
+ }
+
EEO_CASE(EEOP_INNER_VAR)
{
int attnum = op->d.var.attnum;
diff --git a/src/backend/executor/execScan.c b/src/backend/executor/execScan.c
index 9f68be17b99..525af11aa08 100644
--- a/src/backend/executor/execScan.c
+++ b/src/backend/executor/execScan.c
@@ -86,6 +86,24 @@ ExecAssignScanProjectionInfo(ScanState *node)
ExecConditionalAssignProjectionInfo(&node->ps, tupdesc, scan->scanrelid);
}
+/*
+ * ExecAssignScanProjectionInfoWithScanAttrs
+ * As ExecAssignScanProjectionInfo but when 'scan_attrs' is set, use
+ * EEOP_SCAN_SELECTSOME instead of EEOP_SCAN_FETCHSOME to deform only
+ * the mentioned 'scan_attrs' from the scan tuple.
+ */
+void
+ExecAssignScanProjectionInfoWithScanAttrs(ScanState *node,
+ Bitmapset *scan_attrs)
+{
+ Scan *scan = (Scan *) node->ps.plan;
+ TupleDesc tupdesc = node->ss_ScanTupleSlot->tts_tupleDescriptor;
+
+ ExecConditionalAssignProjectionInfoWithScanAttrs(&node->ps, tupdesc,
+ scan->scanrelid,
+ scan_attrs);
+}
+
/*
* ExecAssignScanProjectionInfoWithVarno
* As above, but caller can specify varno expected in Vars in the tlist.
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index d24c07dca9f..a65b4ca2ee6 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -74,6 +74,12 @@ static TupleDesc ExecTypeFromTLInternal(List *targetList,
bool skipjunk);
static pg_attribute_always_inline void slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int reqnatts);
+static pg_attribute_always_inline void slot_selectively_deform_heap_tuple(TupleTableSlot *slot,
+ HeapTuple tuple,
+ uint32 *offp,
+ int last_attnum,
+ AttrNumber *attnums,
+ AttrNumber *attnum_map);
static inline void tts_buffer_heap_store_tuple(TupleTableSlot *slot,
HeapTuple tuple,
Buffer buffer,
@@ -129,7 +135,22 @@ tts_virtual_clear(TupleTableSlot *slot)
static void
tts_virtual_getsomeattrs(TupleTableSlot *slot, int natts)
{
- elog(ERROR, "getsomeattrs is not required to be called on a virtual tuple table slot");
+ elog(ERROR,
+ "%s is not required to be called on a virtual tuple table slot",
+ "getsomeattrs");
+}
+
+/*
+ * VirtualTupleTableSlots always have fully populated tts_values and
+ * tts_isnull arrays. So this function should never be called.
+ */
+static void
+tts_virtual_selectattrs(TupleTableSlot *slot, int last_attnum,
+ AttrNumber *attnums, AttrNumber *attnum_map)
+{
+ elog(ERROR,
+ "%s is not required to be called on a virtual tuple table slot",
+ "selectattrs");
}
/*
@@ -352,6 +373,22 @@ tts_heap_getsomeattrs(TupleTableSlot *slot, int natts)
slot_deform_heap_tuple(slot, hslot->tuple, &hslot->off, natts);
}
+static void
+tts_heap_selectattrs(TupleTableSlot *slot, int last_attnum,
+ AttrNumber *attnums, AttrNumber *attnum_map)
+{
+ HeapTupleTableSlot *hslot = (HeapTupleTableSlot *) slot;
+
+ Assert(!TTS_EMPTY(slot));
+
+ slot_selectively_deform_heap_tuple(slot,
+ hslot->tuple,
+ &hslot->off,
+ last_attnum,
+ attnums,
+ attnum_map);
+}
+
static Datum
tts_heap_getsysattr(TupleTableSlot *slot, int attnum, bool *isnull)
{
@@ -550,6 +587,22 @@ tts_minimal_getsomeattrs(TupleTableSlot *slot, int natts)
slot_deform_heap_tuple(slot, mslot->tuple, &mslot->off, natts);
}
+static void
+tts_minimal_selectattrs(TupleTableSlot *slot, int last_attnum,
+ AttrNumber *attnums, AttrNumber *attnum_map)
+{
+ MinimalTupleTableSlot *mslot = (MinimalTupleTableSlot *) slot;
+
+ Assert(!TTS_EMPTY(slot));
+
+ slot_selectively_deform_heap_tuple(slot,
+ mslot->tuple,
+ &mslot->off,
+ last_attnum,
+ attnums,
+ attnum_map);
+}
+
/*
* MinimalTupleTableSlots never provide system attributes. We generally
* shouldn't get here, but provide a user-friendly message if we do.
@@ -757,6 +810,23 @@ tts_buffer_heap_getsomeattrs(TupleTableSlot *slot, int natts)
slot_deform_heap_tuple(slot, bslot->base.tuple, &bslot->base.off, natts);
}
+static void
+tts_buffer_heap_selectattrs(TupleTableSlot *slot, int last_attnum,
+ AttrNumber *attnums, AttrNumber *attnum_map)
+{
+ BufferHeapTupleTableSlot *bslot = (BufferHeapTupleTableSlot *) slot;
+
+ Assert(!TTS_EMPTY(slot));
+
+ slot_selectively_deform_heap_tuple(slot,
+ bslot->base.tuple,
+ &bslot->base.off,
+ last_attnum,
+ attnums,
+ attnum_map);
+}
+
+
static Datum
tts_buffer_heap_getsysattr(TupleTableSlot *slot, int attnum, bool *isnull)
{
@@ -1242,12 +1312,301 @@ done:
*offp = off;
}
+/*
+ * slot_selectively_deform_heap_tuple
+ * Deform attributes of 'tuple' into the Datum/isnull arrays in 'slot'.
+ * Unlike slot_deform_heap_tuple, which deforms every attribute up to the
+ * given attribute number, here we deform only the attribute numbers
+ * mentioned in the 'attnums' array. When only a few attributes are
+ * required, this can be more efficient. When the attributes have a
+ * known attcacheoff and it's valid to use that, then this version can be
+ * much more efficient than slot_deform_heap_tuple when only a small
+ * number of the total attributes are required.
+ */
+static pg_attribute_always_inline void
+slot_selectively_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple,
+ uint32 *offp, int last_attnum,
+ AttrNumber *attnums,
+ AttrNumber *attnum_map)
+{
+ CompactAttribute *cattrs;
+ CompactAttribute *cattr;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+ HeapTupleHeader tup = tuple->t_data;
+ size_t attnum;
+ int attnums_idx;
+ int firstNonCacheOffsetAttr;
+ int firstNonGuaranteedAttr;
+ int firstNullAttr;
+ int natts;
+ Datum *values;
+ bool *isnull;
+ char *tp; /* ptr to tuple data */
+ uint32 off; /* offset in tuple data */
+ int off_attnum; /* the attnum that 'off' points to */
+
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
+
+ isnull = slot->tts_isnull;
+
+ /*
+ * Some callers may form and deform tuples prior to NOT NULL constraints
+ * being checked. Here we'd like to optimize the case where we only need
+ * to fetch attributes before or up to the point where the attribute is
+ * guaranteed to exist in the tuple. We rely on the slot flag being set
+ * correctly to only enable this optimization when it's valid to do so.
+ * This optimization allows us to save fetching the number of attributes
+ * from the tuple and saves the additional cost of handling non-byval
+ * attrs.
+ */
+ firstNonGuaranteedAttr = Min(last_attnum, slot->tts_first_nonguaranteed);
+ firstNonCacheOffsetAttr = tupleDesc->firstNonCachedOffsetAttr;
+
+ if (HeapTupleHasNulls(tuple))
+ {
+ natts = HeapTupleHeaderGetNatts(tup);
+ tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits) +
+ BITMAPLEN(natts));
+
+ natts = Min(natts, last_attnum);
+ if (natts > firstNonGuaranteedAttr)
+ {
+ bits8 *bp = tup->t_bits;
+
+ /* Find the first NULL attr */
+ firstNullAttr = first_null_attr(bp, natts);
+
+ /*
+ * And populate the isnull array for all attributes up to the last
+ * attribute we're deforming. When not using attcacheoff, we need
+ * to know if an attribute is NULL even when we're not deforming
+ * it, so that we can skip over it when calculating the offset to
+ * attributes that we are deforming.
+ */
+ populate_isnull_array(bp, natts, isnull);
+ }
+ else
+ {
+ /* Otherwise all required columns are guaranteed to exist */
+ firstNullAttr = natts;
+ }
+ }
+ else
+ {
+ tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits));
+
+ /*
+ * We only need to look at the tuple's natts if we need more than the
+ * guaranteed number of columns
+ */
+ if (last_attnum > firstNonGuaranteedAttr)
+ natts = Min(HeapTupleHeaderGetNatts(tup), last_attnum);
+ else
+ {
+ /* No need to access the number of attributes in the tuple */
+ natts = last_attnum;
+ }
+
+ /* All attrs can be fetched without checking for NULLs */
+ firstNullAttr = natts;
+ }
+
+ attnums_idx = attnum_map[slot->tts_nvalid];
+ attnum = attnums[attnums_idx];
+ values = slot->tts_values;
+
+ /*
+ * We store the tupleDesc's CompactAttribute array in 'cattrs' as gcc
+ * seems to be unwilling to optimize accessing the CompactAttribute
+ * element efficiently when accessing it via TupleDescCompactAttr().
+ */
+ cattrs = tupleDesc->compact_attrs;
+
+ /* Ensure we calculated tp correctly */
+ Assert(tp == (char *) tup + tup->t_hoff);
+
+ if (attnum < firstNonGuaranteedAttr)
+ {
+ int attlen;
+
+ /*
+ * We use a do/while loop as the if condition above guarantees at
+ * least one loop and confirms to the compiler that 'attlen' and 'off'
+ * get initialized, which some compilers are not clever enough to
+ * figure out if we were to use a for loop.
+ */
+ do
+ {
+ isnull[attnum] = false;
+ cattr = &cattrs[attnum];
+ attlen = cattr->attlen;
+
+ /* We don't expect any non-byval types */
+ pg_assume(attlen > 0);
+ Assert(cattr->attbyval == true);
+
+ off = cattr->attcacheoff;
+ values[attnum] = fetch_att_noerr(tp + off, true, attlen);
+ attnum = attnums[++attnums_idx];
+ } while (attnum < firstNonGuaranteedAttr);
+
+ off += attlen;
+
+ if (attnum == last_attnum)
+ goto done;
+ }
+
+ /* We can use attcacheoff up until the first NULL */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, firstNullAttr);
+
+ /*
+ * Handle the portion of the tuple that we have cached the offset for up
+ * to the first NULL attribute. The offset is effectively fixed for
+ * these, so we can use the CompactAttribute's attcacheoff.
+ */
+ if (attnum < firstNonCacheOffsetAttr)
+ {
+ int attlen;
+
+ do
+ {
+ isnull[attnum] = false;
+ cattr = &cattrs[attnum];
+ attlen = cattr->attlen;
+ off = cattr->attcacheoff;
+ values[attnum] = fetch_att_noerr(tp + off, cattr->attbyval,
+ attlen);
+ attnum = attnums[++attnums_idx];
+ } while (attnum < firstNonCacheOffsetAttr);
+
+ off += attlen;
+ Assert(attlen > 0);
+
+ if (attnum == last_attnum)
+ goto done;
+ }
+
+
+ if (slot->tts_nvalid >= firstNonCacheOffsetAttr)
+ {
+ /* Restore state from previous execution */
+ off_attnum = slot->tts_nvalid;
+ off = *offp;
+ }
+ else
+ {
+ off_attnum = firstNonCacheOffsetAttr - 1;
+ off = cattrs[off_attnum].attcacheoff;
+ }
+
+ /*
+ * We no longer have the ability to use attcacheoff, so we must look
+ * through all attributes from this point on. For attributes that we are
+ * not selecting, we only calculate the offset to skip them, and don't do
+ * the actual fetch. Here we loop up to the first NULL attribute.
+ */
+ for (; off_attnum < firstNullAttr; off_attnum++)
+ {
+ int attlen;
+
+ cattr = &cattrs[off_attnum];
+ attlen = cattr->attlen;
+
+ /*
+ * cstrings don't exist in heap tuples. Use pg_assume to instruct the
+ * compiler not to emit the cstring-related code in
+ * align_fetch_then_add().
+ */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ off = att_pointer_alignby(off, cattr->attalignby, attlen, tp + off);
+
+ /*
+ * If this is an attribute we want, do the fetch and then move attnum
+ * to the next attribute we want.
+ */
+ if (off_attnum == attnum)
+ {
+ isnull[off_attnum] = false;
+ values[off_attnum] =
+ fetch_att_noerr(tp + off, cattr->attbyval,
+ attlen);
+ attnum = attnums[++attnums_idx];
+
+ }
+ /* Move offset beyond this attribute */
+ off = att_addlength_pointer(off, attlen, tp + off);
+ }
+
+ /*
+ * Now handle any remaining attributes in the tuple up to the requested
+ * attnum. This time, include NULL checks as we're now at the first NULL
+ * attribute.
+ */
+ for (; off_attnum < natts; off_attnum++)
+ {
+ int attlen;
+
+ cattr = &cattrs[off_attnum];
+ attlen = cattr->attlen;
+
+ /* As above, we don't expect cstrings */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ /* Is this an attribute we're selecting? */
+ if (off_attnum == attnum)
+ {
+ attnum = attnums[++attnums_idx];
+
+ if (isnull[off_attnum])
+ {
+ values[off_attnum] = (Datum) 0;
+ continue;
+ }
+
+ /*
+ * align 'off', fetch the datum, and increment off beyond the
+ * datum
+ */
+ values[off_attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ attlen,
+ cattr->attalignby);
+ }
+ else if (!isnull[off_attnum])
+ {
+ /* We don't want this attribute, move beyond it */
+ off = att_pointer_alignby(off, cattr->attalignby, attlen, tp + off);
+ off = att_addlength_pointer(off, attlen, tp + off);
+ }
+
+ }
+
+ /* Fetch any missing attrs and raise an error if reqnatts is invalid */
+ if (unlikely(attnum < last_attnum))
+ {
+ *offp = off;
+ /* XXX worth doing this selectively too? */
+ slot_getmissingattrs(slot, attnum, last_attnum);
+ slot->tts_nvalid = last_attnum;
+ return;
+ }
+done:
+
+ slot->tts_nvalid = last_attnum;
+ /* Save current offset for next execution */
+ *offp = off;
+}
+
const TupleTableSlotOps TTSOpsVirtual = {
.base_slot_size = sizeof(VirtualTupleTableSlot),
.init = tts_virtual_init,
.release = tts_virtual_release,
.clear = tts_virtual_clear,
.getsomeattrs = tts_virtual_getsomeattrs,
+ .selectattrs = tts_virtual_selectattrs,
.getsysattr = tts_virtual_getsysattr,
.materialize = tts_virtual_materialize,
.is_current_xact_tuple = tts_virtual_is_current_xact_tuple,
@@ -1269,6 +1628,7 @@ const TupleTableSlotOps TTSOpsHeapTuple = {
.release = tts_heap_release,
.clear = tts_heap_clear,
.getsomeattrs = tts_heap_getsomeattrs,
+ .selectattrs = tts_heap_selectattrs,
.getsysattr = tts_heap_getsysattr,
.is_current_xact_tuple = tts_heap_is_current_xact_tuple,
.materialize = tts_heap_materialize,
@@ -1287,6 +1647,7 @@ const TupleTableSlotOps TTSOpsMinimalTuple = {
.release = tts_minimal_release,
.clear = tts_minimal_clear,
.getsomeattrs = tts_minimal_getsomeattrs,
+ .selectattrs = tts_minimal_selectattrs,
.getsysattr = tts_minimal_getsysattr,
.is_current_xact_tuple = tts_minimal_is_current_xact_tuple,
.materialize = tts_minimal_materialize,
@@ -1305,6 +1666,7 @@ const TupleTableSlotOps TTSOpsBufferHeapTuple = {
.release = tts_buffer_heap_release,
.clear = tts_buffer_heap_clear,
.getsomeattrs = tts_buffer_heap_getsomeattrs,
+ .selectattrs = tts_buffer_heap_selectattrs,
.getsysattr = tts_buffer_heap_getsysattr,
.is_current_xact_tuple = tts_buffer_is_current_xact_tuple,
.materialize = tts_buffer_heap_materialize,
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index f62582859f9..252e8306631 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -580,8 +580,7 @@ ExecGetCommonChildSlotOps(PlanState *ps)
* ----------------
*/
void
-ExecAssignProjectionInfo(PlanState *planstate,
- TupleDesc inputDesc)
+ExecAssignProjectionInfo(PlanState *planstate, TupleDesc inputDesc)
{
planstate->ps_ProjInfo =
ExecBuildProjectionInfo(planstate->plan->targetlist,
@@ -591,6 +590,28 @@ ExecAssignProjectionInfo(PlanState *planstate,
inputDesc);
}
+/* ----------------
+ * ExecAssignProjectionInfoWithScanAttrs
+ *
+ * As ExecAssignScanProjectionInfo but when 'scan_attrs' is set, use
+ * EEOP_SCAN_SELECTSOME instead of EEOP_SCAN_FETCHSOME to deform only the
+ * mentioned 'scan_attrs' from the scan tuple.
+ * ----------------
+*/
+void
+ExecAssignProjectionInfoWithScanAttrs(PlanState *planstate,
+ TupleDesc inputDesc,
+ Bitmapset *scan_attrs)
+{
+ planstate->ps_ProjInfo =
+ ExecBuildProjectionInfoWithScanAttrs(planstate->plan->targetlist,
+ planstate->ps_ExprContext,
+ planstate->ps_ResultTupleSlot,
+ planstate,
+ inputDesc,
+ scan_attrs);
+}
+
/* ----------------
* ExecConditionalAssignProjectionInfo
@@ -602,6 +623,26 @@ ExecAssignProjectionInfo(PlanState *planstate,
void
ExecConditionalAssignProjectionInfo(PlanState *planstate, TupleDesc inputDesc,
int varno)
+{
+ ExecConditionalAssignProjectionInfoWithScanAttrs(planstate,
+ inputDesc,
+ varno,
+ NULL);
+}
+
+/* ----------------
+ * ExecConditionalAssignProjectionInfoWithScanAttrs
+ *
+ * As ExecConditionalAssignProjectionInfo but when 'scan_attrs' is set, use
+ * EEOP_SCAN_SELECTSOME instead of EEOP_SCAN_FETCHSOME to deform only the
+ * mentioned 'scan_attrs' from the scan tuple.
+ * ----------------
+*/
+void
+ExecConditionalAssignProjectionInfoWithScanAttrs(PlanState *planstate,
+ TupleDesc inputDesc,
+ int varno,
+ Bitmapset *scan_attrs)
{
if (tlist_matches_tupdesc(planstate,
planstate->plan->targetlist,
@@ -622,7 +663,7 @@ ExecConditionalAssignProjectionInfo(PlanState *planstate, TupleDesc inputDesc,
planstate->resultopsfixed = true;
planstate->resultopsset = true;
}
- ExecAssignProjectionInfo(planstate, inputDesc);
+ ExecAssignProjectionInfoWithScanAttrs(planstate, inputDesc, scan_attrs);
}
}
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 8f219f60a93..41de367832c 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -251,13 +251,15 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags)
* Initialize result type and projection.
*/
ExecInitResultTypeTL(&scanstate->ss.ps);
- ExecAssignScanProjectionInfo(&scanstate->ss);
+ ExecAssignScanProjectionInfoWithScanAttrs(&scanstate->ss,
+ node->scan.scan_varattnos);
/*
* initialize child expressions
*/
- scanstate->ss.ps.qual =
- ExecInitQual(node->scan.plan.qual, (PlanState *) scanstate);
+ scanstate->ss.ps.qual = ExecInitQualWithScanAttrs(node->scan.plan.qual,
+ (PlanState *) scanstate,
+ node->scan.scan_varattnos);
/*
* When EvalPlanQual() is not in use, assign ExecProcNode for this node
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 50b0e10308b..4522ac4d4c0 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -118,7 +118,8 @@ static ModifyTable *create_modifytable_plan(PlannerInfo *root, ModifyTablePath *
static Limit *create_limit_plan(PlannerInfo *root, LimitPath *best_path,
int flags);
static SeqScan *create_seqscan_plan(PlannerInfo *root, Path *best_path,
- List *tlist, List *scan_clauses);
+ List *tlist, List *scan_clauses,
+ Bitmapset *tlist_varattnos);
static SampleScan *create_samplescan_plan(PlannerInfo *root, Path *best_path,
List *tlist, List *scan_clauses);
static Scan *create_indexscan_plan(PlannerInfo *root, IndexPath *best_path,
@@ -178,7 +179,8 @@ static void label_sort_with_costsize(PlannerInfo *root, Sort *plan,
double limit_tuples);
static void label_incrementalsort_with_costsize(PlannerInfo *root, IncrementalSort *plan,
List *pathkeys, double limit_tuples);
-static SeqScan *make_seqscan(List *qptlist, List *qpqual, Index scanrelid);
+static SeqScan *make_seqscan(List *qptlist, List *qpqual, Index scanrelid,
+ Bitmapset *scan_varattnos);
static SampleScan *make_samplescan(List *qptlist, List *qpqual, Index scanrelid,
TableSampleClause *tsc);
static IndexScan *make_indexscan(List *qptlist, List *qpqual, Index scanrelid,
@@ -550,6 +552,7 @@ create_plan_recurse(PlannerInfo *root, Path *best_path, int flags)
static Plan *
create_scan_plan(PlannerInfo *root, Path *best_path, int flags)
{
+ Bitmapset *tlist_varattnos = NULL;
RelOptInfo *rel = best_path->parent;
List *scan_clauses;
List *gating_clauses;
@@ -579,6 +582,14 @@ create_scan_plan(PlannerInfo *root, Path *best_path, int flags)
break;
}
+ /*
+ * Figure out which attributes we need from the scan before applying the
+ * physical tlist optimization.
+ */
+ pull_varattnos((Node *) best_path->pathtarget->exprs,
+ rel->relid,
+ &tlist_varattnos);
+
/*
* If this is a parameterized scan, we also need to enforce all the join
* clauses available from the outer relation(s).
@@ -672,7 +683,8 @@ create_scan_plan(PlannerInfo *root, Path *best_path, int flags)
plan = (Plan *) create_seqscan_plan(root,
best_path,
tlist,
- scan_clauses);
+ scan_clauses,
+ tlist_varattnos);
break;
case T_SampleScan:
@@ -2752,10 +2764,13 @@ create_limit_plan(PlannerInfo *root, LimitPath *best_path, int flags)
*/
static SeqScan *
create_seqscan_plan(PlannerInfo *root, Path *best_path,
- List *tlist, List *scan_clauses)
+ List *tlist, List *scan_clauses, Bitmapset *tlist_varattnos)
{
SeqScan *scan_plan;
Index scan_relid = best_path->parent->relid;
+ Bitmapset *scan_varattnos = tlist_varattnos;
+ Bitmapset *non_sys_attrs = NULL;
+ int i;
/* it should be a base rel... */
Assert(scan_relid > 0);
@@ -2767,6 +2782,19 @@ create_seqscan_plan(PlannerInfo *root, Path *best_path,
/* Reduce RestrictInfo list to bare expressions; ignore pseudoconstants */
scan_clauses = extract_actual_clauses(scan_clauses, false);
+ /* Pull varattnos from WHERE clause Vars */
+ pull_varattnos((Node *) scan_clauses, scan_relid, &scan_varattnos);
+
+ /* Don't set these when whole-row var is present */
+ if (!bms_is_member(0 - FirstLowInvalidHeapAttributeNumber, scan_varattnos))
+ {
+ /* XXX invent bms_right_shift_members()? */
+ i = 0 - FirstLowInvalidHeapAttributeNumber;
+ while ((i = bms_next_member(scan_varattnos, i)) >= 0)
+ non_sys_attrs = bms_add_member(non_sys_attrs,
+ i - 1 + FirstLowInvalidHeapAttributeNumber);
+ }
+
/* Replace any outer-relation variables with nestloop params */
if (best_path->param_info)
{
@@ -2776,7 +2804,8 @@ create_seqscan_plan(PlannerInfo *root, Path *best_path,
scan_plan = make_seqscan(tlist,
scan_clauses,
- scan_relid);
+ scan_relid,
+ non_sys_attrs);
copy_generic_path_info(&scan_plan->scan.plan, best_path);
@@ -5487,7 +5516,8 @@ bitmap_subplan_mark_shared(Plan *plan)
static SeqScan *
make_seqscan(List *qptlist,
List *qpqual,
- Index scanrelid)
+ Index scanrelid,
+ Bitmapset *scan_varattnos)
{
SeqScan *node = makeNode(SeqScan);
Plan *plan = &node->scan.plan;
@@ -5497,6 +5527,7 @@ make_seqscan(List *qptlist,
plan->lefttree = NULL;
plan->righttree = NULL;
node->scan.scanrelid = scanrelid;
+ node->scan.scan_varattnos = scan_varattnos;
return node;
}
diff --git a/src/include/executor/execExpr.h b/src/include/executor/execExpr.h
index aa9b361fa31..f29d9dd799b 100644
--- a/src/include/executor/execExpr.h
+++ b/src/include/executor/execExpr.h
@@ -78,6 +78,9 @@ typedef enum ExprEvalOp
EEOP_OLD_FETCHSOME,
EEOP_NEW_FETCHSOME,
+ /* apply slot_selectattrs on the corresponding tuple slot */
+ EEOP_SCAN_SELECTSOME,
+
/* compute non-system Var value */
EEOP_INNER_VAR,
EEOP_OUTER_VAR,
@@ -318,15 +321,34 @@ typedef struct ExprEvalStep
*/
union
{
- /* for EEOP_INNER/OUTER/SCAN/OLD/NEW_FETCHSOME */
+ /*
+ * for EEOP_INNER/OUTER/SCAN/OLD/NEW_FETCHSOME and
+ * EEOP_SCAN_SELECTSOME
+ */
struct
{
/* attribute number up to which to fetch (inclusive) */
int last_var;
/* will the type of slot be the same for every invocation */
bool fixed;
+ /* Number of elements in req_attnums array. XXX needed? */
+ AttrNumber natts;
+
+ /* One element for each attnum to select, ordered by attnum */
+ AttrNumber *req_attnums;
+
+ /*
+ * Provides mapping of 0-based attnums back to the index of the
+ * req_attnums array that deforming should continue from. This
+ * allows us to re-find the element of req_attnums using the
+ * slot's tts_nvalid so that we can continue deforming from the
+ * last defromed attribute.
+ */
+ AttrNumber *next_req_attnums_index;
+
/* tuple descriptor, if known */
TupleDesc known_desc;
+
/* type of slot, can only be relied upon if fixed is set */
const TupleTableSlotOps *kind;
} fetch;
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index b1820653506..ea86c3822ee 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -325,8 +325,12 @@ ExecProcNode(PlanState *node)
*/
extern ExprState *ExecInitExpr(Expr *node, PlanState *parent);
extern ExprState *ExecInitExprWithContext(Expr *node, PlanState *parent, Node *escontext);
+extern ExprState *ExecInitExprWithScanAttrs(Expr *node, PlanState *parent,
+ Bitmapset *scan_attrs);
extern ExprState *ExecInitExprWithParams(Expr *node, ParamListInfo ext_params);
extern ExprState *ExecInitQual(List *qual, PlanState *parent);
+extern ExprState *ExecInitQualWithScanAttrs(List *qual, PlanState *parent,
+ Bitmapset *scan_attrs);
extern ExprState *ExecInitCheck(List *qual, PlanState *parent);
extern List *ExecInitExprList(List *nodes, PlanState *parent);
extern ExprState *ExecBuildAggTrans(AggState *aggstate, struct AggStatePerPhaseData *phase,
@@ -365,6 +369,12 @@ extern ProjectionInfo *ExecBuildProjectionInfo(List *targetList,
TupleTableSlot *slot,
PlanState *parent,
TupleDesc inputDesc);
+extern ProjectionInfo *ExecBuildProjectionInfoWithScanAttrs(List *targetList,
+ ExprContext *econtext,
+ TupleTableSlot *slot,
+ PlanState *parent,
+ TupleDesc inputDesc,
+ Bitmapset *scan_attrs);
extern ProjectionInfo *ExecBuildUpdateProjection(List *targetList,
bool evalTargetList,
List *targetColnos,
@@ -584,6 +594,8 @@ typedef bool (*ExecScanRecheckMtd) (ScanState *node, TupleTableSlot *slot);
extern TupleTableSlot *ExecScan(ScanState *node, ExecScanAccessMtd accessMtd,
ExecScanRecheckMtd recheckMtd);
extern void ExecAssignScanProjectionInfo(ScanState *node);
+extern void ExecAssignScanProjectionInfoWithScanAttrs(ScanState *node,
+ Bitmapset *scan_attrs);
extern void ExecAssignScanProjectionInfoWithVarno(ScanState *node, int varno);
extern void ExecScanReScan(ScanState *node);
@@ -680,8 +692,15 @@ extern const TupleTableSlotOps *ExecGetCommonSlotOps(PlanState **planstates,
extern const TupleTableSlotOps *ExecGetCommonChildSlotOps(PlanState *ps);
extern void ExecAssignProjectionInfo(PlanState *planstate,
TupleDesc inputDesc);
+extern void ExecAssignProjectionInfoWithScanAttrs(PlanState *planstate,
+ TupleDesc inputDesc,
+ Bitmapset *scan_attrs);
extern void ExecConditionalAssignProjectionInfo(PlanState *planstate,
TupleDesc inputDesc, int varno);
+extern void ExecConditionalAssignProjectionInfoWithScanAttrs(PlanState *planstate,
+ TupleDesc inputDesc,
+ int varno,
+ Bitmapset *scan_attrs);
extern void ExecAssignScanType(ScanState *scanstate, TupleDesc tupDesc);
extern void ExecCreateScanSlotFromOuterPlan(EState *estate,
ScanState *scanstate,
diff --git a/src/include/executor/tuptable.h b/src/include/executor/tuptable.h
index 78558098fa3..cc2c5a257d0 100644
--- a/src/include/executor/tuptable.h
+++ b/src/include/executor/tuptable.h
@@ -168,6 +168,16 @@ struct TupleTableSlotOps
*/
void (*getsomeattrs) (TupleTableSlot *slot, int natts);
+ /*
+ * Populate the tts_values and tts_isnull elements of the given slot with
+ * the values of the corresponding attribute from the tuple stored in the
+ * slot. Populate up as far as last_attnum and store each attribute
+ * mentioned in the attnums array. Use attnum_map to determine the
+ * starting element in the attnums array from the slot's tts_nvalid.
+ */
+ void (*selectattrs) (TupleTableSlot *slot, int last_attnum,
+ AttrNumber *attnums, AttrNumber *attnum_map);
+
/*
* Returns value of the given system attribute as a datum and sets isnull
* to false, if it's not NULL. Throws an error if the slot type does not
@@ -374,6 +384,18 @@ slot_getsomeattrs(TupleTableSlot *slot, int attnum)
slot->tts_ops->getsomeattrs(slot, attnum);
}
+static inline void
+slot_selectattrs(TupleTableSlot *slot, int last_attnum, AttrNumber *attnums,
+ AttrNumber *attnum_map)
+{
+ /*
+ * Populate slot only attributes mentioned in the attnums array, up to
+ * 'last_attnum', if it's not already
+ */
+ if (slot->tts_nvalid < last_attnum)
+ slot->tts_ops->selectattrs(slot, last_attnum, attnums, attnum_map);
+}
+
/*
* slot_getallattrs
* This function forces all the entries of the slot's Datum/isnull
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 8c9321aab8c..08dcf02b8bb 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -540,6 +540,14 @@ typedef struct Scan
Plan plan;
/* relid is index into the range table */
Index scanrelid;
+
+ /*
+ * All varattnos that are required from the scanrelid. Does not include
+ * any added due to the physical tlist optimization or system attributes
+ * or whole-row attributes. User attributes are 0 based, i.e attnum==1 is
+ * member 0.
+ */
+ Bitmapset *scan_varattnos;
} Scan;
/* ----------------
diff --git a/src/test/modules/deform_bench/deform_bench--1.0.sql b/src/test/modules/deform_bench/deform_bench--1.0.sql
index 492b71dba3b..4a419fde35c 100644
--- a/src/test/modules/deform_bench/deform_bench--1.0.sql
+++ b/src/test/modules/deform_bench/deform_bench--1.0.sql
@@ -6,3 +6,7 @@
CREATE FUNCTION deform_bench(tableoid Oid, attnum int[]) RETURNS FLOAT
AS 'MODULE_PATHNAME', 'deform_bench'
LANGUAGE C VOLATILE STRICT;
+
+CREATE FUNCTION deform_bench_select(tableoid Oid, attnum int[]) RETURNS FLOAT
+AS 'MODULE_PATHNAME', 'deform_bench_select'
+LANGUAGE C VOLATILE STRICT;
diff --git a/src/test/modules/deform_bench/deform_bench.c b/src/test/modules/deform_bench/deform_bench.c
index 4f104989297..f929d952596 100644
--- a/src/test/modules/deform_bench/deform_bench.c
+++ b/src/test/modules/deform_bench/deform_bench.c
@@ -16,6 +16,7 @@
#include "catalog/pg_type_d.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "nodes/bitmapset.h"
#include "utils/array.h"
#include "utils/arrayaccess.h"
#include "utils/builtins.h"
@@ -23,6 +24,7 @@
PG_MODULE_MAGIC;
PG_FUNCTION_INFO_V1(deform_bench);
+PG_FUNCTION_INFO_V1(deform_bench_select);
Datum
deform_bench(PG_FUNCTION_ARGS)
@@ -104,6 +106,102 @@ deform_bench(PG_FUNCTION_ARGS)
relation_close(rel, AccessShareLock);
+ /* Returns the number of milliseconds to run the test */
+ PG_RETURN_FLOAT8((double) (end - start) / (CLOCKS_PER_SEC / 1000));
+}
+
+Datum
+deform_bench_select(PG_FUNCTION_ARGS)
+{
+ Oid tableoid = PG_GETARG_OID(0);
+ ArrayType *array = PG_GETARG_ARRAYTYPE_P(1);
+ TableScanDesc scan;
+ Relation rel;
+ TupleDesc tupdesc;
+ TupleTableSlot *slot;
+ Datum *elem_datums = NULL;
+ bool *elem_nulls = NULL;
+ int elem_count;
+ int i;
+ int attnum;
+ int last_attnum;
+ AttrNumber *attnums;
+ Bitmapset *attrs = NULL;
+ clock_t start,
+ end;
+
+ rel = relation_open(tableoid, AccessShareLock);
+
+ if (rel->rd_rel->relam != HEAP_TABLE_AM_OID)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("only heap AM is supported")));
+
+ tupdesc = RelationGetDescr(rel);
+ slot = MakeTupleTableSlot(tupdesc,
+ &TTSOpsBufferHeapTuple,
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS);
+ scan = table_beginscan_strat(rel, GetActiveSnapshot(), 0, NULL, true, false);
+
+ /*
+ * The array is used to allow callers to specify which attributes to
+ * deform. E.g: '{1,10}'::int[] would deform only attnum=1 and attnum=10.
+ *
+ * You'll get an ERROR if you pass an attnum that does not exist. NULL
+ * elements are ignored.
+ */
+ deconstruct_array(array,
+ INT4OID,
+ sizeof(int32),
+ true,
+ 'i',
+ &elem_datums,
+ &elem_nulls,
+ &elem_count);
+
+ for (i = 0; i < elem_count; i++)
+ {
+ /* Make a NULL element mean all attributes */
+ if (elem_nulls[i])
+ continue;
+
+ attnum = DatumGetInt32(elem_datums[i]);
+
+ if (attnum <= 0)
+ elog(ERROR, "only user attributes can be deformed by deform_bench_select");
+ if (attnum >= 0xffff)
+ elog(ERROR, "invalid attnum %d", attnum);
+
+ attrs = bms_add_member(attrs, attnum - 1);
+ }
+
+ attnums = palloc_array(AttrNumber, bms_num_members(attrs));
+
+ attnum = -1;
+ i = 0;
+ while ((attnum = bms_next_member(attrs, attnum)) >= 0)
+ attnums[i++] = (AttrNumber) attnum;
+
+ last_attnum = attnums[i - 1];
+
+ start = clock();
+
+ while (heap_getnextslot(scan, ForwardScanDirection, slot))
+ {
+ AttrNumber attnum_map = 0;
+ CHECK_FOR_INTERRUPTS();
+
+ /* Pass in a faked up attnum_map. tts_nvalid will always be 0 */
+ slot_selectattrs(slot, last_attnum, attnums, &attnum_map);
+ }
+
+ end = clock();
+
+ ExecDropSingleTupleTableSlot(slot);
+ table_endscan(scan);
+ relation_close(rel, AccessShareLock);
+
+
/* Returns the number of milliseconds to run the test */
PG_RETURN_FLOAT8((double) (end - start) / (CLOCKS_PER_SEC / 1000));
}
--
2.51.0
Attachments:
[text/plain] v13-0001-Introduce-deform_bench-test-module.patch (7.3K, 2-v13-0001-Introduce-deform_bench-test-module.patch)
download | inline diff:
From cc6d328cf1eb3e25df88afe5733d35817ac4f3e0 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Tue, 27 Jan 2026 15:08:09 +1300
Subject: [PATCH v13 1/6] Introduce deform_bench test module
For benchmarking tuple deformation.
---
src/test/modules/deform_bench/.gitignore | 4 +
src/test/modules/deform_bench/Makefile | 21 ++++
.../deform_bench/deform_bench--1.0.sql | 8 ++
src/test/modules/deform_bench/deform_bench.c | 107 ++++++++++++++++++
.../modules/deform_bench/deform_bench.control | 4 +
src/test/modules/deform_bench/meson.build | 22 ++++
src/test/modules/meson.build | 1 +
7 files changed, 167 insertions(+)
create mode 100644 src/test/modules/deform_bench/.gitignore
create mode 100644 src/test/modules/deform_bench/Makefile
create mode 100644 src/test/modules/deform_bench/deform_bench--1.0.sql
create mode 100644 src/test/modules/deform_bench/deform_bench.c
create mode 100644 src/test/modules/deform_bench/deform_bench.control
create mode 100644 src/test/modules/deform_bench/meson.build
diff --git a/src/test/modules/deform_bench/.gitignore b/src/test/modules/deform_bench/.gitignore
new file mode 100644
index 00000000000..5dcb3ff9723
--- /dev/null
+++ b/src/test/modules/deform_bench/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/src/test/modules/deform_bench/Makefile b/src/test/modules/deform_bench/Makefile
new file mode 100644
index 00000000000..b5fc0f7a583
--- /dev/null
+++ b/src/test/modules/deform_bench/Makefile
@@ -0,0 +1,21 @@
+# src/test/modules/deform_bench/Makefile
+
+MODULE_big = deform_bench
+OBJS = deform_bench.o
+
+EXTENSION = deform_bench
+DATA = deform_bench--1.0.sql
+PGFILEDESC = "deform_bench - tuple deform benchmarking"
+
+REGRESS = deform_bench
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = src/test/modules/deform_bench
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/src/test/modules/deform_bench/deform_bench--1.0.sql b/src/test/modules/deform_bench/deform_bench--1.0.sql
new file mode 100644
index 00000000000..492b71dba3b
--- /dev/null
+++ b/src/test/modules/deform_bench/deform_bench--1.0.sql
@@ -0,0 +1,8 @@
+/* deform_bench--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION deform_bench" to load this file. \quit
+
+CREATE FUNCTION deform_bench(tableoid Oid, attnum int[]) RETURNS FLOAT
+AS 'MODULE_PATHNAME', 'deform_bench'
+LANGUAGE C VOLATILE STRICT;
diff --git a/src/test/modules/deform_bench/deform_bench.c b/src/test/modules/deform_bench/deform_bench.c
new file mode 100644
index 00000000000..7838f639bef
--- /dev/null
+++ b/src/test/modules/deform_bench/deform_bench.c
@@ -0,0 +1,107 @@
+/*-------------------------------------------------------------------------
+ *
+ * deform_bench.c
+ *
+ * for benchmarking tuple deformation routines
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include <time.h>
+#include <sys/time.h>
+
+#include "access/heapam.h"
+#include "access/relscan.h"
+#include "catalog/pg_am_d.h"
+#include "catalog/pg_type_d.h"
+#include "funcapi.h"
+#include "miscadmin.h"
+#include "utils/array.h"
+#include "utils/arrayaccess.h"
+#include "utils/builtins.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(deform_bench);
+
+Datum
+deform_bench(PG_FUNCTION_ARGS)
+{
+ Oid tableoid = PG_GETARG_OID(0);
+ ArrayType *array = PG_GETARG_ARRAYTYPE_P(1);
+ TableScanDesc scan;
+ Relation rel;
+ TupleDesc tupdesc;
+ TupleTableSlot *slot;
+ Datum *elem_datums = NULL;
+ bool *elem_nulls = NULL;
+ int elem_count;
+ int *attnums;
+ clock_t start,
+ end;
+
+ rel = relation_open(tableoid, AccessShareLock);
+
+ if (rel->rd_rel->relam != HEAP_TABLE_AM_OID)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("only heap AM is supported")));
+
+ tupdesc = RelationGetDescr(rel);
+ slot = MakeTupleTableSlot(tupdesc, &TTSOpsBufferHeapTuple);
+ scan = table_beginscan_strat(rel, GetActiveSnapshot(), 0, NULL, true, false);
+
+ /*
+ * The array is used to allow callers to define how many atts to deform.
+ * e.g: '{1,10}'::int[] would deform attnum=1, then in a 2nd pass deform
+ * the remainder up to attnum=10. Passing an element as NULL means all
+ * attnums. This allows simulation of incremental deformation. Generally
+ * if you're passing an array with more than 1 element, then the array
+ * should be in ascending order. Doing something like '{10,1}' would mean
+ * we've already deformed 10 attributes and on the 2nd pass there's
+ * nothing to do since attnum=1 was already deformed in the first pass.
+ *
+ * You'll get an ERROR if you pass a number higher than the number of
+ * attributes in the table.
+ */
+ deconstruct_array(array,
+ INT4OID,
+ sizeof(int32),
+ true,
+ 'i',
+ &elem_datums,
+ &elem_nulls,
+ &elem_count);
+
+ attnums = palloc_array(int, elem_count);
+
+ for (int i = 0; i < elem_count; i++)
+ {
+ /* Make a NULL element mean all attributes */
+ if (elem_nulls[i])
+ attnums[i] = tupdesc->natts;
+ else
+ attnums[i] = DatumGetInt32(elem_datums[i]);
+ }
+
+ start = clock();
+
+ while (heap_getnextslot(scan, ForwardScanDirection, slot))
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ /* Deform in stages according to the attnums array */
+ for (int i = 0; i < elem_count; i++)
+ slot_getsomeattrs(slot, attnums[i]);
+ }
+
+ end = clock();
+
+ ExecDropSingleTupleTableSlot(slot);
+ table_endscan(scan);
+ relation_close(rel, AccessShareLock);
+
+
+ /* Returns the number of milliseconds to run the test */
+ PG_RETURN_FLOAT8((double) (end - start) / (CLOCKS_PER_SEC / 1000));
+}
diff --git a/src/test/modules/deform_bench/deform_bench.control b/src/test/modules/deform_bench/deform_bench.control
new file mode 100644
index 00000000000..a2023f9d738
--- /dev/null
+++ b/src/test/modules/deform_bench/deform_bench.control
@@ -0,0 +1,4 @@
+# deform_bench extension
+comment = 'functions for benchmarking tuple deformation'
+default_version = '1.0'
+module_pathname = '$libdir/deform_bench'
diff --git a/src/test/modules/deform_bench/meson.build b/src/test/modules/deform_bench/meson.build
new file mode 100644
index 00000000000..82049585244
--- /dev/null
+++ b/src/test/modules/deform_bench/meson.build
@@ -0,0 +1,22 @@
+# Copyright (c) 2026, PostgreSQL Global Development Group
+
+deform_bench_sources = files(
+ 'deform_bench.c',
+)
+
+if host_system == 'windows'
+ deform_bench_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+ '--NAME', 'deform_bench',
+ '--FILEDESC', 'deform_bench - benchmarking tuple deformation',])
+endif
+
+deform_bench = shared_module('deform_bench',
+ deform_bench_sources,
+ kwargs: pg_test_mod_args,
+)
+test_install_libs += deform_bench
+
+test_install_data += files(
+ 'deform_bench--1.0.sql',
+ 'deform_bench.control',
+)
diff --git a/src/test/modules/meson.build b/src/test/modules/meson.build
index e2b3eef4136..1312f3b4d7b 100644
--- a/src/test/modules/meson.build
+++ b/src/test/modules/meson.build
@@ -2,6 +2,7 @@
subdir('brin')
subdir('commit_ts')
+subdir('deform_bench')
subdir('delay_execution')
subdir('dummy_index_am')
subdir('dummy_seclabel')
--
2.51.0
[text/plain] v13-0002-Allow-sibling-call-optimization-in-slot_getsomea.patch (7.3K, 3-v13-0002-Allow-sibling-call-optimization-in-slot_getsomea.patch)
download | inline diff:
From 1fef8685b5ea019086f68b41fa2bfc995da4353c Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Mon, 16 Feb 2026 14:20:19 +1300
Subject: [PATCH v13 2/6] Allow sibling call optimization in
slot_getsomeattrs_int()
This changes the TupleTableSlotOps contract to make it so the
getsomeattrs() function is in charge of calling
slot_getmissingattrs().
Since this removes all code from slot_getsomeattrs_int() aside from the
getsomeattrs() call itself, we may as well adjust slot_getsomeattrs() so
that it calls getsomeattrs() directly. We leave slot_getsomeattrs_int()
intact as this is still called from the JIT code.
---
src/backend/executor/execTuples.c | 58 ++++++++++++++++---------------
src/include/executor/tuptable.h | 13 ++++---
2 files changed, 38 insertions(+), 33 deletions(-)
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index b768eae9e53..7effe954286 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -73,7 +73,7 @@
static TupleDesc ExecTypeFromTLInternal(List *targetList,
bool skipjunk);
static pg_attribute_always_inline void slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
- int natts);
+ int reqnatts);
static inline void tts_buffer_heap_store_tuple(TupleTableSlot *slot,
HeapTuple tuple,
Buffer buffer,
@@ -1108,7 +1108,10 @@ slot_deform_heap_tuple_internal(TupleTableSlot *slot, HeapTuple tuple,
* slot_deform_heap_tuple
* Given a TupleTableSlot, extract data from the slot's physical tuple
* into its Datum/isnull arrays. Data is extracted up through the
- * natts'th column (caller must ensure this is a legal column number).
+ * reqnatts'th column. If there are insufficient attributes in the given
+ * tuple, then slot_getmissingattrs() is called to populate the
+ * remainder. If reqnatts is above the number of attributes in the
+ * slot's TupleDesc, an error is raised.
*
* This is essentially an incremental version of heap_deform_tuple:
* on each call we extract attributes up to the one needed, without
@@ -1120,21 +1123,23 @@ slot_deform_heap_tuple_internal(TupleTableSlot *slot, HeapTuple tuple,
*/
static pg_attribute_always_inline void
slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
- int natts)
+ int reqnatts)
{
bool hasnulls = HeapTupleHasNulls(tuple);
int attnum;
+ int natts;
uint32 off; /* offset in tuple data */
bool slow; /* can we use/set attcacheoff? */
/* We can only fetch as many attributes as the tuple has. */
- natts = Min(HeapTupleHeaderGetNatts(tuple->t_data), natts);
+ natts = Min(HeapTupleHeaderGetNatts(tuple->t_data), reqnatts);
/*
* Check whether the first call for this tuple, and initialize or restore
* loop state.
*/
attnum = slot->tts_nvalid;
+ slot->tts_nvalid = reqnatts;
if (attnum == 0)
{
/* Start from the first attribute */
@@ -1199,12 +1204,15 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
/*
* Save state for next execution
*/
- slot->tts_nvalid = attnum;
*offp = off;
if (slow)
slot->tts_flags |= TTS_FLAG_SLOW;
else
slot->tts_flags &= ~TTS_FLAG_SLOW;
+
+ /* Fetch any missing attrs and raise an error if reqnatts is invalid. */
+ if (unlikely(attnum < reqnatts))
+ slot_getmissingattrs(slot, attnum, reqnatts);
}
const TupleTableSlotOps TTSOpsVirtual = {
@@ -2058,34 +2066,36 @@ slot_getmissingattrs(TupleTableSlot *slot, int startAttNum, int lastAttNum)
{
AttrMissing *attrmiss = NULL;
+ /* Check for invalid attnums */
+ if (unlikely(lastAttNum > slot->tts_tupleDescriptor->natts))
+ elog(ERROR, "invalid attribute number %d", lastAttNum);
+
if (slot->tts_tupleDescriptor->constr)
attrmiss = slot->tts_tupleDescriptor->constr->missing;
if (!attrmiss)
{
/* no missing values array at all, so just fill everything in as NULL */
- memset(slot->tts_values + startAttNum, 0,
- (lastAttNum - startAttNum) * sizeof(Datum));
- memset(slot->tts_isnull + startAttNum, 1,
- (lastAttNum - startAttNum) * sizeof(bool));
+ for (int attnum = startAttNum; attnum < lastAttNum; attnum++)
+ {
+ slot->tts_values[attnum] = (Datum) 0;
+ slot->tts_isnull[attnum] = true;
+ }
}
else
{
- int missattnum;
-
- /* if there is a missing values array we must process them one by one */
- for (missattnum = startAttNum;
- missattnum < lastAttNum;
- missattnum++)
+ /* use attrmiss to set the missing values */
+ for (int attnum = startAttNum; attnum < lastAttNum; attnum++)
{
- slot->tts_values[missattnum] = attrmiss[missattnum].am_value;
- slot->tts_isnull[missattnum] = !attrmiss[missattnum].am_present;
+ slot->tts_values[attnum] = attrmiss[attnum].am_value;
+ slot->tts_isnull[attnum] = !attrmiss[attnum].am_present;
}
}
}
/*
- * slot_getsomeattrs_int - workhorse for slot_getsomeattrs()
+ * slot_getsomeattrs_int
+ * external function to call getsomeattrs() for use in JIT
*/
void
slot_getsomeattrs_int(TupleTableSlot *slot, int attnum)
@@ -2094,21 +2104,13 @@ slot_getsomeattrs_int(TupleTableSlot *slot, int attnum)
Assert(slot->tts_nvalid < attnum); /* checked in slot_getsomeattrs */
Assert(attnum > 0);
- if (unlikely(attnum > slot->tts_tupleDescriptor->natts))
- elog(ERROR, "invalid attribute number %d", attnum);
-
/* Fetch as many attributes as possible from the underlying tuple. */
slot->tts_ops->getsomeattrs(slot, attnum);
/*
- * If the underlying tuple doesn't have enough attributes, tuple
- * descriptor must have the missing attributes.
+ * Avoid putting new code here as that would prevent the compiler from
+ * using the sibling call optimization for the above function.
*/
- if (unlikely(slot->tts_nvalid < attnum))
- {
- slot_getmissingattrs(slot, slot->tts_nvalid, attnum);
- slot->tts_nvalid = attnum;
- }
}
/* ----------------------------------------------------------------
diff --git a/src/include/executor/tuptable.h b/src/include/executor/tuptable.h
index a2dfd707e78..3b09abbf99f 100644
--- a/src/include/executor/tuptable.h
+++ b/src/include/executor/tuptable.h
@@ -151,10 +151,12 @@ struct TupleTableSlotOps
/*
* Fill up first natts entries of tts_values and tts_isnull arrays with
- * values from the tuple contained in the slot. The function may be called
- * with natts more than the number of attributes available in the tuple,
- * in which case it should set tts_nvalid to the number of returned
- * columns.
+ * values from the tuple contained in the slot and set the slot's
+ * tts_nvalid to natts. The function may be called with an natts value
+ * more than the number of attributes available in the tuple, in which
+ * case the function must call slot_getmissingattrs() to populate the
+ * remaining attributes. The function must raise an ERROR if 'natts' is
+ * higher than the number of attributes in the slot's TupleDesc.
*/
void (*getsomeattrs) (TupleTableSlot *slot, int natts);
@@ -357,8 +359,9 @@ extern void slot_getsomeattrs_int(TupleTableSlot *slot, int attnum);
static inline void
slot_getsomeattrs(TupleTableSlot *slot, int attnum)
{
+ /* Populate slot with attributes up to 'attnum', if it's not already */
if (slot->tts_nvalid < attnum)
- slot_getsomeattrs_int(slot, attnum);
+ slot->tts_ops->getsomeattrs(slot, attnum);
}
/*
--
2.51.0
[text/plain] v13-0003-Add-empty-TupleDescFinalize-function.patch (29.0K, 4-v13-0003-Add-empty-TupleDescFinalize-function.patch)
download | inline diff:
From a0d67378e4b1d30b4afead1a37d7cf93490dc656 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Wed, 21 Jan 2026 15:41:37 +1300
Subject: [PATCH v13 3/6] Add empty TupleDescFinalize() function
Currently does nothing, but will in a future commit.
---
contrib/dblink/dblink.c | 4 ++++
contrib/pg_buffercache/pg_buffercache_pages.c | 2 ++
contrib/pg_visibility/pg_visibility.c | 2 ++
src/backend/access/brin/brin_tuple.c | 1 +
src/backend/access/common/tupdesc.c | 13 +++++++++++++
src/backend/access/gin/ginutil.c | 1 +
src/backend/access/gist/gistscan.c | 1 +
src/backend/access/spgist/spgutils.c | 1 +
src/backend/access/transam/twophase.c | 1 +
src/backend/access/transam/xlogfuncs.c | 1 +
src/backend/backup/basebackup_copy.c | 3 +++
src/backend/catalog/index.c | 2 ++
src/backend/catalog/pg_publication.c | 1 +
src/backend/catalog/toasting.c | 6 ++++++
src/backend/commands/explain.c | 1 +
src/backend/commands/functioncmds.c | 1 +
src/backend/commands/sequence.c | 1 +
src/backend/commands/tablecmds.c | 4 ++++
src/backend/commands/wait.c | 1 +
src/backend/executor/execSRF.c | 2 ++
src/backend/executor/execTuples.c | 4 ++++
src/backend/executor/nodeFunctionscan.c | 2 ++
src/backend/parser/parse_relation.c | 4 +++-
src/backend/parser/parse_target.c | 2 ++
.../replication/libpqwalreceiver/libpqwalreceiver.c | 1 +
src/backend/replication/walsender.c | 5 +++++
src/backend/utils/adt/acl.c | 1 +
src/backend/utils/adt/genfile.c | 1 +
src/backend/utils/adt/lockfuncs.c | 1 +
src/backend/utils/adt/orderedsetaggs.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 5 +++++
src/backend/utils/adt/tsvector_op.c | 1 +
src/backend/utils/cache/relcache.c | 8 ++++++++
src/backend/utils/fmgr/funcapi.c | 6 ++++++
src/backend/utils/misc/guc_funcs.c | 5 +++++
src/include/access/tupdesc.h | 1 +
src/pl/plpgsql/src/pl_comp.c | 2 ++
.../test_custom_stats/test_custom_fixed_stats.c | 1 +
src/test/modules/test_predtest/test_predtest.c | 1 +
39 files changed, 100 insertions(+), 1 deletion(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 2498d80c8e7..4038950a6ef 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -881,6 +881,7 @@ materializeResult(FunctionCallInfo fcinfo, PGconn *conn, PGresult *res)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "status",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
ntuples = 1;
nfields = 1;
}
@@ -1044,6 +1045,7 @@ materializeQueryResult(FunctionCallInfo fcinfo,
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "status",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
attinmeta = TupleDescGetAttInMetadata(tupdesc);
oldcontext = MemoryContextSwitchTo(rsinfo->econtext->ecxt_per_query_memory);
@@ -1529,6 +1531,8 @@ dblink_get_pkey(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 2, "colname",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+
/*
* Generate attribute metadata needed later to produce tuples from raw
* C strings
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 89b86855243..a6b4fb5252b 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -174,6 +174,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupledesc, (AttrNumber) 9, "pinning_backends",
INT4OID, -1, 0);
+ TupleDescFinalize(tupledesc);
fctx->tupdesc = BlessTupleDesc(tupledesc);
/* Allocate NBuffers worth of BufferCachePagesRec records. */
@@ -442,6 +443,7 @@ pg_buffercache_os_pages_internal(FunctionCallInfo fcinfo, bool include_numa)
TupleDescInitEntry(tupledesc, (AttrNumber) 3, "numa_node",
INT4OID, -1, 0);
+ TupleDescFinalize(tupledesc);
fctx->tupdesc = BlessTupleDesc(tupledesc);
fctx->include_numa = include_numa;
diff --git a/contrib/pg_visibility/pg_visibility.c b/contrib/pg_visibility/pg_visibility.c
index 9bc3a784bf7..dfab0b64cf5 100644
--- a/contrib/pg_visibility/pg_visibility.c
+++ b/contrib/pg_visibility/pg_visibility.c
@@ -469,6 +469,8 @@ pg_visibility_tupdesc(bool include_blkno, bool include_pd)
TupleDescInitEntry(tupdesc, ++a, "pd_all_visible", BOOLOID, -1, 0);
Assert(a == maxattr);
+ TupleDescFinalize(tupdesc);
+
return BlessTupleDesc(tupdesc);
}
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 69c233c62eb..742ac089a28 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -84,6 +84,7 @@ brtuple_disk_tupdesc(BrinDesc *brdesc)
MemoryContextSwitchTo(oldcxt);
+ TupleDescFinalize(tupdesc);
brdesc->bd_disktdesc = tupdesc;
}
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index b69d10f0a45..2137385a833 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -221,6 +221,9 @@ CreateTupleDesc(int natts, Form_pg_attribute *attrs)
memcpy(TupleDescAttr(desc, i), attrs[i], ATTRIBUTE_FIXED_PART_SIZE);
populate_compact_attribute(desc, i);
}
+
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -265,6 +268,8 @@ CreateTupleDescCopy(TupleDesc tupdesc)
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -311,6 +316,8 @@ CreateTupleDescTruncatedCopy(TupleDesc tupdesc, int natts)
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -396,6 +403,8 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -438,6 +447,8 @@ TupleDescCopy(TupleDesc dst, TupleDesc src)
* source's refcount would be wrong in any case.)
*/
dst->tdrefcount = -1;
+
+ TupleDescFinalize(dst);
}
/*
@@ -1065,6 +1076,8 @@ BuildDescFromLists(const List *names, const List *types, const List *typmods, co
TupleDescInitEntryCollation(desc, attnum, attcollation);
}
+ TupleDescFinalize(desc);
+
return desc;
}
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index ff927279cc3..fe7b984ff32 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -129,6 +129,7 @@ initGinState(GinState *state, Relation index)
attr->attndims);
TupleDescInitEntryCollation(state->tupdesc[i], (AttrNumber) 2,
attr->attcollation);
+ TupleDescFinalize(state->tupdesc[i]);
}
/*
diff --git a/src/backend/access/gist/gistscan.c b/src/backend/access/gist/gistscan.c
index f23bc4a6757..c65f93abdae 100644
--- a/src/backend/access/gist/gistscan.c
+++ b/src/backend/access/gist/gistscan.c
@@ -201,6 +201,7 @@ gistrescan(IndexScanDesc scan, ScanKey key, int nkeys,
attno - 1)->atttypid,
-1, 0);
}
+ TupleDescFinalize(so->giststate->fetchTupdesc);
scan->xs_hitupdesc = so->giststate->fetchTupdesc;
/* Also create a memory context that will hold the returned tuples */
diff --git a/src/backend/access/spgist/spgutils.c b/src/backend/access/spgist/spgutils.c
index 9f5379b87ac..b246e8127db 100644
--- a/src/backend/access/spgist/spgutils.c
+++ b/src/backend/access/spgist/spgutils.c
@@ -340,6 +340,7 @@ getSpGistTupleDesc(Relation index, SpGistTypeDesc *keyType)
TupleDescCompactAttr(outTupDesc, i)->attcacheoff = -1;
populate_compact_attribute(outTupDesc, spgKeyColumn);
+ TupleDescFinalize(outTupDesc);
}
return outTupDesc;
}
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 55b9f38927d..d468c9774b3 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -745,6 +745,7 @@ pg_prepared_xact(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 5, "dbid",
OIDOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
/*
diff --git a/src/backend/access/transam/xlogfuncs.c b/src/backend/access/transam/xlogfuncs.c
index ecb3c8c0820..4e35311b2f3 100644
--- a/src/backend/access/transam/xlogfuncs.c
+++ b/src/backend/access/transam/xlogfuncs.c
@@ -430,6 +430,7 @@ pg_walfile_name_offset(PG_FUNCTION_ARGS)
TupleDescInitEntry(resultTupleDesc, (AttrNumber) 2, "file_offset",
INT4OID, -1, 0);
+ TupleDescFinalize(resultTupleDesc);
resultTupleDesc = BlessTupleDesc(resultTupleDesc);
/*
diff --git a/src/backend/backup/basebackup_copy.c b/src/backend/backup/basebackup_copy.c
index 07f58b39d8c..6c3453efd80 100644
--- a/src/backend/backup/basebackup_copy.c
+++ b/src/backend/backup/basebackup_copy.c
@@ -357,6 +357,8 @@ SendXlogRecPtrResult(XLogRecPtr ptr, TimeLineID tli)
*/
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "tli", INT8OID, -1, 0);
+ TupleDescFinalize(tupdesc);
+
/* send RowDescription */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -388,6 +390,7 @@ SendTablespaceList(List *tablespaces)
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 1, "spcoid", OIDOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "spclocation", TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 3, "size", INT8OID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* send RowDescription */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 5ee6389d39c..0e93ababa87 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -481,6 +481,8 @@ ConstructTupleDescriptor(Relation heapRelation,
populate_compact_attribute(indexTupDesc, i);
}
+ TupleDescFinalize(indexTupDesc);
+
return indexTupDesc;
}
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index aadc7c202c6..a79157c43bf 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -1322,6 +1322,7 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 4, "qual",
PG_NODE_TREEOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
funcctx->user_fctx = table_infos;
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index c78dcea98c1..078a1cf5127 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -229,6 +229,12 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attcompression = InvalidCompressionMethod;
TupleDescAttr(tupdesc, 2)->attcompression = InvalidCompressionMethod;
+ populate_compact_attribute(tupdesc, 0);
+ populate_compact_attribute(tupdesc, 1);
+ populate_compact_attribute(tupdesc, 2);
+
+ TupleDescFinalize(tupdesc);
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 93918a223b8..5f922c3f5c2 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -281,6 +281,7 @@ ExplainResultDesc(ExplainStmt *stmt)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "QUERY PLAN",
result_type, -1, 0);
+ TupleDescFinalize(tupdesc);
return tupdesc;
}
diff --git a/src/backend/commands/functioncmds.c b/src/backend/commands/functioncmds.c
index 242372b1e68..3afd762e9dc 100644
--- a/src/backend/commands/functioncmds.c
+++ b/src/backend/commands/functioncmds.c
@@ -2424,6 +2424,7 @@ CallStmtResultDesc(CallStmt *stmt)
-1,
0);
}
+ TupleDescFinalize(tupdesc);
}
return tupdesc;
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index e1b808bbb60..551667650ba 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -1808,6 +1808,7 @@ pg_get_sequence_data(PG_FUNCTION_ARGS)
BOOLOID, -1, 0);
TupleDescInitEntry(resultTupleDesc, (AttrNumber) 3, "page_lsn",
LSNOID, -1, 0);
+ TupleDescFinalize(resultTupleDesc);
resultTupleDesc = BlessTupleDesc(resultTupleDesc);
seqrel = try_relation_open(relid, AccessShareLock);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index cd6d720386f..a2e3b72f156 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -1038,6 +1038,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
}
}
+ TupleDescFinalize(descriptor);
+
/*
* For relations with table AM and partitioned tables, select access
* method to use: an explicitly indicated one, or (in the case of a
@@ -1466,6 +1468,8 @@ BuildDescForRelation(const List *columns)
populate_compact_attribute(desc, attnum - 1);
}
+ TupleDescFinalize(desc);
+
return desc;
}
diff --git a/src/backend/commands/wait.c b/src/backend/commands/wait.c
index 1290df10c6f..8e920a72372 100644
--- a/src/backend/commands/wait.c
+++ b/src/backend/commands/wait.c
@@ -338,5 +338,6 @@ WaitStmtResultDesc(WaitStmt *stmt)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitEntry(tupdesc, (AttrNumber) 1, "status",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
return tupdesc;
}
diff --git a/src/backend/executor/execSRF.c b/src/backend/executor/execSRF.c
index a0b111dc0e4..b481e50acfb 100644
--- a/src/backend/executor/execSRF.c
+++ b/src/backend/executor/execSRF.c
@@ -272,6 +272,7 @@ ExecMakeTableFunctionResult(SetExprState *setexpr,
funcrettype,
-1,
0);
+ TupleDescFinalize(tupdesc);
rsinfo.setDesc = tupdesc;
}
MemoryContextSwitchTo(oldcontext);
@@ -776,6 +777,7 @@ init_sexpr(Oid foid, Oid input_collation, Expr *node,
funcrettype,
-1,
0);
+ TupleDescFinalize(tupdesc);
sexpr->funcResultDesc = tupdesc;
sexpr->funcReturnsTuple = false;
}
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 7effe954286..07b248aa5f3 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -2175,6 +2175,8 @@ ExecTypeFromTLInternal(List *targetList, bool skipjunk)
cur_resno++;
}
+ TupleDescFinalize(typeInfo);
+
return typeInfo;
}
@@ -2209,6 +2211,8 @@ ExecTypeFromExprList(List *exprList)
cur_resno++;
}
+ TupleDescFinalize(typeInfo);
+
return typeInfo;
}
diff --git a/src/backend/executor/nodeFunctionscan.c b/src/backend/executor/nodeFunctionscan.c
index 63e605e1f81..feb82d64967 100644
--- a/src/backend/executor/nodeFunctionscan.c
+++ b/src/backend/executor/nodeFunctionscan.c
@@ -414,6 +414,7 @@ ExecInitFunctionScan(FunctionScan *node, EState *estate, int eflags)
TupleDescInitEntryCollation(tupdesc,
(AttrNumber) 1,
exprCollation(funcexpr));
+ TupleDescFinalize(tupdesc);
}
else
{
@@ -485,6 +486,7 @@ ExecInitFunctionScan(FunctionScan *node, EState *estate, int eflags)
0);
}
+ TupleDescFinalize(scan_tupdesc);
Assert(attno == natts);
}
diff --git a/src/backend/parser/parse_relation.c b/src/backend/parser/parse_relation.c
index e003db520de..9c415e166ee 100644
--- a/src/backend/parser/parse_relation.c
+++ b/src/backend/parser/parse_relation.c
@@ -1883,6 +1883,7 @@ addRangeTableEntryForFunction(ParseState *pstate,
TupleDescInitEntryCollation(tupdesc,
(AttrNumber) 1,
exprCollation(funcexpr));
+ TupleDescFinalize(tupdesc);
}
else if (functypclass == TYPEFUNC_RECORD)
{
@@ -1940,6 +1941,7 @@ addRangeTableEntryForFunction(ParseState *pstate,
i++;
}
+ TupleDescFinalize(tupdesc);
/*
* Ensure that the coldeflist defines a legal set of names (no
@@ -2008,7 +2010,7 @@ addRangeTableEntryForFunction(ParseState *pstate,
0);
/* no need to set collation */
}
-
+ TupleDescFinalize(tupdesc);
Assert(natts == totalatts);
}
else
diff --git a/src/backend/parser/parse_target.c b/src/backend/parser/parse_target.c
index 3bcfc1f5e3d..f57c4d41080 100644
--- a/src/backend/parser/parse_target.c
+++ b/src/backend/parser/parse_target.c
@@ -1572,6 +1572,8 @@ expandRecordVariable(ParseState *pstate, Var *var, int levelsup)
}
Assert(lname == NULL && lvar == NULL); /* lists same length? */
+ TupleDescFinalize(tupleDesc);
+
return tupleDesc;
}
diff --git a/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c b/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
index 7c8639b32e9..9f04c9ed25d 100644
--- a/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
+++ b/src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
@@ -1073,6 +1073,7 @@ libpqrcv_processTuples(PGresult *pgres, WalRcvExecResult *walres,
for (coln = 0; coln < nRetTypes; coln++)
TupleDescInitEntry(walres->tupledesc, (AttrNumber) coln + 1,
PQfname(pgres, coln), retTypes[coln], -1, 0);
+ TupleDescFinalize(walres->tupledesc);
attinmeta = TupleDescGetAttInMetadata(walres->tupledesc);
/* No point in doing more here if there were no tuples returned. */
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c
index 79fc192b171..376ff46340d 100644
--- a/src/backend/replication/walsender.c
+++ b/src/backend/replication/walsender.c
@@ -452,6 +452,7 @@ IdentifySystem(void)
TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 4, "dbname",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -497,6 +498,7 @@ ReadReplicationSlot(ReadReplicationSlotCmd *cmd)
/* TimeLineID is unsigned, so int4 is not wide enough. */
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 3, "restart_tli",
INT8OID, -1, 0);
+ TupleDescFinalize(tupdesc);
memset(nulls, true, READ_REPLICATION_SLOT_COLS * sizeof(bool));
@@ -599,6 +601,7 @@ SendTimeLineHistory(TimeLineHistoryCmd *cmd)
tupdesc = CreateTemplateTupleDesc(2);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 1, "filename", TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "content", TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
TLHistoryFileName(histfname, cmd->timeline);
TLHistoryFilePath(path, cmd->timeline);
@@ -1016,6 +1019,7 @@ StartReplication(StartReplicationCmd *cmd)
INT8OID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 2, "next_tli_startpos",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuple */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -1370,6 +1374,7 @@ CreateReplicationSlot(CreateReplicationSlotCmd *cmd)
TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 4, "output_plugin",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
diff --git a/src/backend/utils/adt/acl.c b/src/backend/utils/adt/acl.c
index 071e3f2c49e..e210d6472be 100644
--- a/src/backend/utils/adt/acl.c
+++ b/src/backend/utils/adt/acl.c
@@ -1831,6 +1831,7 @@ aclexplode(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 4, "is_grantable",
BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
/* allocate memory for user context */
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index c083608b1d5..bfb949401d0 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -454,6 +454,7 @@ pg_stat_file(PG_FUNCTION_ARGS)
"creation", TIMESTAMPTZOID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 6,
"isdir", BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
memset(isnull, false, sizeof(isnull));
diff --git a/src/backend/utils/adt/lockfuncs.c b/src/backend/utils/adt/lockfuncs.c
index 9dadd6da672..4481c354fd6 100644
--- a/src/backend/utils/adt/lockfuncs.c
+++ b/src/backend/utils/adt/lockfuncs.c
@@ -146,6 +146,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 16, "waitstart",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = BlessTupleDesc(tupdesc);
/*
diff --git a/src/backend/utils/adt/orderedsetaggs.c b/src/backend/utils/adt/orderedsetaggs.c
index 3b6da8e36ac..fd8b8676470 100644
--- a/src/backend/utils/adt/orderedsetaggs.c
+++ b/src/backend/utils/adt/orderedsetaggs.c
@@ -233,6 +233,7 @@ ordered_set_startup(FunctionCallInfo fcinfo, bool use_tuples)
-1,
0);
+ TupleDescFinalize(newdesc);
FreeTupleDesc(qstate->tupdesc);
qstate->tupdesc = newdesc;
}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 5ac022274a7..bad5642d9c9 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -770,6 +770,7 @@ pg_stat_get_backend_subxact(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 2, "subxact_overflow",
BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
if ((local_beentry = pgstat_get_local_beentry_by_proc_number(procNumber)) != NULL)
@@ -1671,6 +1672,7 @@ pg_stat_wal_build_tuple(PgStat_WalCounters wal_counters,
TupleDescInitEntry(tupdesc, (AttrNumber) 6, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
/* Fill values and NULLs */
@@ -2098,6 +2100,7 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 7, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
/* Get statistics about the archiver process */
@@ -2179,6 +2182,7 @@ pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
TIMESTAMPTZOID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 13, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
namestrcpy(&slotname, text_to_cstring(slotname_text));
@@ -2266,6 +2270,7 @@ pg_stat_get_subscription_stats(PG_FUNCTION_ARGS)
INT8OID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 13, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
if (!subentry)
diff --git a/src/backend/utils/adt/tsvector_op.c b/src/backend/utils/adt/tsvector_op.c
index 71c7c7d3b3c..d8dece42b9b 100644
--- a/src/backend/utils/adt/tsvector_op.c
+++ b/src/backend/utils/adt/tsvector_op.c
@@ -651,6 +651,7 @@ tsvector_unnest(PG_FUNCTION_ARGS)
TEXTARRAYOID, -1, 0);
if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
elog(ERROR, "return type must be a row type");
+ TupleDescFinalize(tupdesc);
funcctx->tuple_desc = tupdesc;
funcctx->user_fctx = PG_GETARG_TSVECTOR_COPY(0);
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a1c88c6b1b6..d27ac216e6d 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -729,6 +729,8 @@ RelationBuildTupleDesc(Relation relation)
pfree(constr);
relation->rd_att->constr = NULL;
}
+
+ TupleDescFinalize(relation->rd_att);
}
/*
@@ -1985,6 +1987,7 @@ formrdesc(const char *relationName, Oid relationReltype,
/* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
TupleDescCompactAttr(relation->rd_att, 0)->attcacheoff = 0;
+ TupleDescFinalize(relation->rd_att);
/* mark not-null status */
if (has_not_null)
@@ -3688,6 +3691,8 @@ RelationBuildLocalRelation(const char *relname,
for (i = 0; i < natts; i++)
TupleDescAttr(rel->rd_att, i)->attrelid = relid;
+ TupleDescFinalize(rel->rd_att);
+
rel->rd_rel->reltablespace = reltablespace;
if (mapped_relation)
@@ -4443,6 +4448,7 @@ BuildHardcodedDescriptor(int natts, const FormData_pg_attribute *attrs)
/* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
TupleDescCompactAttr(result, 0)->attcacheoff = 0;
+ TupleDescFinalize(result);
/* Note: we don't bother to set up a TupleConstr entry */
@@ -6291,6 +6297,8 @@ load_relcache_init_file(bool shared)
populate_compact_attribute(rel->rd_att, i);
}
+ TupleDescFinalize(rel->rd_att);
+
/* next read the access method specific field */
if (fread(&len, 1, sizeof(len), fp) != sizeof(len))
goto read_failed;
diff --git a/src/backend/utils/fmgr/funcapi.c b/src/backend/utils/fmgr/funcapi.c
index 8a934ea8dca..516d02cfb82 100644
--- a/src/backend/utils/fmgr/funcapi.c
+++ b/src/backend/utils/fmgr/funcapi.c
@@ -340,6 +340,8 @@ get_expr_result_type(Node *expr,
exprCollation(col));
i++;
}
+ TupleDescFinalize(tupdesc);
+
if (resultTypeId)
*resultTypeId = rexpr->row_typeid;
if (resultTupleDesc)
@@ -1044,6 +1046,7 @@ resolve_polymorphic_tupdesc(TupleDesc tupdesc, oidvector *declared_args,
}
}
+ TupleDescFinalize(tupdesc);
return true;
}
@@ -1853,6 +1856,8 @@ build_function_result_tupdesc_d(char prokind,
0);
}
+ TupleDescFinalize(desc);
+
return desc;
}
@@ -1970,6 +1975,7 @@ TypeGetTupleDesc(Oid typeoid, List *colaliases)
typeoid,
-1,
0);
+ TupleDescFinalize(tupdesc);
}
else if (functypclass == TYPEFUNC_RECORD)
{
diff --git a/src/backend/utils/misc/guc_funcs.c b/src/backend/utils/misc/guc_funcs.c
index 8524dd3a981..472cb5393ce 100644
--- a/src/backend/utils/misc/guc_funcs.c
+++ b/src/backend/utils/misc/guc_funcs.c
@@ -444,6 +444,7 @@ GetPGVariableResultDesc(const char *name)
TupleDescInitEntry(tupdesc, (AttrNumber) 1, varname,
TEXTOID, -1, 0);
}
+ TupleDescFinalize(tupdesc);
return tupdesc;
}
@@ -465,6 +466,7 @@ ShowGUCConfigOption(const char *name, DestReceiver *dest)
tupdesc = CreateTemplateTupleDesc(1);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 1, varname,
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -499,6 +501,7 @@ ShowAllGUCConfig(DestReceiver *dest)
TEXTOID, -1, 0);
TupleDescInitBuiltinEntry(tupdesc, (AttrNumber) 3, "description",
TEXTOID, -1, 0);
+ TupleDescFinalize(tupdesc);
/* prepare for projection of tuples */
tstate = begin_tup_output_tupdesc(dest, tupdesc, &TTSOpsVirtual);
@@ -934,6 +937,8 @@ show_all_settings(PG_FUNCTION_ARGS)
TupleDescInitEntry(tupdesc, (AttrNumber) 17, "pending_restart",
BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+
/*
* Generate attribute metadata needed later to produce tuples from raw
* C strings
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index d46cdbf7a3c..595413dbbc5 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -195,6 +195,7 @@ extern TupleDesc CreateTupleDescTruncatedCopy(TupleDesc tupdesc, int natts);
extern TupleDesc CreateTupleDescCopyConstr(TupleDesc tupdesc);
+#define TupleDescFinalize(d) ((void) 0)
#define TupleDescSize(src) \
(offsetof(struct TupleDescData, compact_attrs) + \
(src)->natts * sizeof(CompactAttribute) + \
diff --git a/src/pl/plpgsql/src/pl_comp.c b/src/pl/plpgsql/src/pl_comp.c
index 5ecc7766757..b72c963b3be 100644
--- a/src/pl/plpgsql/src/pl_comp.c
+++ b/src/pl/plpgsql/src/pl_comp.c
@@ -1912,6 +1912,8 @@ build_row_from_vars(PLpgSQL_variable **vars, int numvars)
TupleDescInitEntryCollation(row->rowtupdesc, i + 1, typcoll);
}
+ TupleDescFinalize(row->rowtupdesc);
+
return row;
}
diff --git a/src/test/modules/test_custom_stats/test_custom_fixed_stats.c b/src/test/modules/test_custom_stats/test_custom_fixed_stats.c
index 485e08e5c19..f9e7c717280 100644
--- a/src/test/modules/test_custom_stats/test_custom_fixed_stats.c
+++ b/src/test/modules/test_custom_stats/test_custom_fixed_stats.c
@@ -206,6 +206,7 @@ test_custom_stats_fixed_report(PG_FUNCTION_ARGS)
INT8OID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 2, "stats_reset",
TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
BlessTupleDesc(tupdesc);
values[0] = Int64GetDatum(stats->numcalls);
diff --git a/src/test/modules/test_predtest/test_predtest.c b/src/test/modules/test_predtest/test_predtest.c
index 679a5de456d..48ca2a4ea70 100644
--- a/src/test/modules/test_predtest/test_predtest.c
+++ b/src/test/modules/test_predtest/test_predtest.c
@@ -230,6 +230,7 @@ test_predtest(PG_FUNCTION_ARGS)
"s_r_holds", BOOLOID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 8,
"w_r_holds", BOOLOID, -1, 0);
+ TupleDescFinalize(tupdesc);
tupdesc = BlessTupleDesc(tupdesc);
values[0] = BoolGetDatum(strong_implied_by);
--
2.51.0
[text/plain] v13-0004-Optimize-tuple-deformation.patch (81.1K, 5-v13-0004-Optimize-tuple-deformation.patch)
download | inline diff:
From ab5fc2c7de7c6661d568e677eaad37c43977bef8 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Tue, 31 Dec 2024 09:19:24 +1300
Subject: [PATCH v13 4/6] Optimize tuple deformation
This commit includes various optimizations to improve the performance of
tuple deformation.
We now precalculate CompactAttribute's attcacheoff, which allows us to
remove the code from the deform routines which was setting the
attcacheoff. Setting the attcacheoff is handled by TupleDescFinalize(),
which must be called before the TupleDesc is used for anything. Having
this TupleDescFinalize() function means we can store the first
attribute in the TupleDesc which does not have an offset cached. That
allows us to add a dedicated deforming loop to deform all attributes up
to the final one with an attcacheoff set, or up to the first NULL
attribute, whichever comes first.
We also record the maximum attribute number which is guaranteed to exist
in the tuple, that is, has a NOT NULL constraint and isn't an
atthasmissing attribute. When deforming only attributes prior to the
guaranteed attnum, we've no need to access the tuple's natt count. As an
additional optimization, we only count fixed-width columns when
calculating the maximum guaranteed column as this eliminates the need to
emit code to fetch byref types in the deformation loop for guaranteed
attributes.
Some locations in the code deform tuples that have yet to go through NOT
NULL constraint validation. We're unable to perform the guaranteed
attribute optimization when that's the case. The optimization is opt-in
via the TupleTableSlot using the TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS
flag.
This commit also adds a more efficient way of populating the isnull
array by using a bit-wise trick which performs multiplication on the
inverse of the tuple's bitmap byte and masking out all but the lower bit
of each of the boolean's byte. This results in much more optimal code
when compared to determining the NULLness via att_isnull(). 8 isnull
elements are processed at once using this method, which means we need to
round the tts_isnull array size up to the next 8 bytes. The palloc code
does this anyway, but the round-up needed to be formalized so as not to
overwrite the sentinel byte in debug builds.
---
src/backend/access/common/heaptuple.c | 360 +++++++--------
src/backend/access/common/indextuple.c | 363 +++++++--------
src/backend/access/common/tupdesc.c | 51 +++
src/backend/access/spgist/spgutils.c | 3 -
src/backend/executor/execMain.c | 8 +-
src/backend/executor/execTuples.c | 415 ++++++++++--------
src/backend/executor/execUtils.c | 2 +-
src/backend/executor/nodeAgg.c | 2 +-
src/backend/executor/nodeBitmapHeapscan.c | 5 +-
src/backend/executor/nodeCtescan.c | 2 +-
src/backend/executor/nodeCustom.c | 4 +-
src/backend/executor/nodeForeignscan.c | 4 +-
src/backend/executor/nodeFunctionscan.c | 2 +-
src/backend/executor/nodeIndexonlyscan.c | 7 +-
src/backend/executor/nodeIndexscan.c | 5 +-
.../executor/nodeNamedtuplestorescan.c | 2 +-
src/backend/executor/nodeSamplescan.c | 6 +-
src/backend/executor/nodeSeqscan.c | 3 +-
src/backend/executor/nodeSubqueryscan.c | 3 +-
src/backend/executor/nodeTableFuncscan.c | 2 +-
src/backend/executor/nodeTidrangescan.c | 5 +-
src/backend/executor/nodeTidscan.c | 5 +-
src/backend/executor/nodeValuesscan.c | 2 +-
src/backend/executor/nodeWorktablescan.c | 2 +-
src/backend/jit/llvm/llvmjit_deform.c | 6 -
src/backend/replication/pgoutput/pgoutput.c | 4 +-
src/backend/utils/cache/relcache.c | 12 -
src/include/access/tupdesc.h | 20 +-
src/include/access/tupmacs.h | 224 +++++++++-
src/include/executor/executor.h | 3 +-
src/include/executor/tuptable.h | 28 +-
src/test/modules/deform_bench/deform_bench.c | 4 +-
32 files changed, 900 insertions(+), 664 deletions(-)
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 11bec20e82e..b2ac7fef35b 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -498,19 +498,7 @@ heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc)
* nocachegetattr
*
* This only gets called from fastgetattr(), in cases where we
- * can't use a cacheoffset and the value is not null.
- *
- * This caches attribute offsets in the attribute descriptor.
- *
- * An alternative way to speed things up would be to cache offsets
- * with the tuple, but that seems more difficult unless you take
- * the storage hit of actually putting those offsets into the
- * tuple you send to disk. Yuck.
- *
- * This scheme will be slightly slower than that, but should
- * perform well for queries which hit large #'s of tuples. After
- * you cache the offsets once, examining all the other tuples using
- * the same attribute descriptor will go much quicker. -cim 5/4/91
+ * can't use the attcacheoff and the value is not null.
*
* NOTE: if you need to change this code, see also heap_deform_tuple.
* Also see nocache_index_getattr, which is the same code for index
@@ -522,194 +510,125 @@ nocachegetattr(HeapTuple tup,
int attnum,
TupleDesc tupleDesc)
{
+ CompactAttribute *cattr;
HeapTupleHeader td = tup->t_data;
char *tp; /* ptr to data part of tuple */
bits8 *bp = td->t_bits; /* ptr to null bitmap in tuple */
- bool slow = false; /* do we have to walk attrs? */
int off; /* current offset within data */
+ int startAttr;
+ int firstNullAttr;
+ int i;
+ bool hasnulls = HeapTupleHasNulls(tup);
- /* ----------------
- * Three cases:
- *
- * 1: No nulls and no variable-width attributes.
- * 2: Has a null or a var-width AFTER att.
- * 3: Has nulls or var-widths BEFORE att.
- * ----------------
- */
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
attnum--;
- if (!HeapTupleNoNulls(tup))
+ /*
+ * To minimize the number of attributes we need to look at, start walking
+ * the tuple at the attribute with the highest attcacheoff prior to attnum
+ * or the first NULL attribute prior to attnum, whichever comes first.
+ */
+ if (hasnulls)
+ firstNullAttr = first_null_attr(bp, attnum);
+ else
+ firstNullAttr = attnum;
+
+ if (tupleDesc->firstNonCachedOffsetAttr > 0)
{
/*
- * there's a null somewhere in the tuple
- *
- * check to see if any preceding bits are null...
+ * Start at the highest attcacheoff attribute with no NULLs in prior
+ * attributes.
*/
- int byte = attnum >> 3;
- int finalbit = attnum & 0x07;
-
- /* check for nulls "before" final bit of last byte */
- if ((~bp[byte]) & ((1 << finalbit) - 1))
- slow = true;
- else
- {
- /* check for nulls in any "earlier" bytes */
- int i;
-
- for (i = 0; i < byte; i++)
- {
- if (bp[i] != 0xFF)
- {
- slow = true;
- break;
- }
- }
- }
+ startAttr = Min(tupleDesc->firstNonCachedOffsetAttr - 1, firstNullAttr);
+ off = TupleDescCompactAttr(tupleDesc, startAttr)->attcacheoff;
+ }
+ else
+ {
+ /* Otherwise, start at the beginning... */
+ startAttr = 0;
+ off = 0;
}
tp = (char *) td + td->t_hoff;
- if (!slow)
+ /*
+ * Calculate 'off' up to the first NULL attr. We use two cheaper loops
+ * when the tuple has no variable-width columns. When variable-width
+ * columns exists, we use att_addlength_pointer() to move the offset
+ * beyond the current attribute.
+ */
+ if (!HeapTupleHasVarWidth(tup))
{
- CompactAttribute *att;
-
- /*
- * If we get here, there are no nulls up to and including the target
- * attribute. If we have a cached offset, we can use it.
- */
- att = TupleDescCompactAttr(tupleDesc, attnum);
- if (att->attcacheoff >= 0)
- return fetchatt(att, tp + att->attcacheoff);
-
- /*
- * Otherwise, check for non-fixed-length attrs up to and including
- * target. If there aren't any, it's safe to cheaply initialize the
- * cached offsets for these attrs.
- */
- if (HeapTupleHasVarWidth(tup))
+ for (i = startAttr; i < firstNullAttr; i++)
{
- int j;
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- for (j = 0; j <= attnum; j++)
- {
- if (TupleDescCompactAttr(tupleDesc, j)->attlen <= 0)
- {
- slow = true;
- break;
- }
- }
+ off = att_nominal_alignby(off, cattr->attalignby);
+ off += cattr->attlen;
}
- }
-
- if (!slow)
- {
- int natts = tupleDesc->natts;
- int j = 1;
-
- /*
- * If we get here, we have a tuple with no nulls or var-widths up to
- * and including the target attribute, so we can use the cached offset
- * ... only we don't have it yet, or we'd not have got here. Since
- * it's cheap to compute offsets for fixed-width columns, we take the
- * opportunity to initialize the cached offsets for *all* the leading
- * fixed-width columns, in hope of avoiding future visits to this
- * routine.
- */
- TupleDescCompactAttr(tupleDesc, 0)->attcacheoff = 0;
-
- /* we might have set some offsets in the slow path previously */
- while (j < natts && TupleDescCompactAttr(tupleDesc, j)->attcacheoff > 0)
- j++;
-
- off = TupleDescCompactAttr(tupleDesc, j - 1)->attcacheoff +
- TupleDescCompactAttr(tupleDesc, j - 1)->attlen;
- for (; j < natts; j++)
+ for (; i < attnum; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, j);
+ if (att_isnull(i, bp))
+ continue;
- if (att->attlen <= 0)
- break;
-
- off = att_nominal_alignby(off, att->attalignby);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- att->attcacheoff = off;
-
- off += att->attlen;
+ off = att_nominal_alignby(off, cattr->attalignby);
+ off += cattr->attlen;
}
-
- Assert(j > attnum);
-
- off = TupleDescCompactAttr(tupleDesc, attnum)->attcacheoff;
}
else
{
- bool usecache = true;
- int i;
-
- /*
- * Now we know that we have to walk the tuple CAREFULLY. But we still
- * might be able to cache some offsets for next time.
- *
- * Note - This loop is a little tricky. For each non-null attribute,
- * we have to first account for alignment padding before the attr,
- * then advance over the attr based on its length. Nulls have no
- * storage and no alignment padding either. We can use/set
- * attcacheoff until we reach either a null or a var-width attribute.
- */
- off = 0;
- for (i = 0;; i++) /* loop exit is at "break" */
+ for (i = startAttr; i < firstNullAttr; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, i);
+ int attlen;
- if (HeapTupleHasNulls(tup) && att_isnull(i, bp))
- {
- usecache = false;
- continue; /* this cannot be the target att */
- }
+ cattr = TupleDescCompactAttr(tupleDesc, i);
+ attlen = cattr->attlen;
- /* If we know the next offset, we can skip the rest */
- if (usecache && att->attcacheoff >= 0)
- off = att->attcacheoff;
- else if (att->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be
- * no pad bytes in any case: then the offset will be valid for
- * either an aligned or unaligned value.
- */
- if (usecache &&
- off == att_nominal_alignby(off, att->attalignby))
- att->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, att->attalignby, -1,
- tp + off);
- usecache = false;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, att->attalignby);
+ /*
+ * cstrings don't exist in heap tuples. Use pg_assume to instruct
+ * the compiler not to emit the cstring-related code in
+ * att_addlength_pointer().
+ */
+ pg_assume(attlen > 0 || attlen == -1);
- if (usecache)
- att->attcacheoff = off;
- }
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ attlen,
+ tp + off);
+ off = att_addlength_pointer(off, attlen, tp + off);
+ }
- if (i == attnum)
- break;
+ for (; i < attnum; i++)
+ {
+ int attlen;
- off = att_addlength_pointer(off, att->attlen, tp + off);
+ if (att_isnull(i, bp))
+ continue;
- if (usecache && att->attlen <= 0)
- usecache = false;
+ cattr = TupleDescCompactAttr(tupleDesc, i);
+ attlen = cattr->attlen;
+
+ /* As above, heaptuples have no cstrings */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ off = att_pointer_alignby(off, cattr->attalignby, attlen,
+ tp + off);
+ off = att_addlength_pointer(off, attlen, tp + off);
}
}
- return fetchatt(TupleDescCompactAttr(tupleDesc, attnum), tp + off);
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+
+ return fetchatt(cattr, tp + off);
}
/* ----------------
@@ -1347,6 +1266,7 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,
Datum *values, bool *isnull)
{
HeapTupleHeader tup = tuple->t_data;
+ CompactAttribute *cattr;
bool hasnulls = HeapTupleHasNulls(tuple);
int tdesc_natts = tupleDesc->natts;
int natts; /* number of atts to extract */
@@ -1354,70 +1274,98 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,
char *tp; /* ptr to tuple data */
uint32 off; /* offset in tuple data */
bits8 *bp = tup->t_bits; /* ptr to null bitmap in tuple */
- bool slow = false; /* can we use/set attcacheoff? */
+ int firstNonCacheOffsetAttr;
+ int firstNullAttr;
natts = HeapTupleHeaderGetNatts(tup);
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
+
/*
* In inheritance situations, it is possible that the given tuple actually
* has more fields than the caller is expecting. Don't run off the end of
* the caller's arrays.
*/
natts = Min(natts, tdesc_natts);
+ firstNonCacheOffsetAttr = Min(tupleDesc->firstNonCachedOffsetAttr, natts);
+
+ if (hasnulls)
+ {
+ firstNullAttr = first_null_attr(bp, natts);
+
+ /*
+ * XXX: it'd be nice to use populate_isnull_array() here, but that
+ * requires that the isnull array's size is rounded up to the next
+ * multiple of 8. Doing that would require adjusting many locations
+ * that allocate the array.
+ */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, firstNullAttr);
+ }
+ else
+ firstNullAttr = natts;
tp = (char *) tup + tup->t_hoff;
+ attnum = 0;
- off = 0;
+ if (firstNonCacheOffsetAttr > 0)
+ {
+#ifdef USE_ASSERT_CHECKING
+ /* In Assert enabled builds, verify attcacheoff is correct */
+ int offcheck = 0;
+#endif
+ do
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ off = cattr->attcacheoff;
- for (attnum = 0; attnum < natts; attnum++)
+#ifdef USE_ASSERT_CHECKING
+ offcheck = att_nominal_alignby(offcheck, cattr->attalignby);
+ Assert(offcheck == cattr->attcacheoff);
+ offcheck += cattr->attlen;
+#endif
+
+ values[attnum] = fetch_att_noerr(tp + off,
+ cattr->attbyval,
+ cattr->attlen);
+ } while (++attnum < firstNonCacheOffsetAttr);
+ off += cattr->attlen;
+ }
+ else
+ off = 0;
+
+ for (; attnum < firstNullAttr; attnum++)
{
- CompactAttribute *thisatt = TupleDescCompactAttr(tupleDesc, attnum);
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
+ }
+
+ for (; attnum < natts; attnum++)
+ {
+ Assert(hasnulls);
- if (hasnulls && att_isnull(attnum, bp))
+ if (att_isnull(attnum, bp))
{
values[attnum] = (Datum) 0;
isnull[attnum] = true;
- slow = true; /* can't use attcacheoff anymore */
continue;
}
isnull[attnum] = false;
-
- if (!slow && thisatt->attcacheoff >= 0)
- off = thisatt->attcacheoff;
- else if (thisatt->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be no
- * pad bytes in any case: then the offset will be valid for either
- * an aligned or unaligned value.
- */
- if (!slow &&
- off == att_nominal_alignby(off, thisatt->attalignby))
- thisatt->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, thisatt->attalignby, -1,
- tp + off);
- slow = true;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, thisatt->attalignby);
-
- if (!slow)
- thisatt->attcacheoff = off;
- }
-
- values[attnum] = fetchatt(thisatt, tp + off);
-
- off = att_addlength_pointer(off, thisatt->attlen, tp + off);
-
- if (thisatt->attlen <= 0)
- slow = true; /* can't use attcacheoff anymore */
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
}
/*
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index d6350201e01..8c410853191 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -223,18 +223,6 @@ index_form_tuple_context(TupleDesc tupleDescriptor,
*
* This gets called from index_getattr() macro, and only in cases
* where we can't use cacheoffset and the value is not null.
- *
- * This caches attribute offsets in the attribute descriptor.
- *
- * An alternative way to speed things up would be to cache offsets
- * with the tuple, but that seems more difficult unless you take
- * the storage hit of actually putting those offsets into the
- * tuple you send to disk. Yuck.
- *
- * This scheme will be slightly slower than that, but should
- * perform well for queries which hit large #'s of tuples. After
- * you cache the offsets once, examining all the other tuples using
- * the same attribute descriptor will go much quicker. -cim 5/4/91
* ----------------
*/
Datum
@@ -242,205 +230,124 @@ nocache_index_getattr(IndexTuple tup,
int attnum,
TupleDesc tupleDesc)
{
+ CompactAttribute *cattr;
char *tp; /* ptr to data part of tuple */
bits8 *bp = NULL; /* ptr to null bitmap in tuple */
- bool slow = false; /* do we have to walk attrs? */
int data_off; /* tuple data offset */
int off; /* current offset within data */
+ int startAttr;
+ int firstNullAttr;
+ bool hasnulls = IndexTupleHasNulls(tup);
+ int i;
- /* ----------------
- * Three cases:
- *
- * 1: No nulls and no variable-width attributes.
- * 2: Has a null or a var-width AFTER att.
- * 3: Has nulls or var-widths BEFORE att.
- * ----------------
- */
-
- data_off = IndexInfoFindDataOffset(tup->t_info);
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
attnum--;
- if (IndexTupleHasNulls(tup))
- {
- /*
- * there's a null somewhere in the tuple
- *
- * check to see if desired att is null
- */
+ data_off = IndexInfoFindDataOffset(tup->t_info);
+ tp = (char *) tup + data_off;
- /* XXX "knows" t_bits are just after fixed tuple header! */
+ /*
+ * To minimize the number of attributes we need to look at, start walking
+ * the tuple at the attribute with the highest attcacheoff prior to attnum
+ * or the first NULL attribute prior to attnum, whichever comes first.
+ */
+ if (hasnulls)
+ {
bp = (bits8 *) ((char *) tup + sizeof(IndexTupleData));
-
- /*
- * Now check to see if any preceding bits are null...
- */
- {
- int byte = attnum >> 3;
- int finalbit = attnum & 0x07;
-
- /* check for nulls "before" final bit of last byte */
- if ((~bp[byte]) & ((1 << finalbit) - 1))
- slow = true;
- else
- {
- /* check for nulls in any "earlier" bytes */
- int i;
-
- for (i = 0; i < byte; i++)
- {
- if (bp[i] != 0xFF)
- {
- slow = true;
- break;
- }
- }
- }
- }
+ firstNullAttr = first_null_attr(bp, attnum);
}
+ else
+ firstNullAttr = attnum;
- tp = (char *) tup + data_off;
-
- if (!slow)
+ if (tupleDesc->firstNonCachedOffsetAttr > 0)
{
- CompactAttribute *att;
-
- /*
- * If we get here, there are no nulls up to and including the target
- * attribute. If we have a cached offset, we can use it.
- */
- att = TupleDescCompactAttr(tupleDesc, attnum);
- if (att->attcacheoff >= 0)
- return fetchatt(att, tp + att->attcacheoff);
-
/*
- * Otherwise, check for non-fixed-length attrs up to and including
- * target. If there aren't any, it's safe to cheaply initialize the
- * cached offsets for these attrs.
+ * Start at the highest attcacheoff attribute with no NULLs in prior
+ * attributes.
*/
- if (IndexTupleHasVarwidths(tup))
- {
- int j;
-
- for (j = 0; j <= attnum; j++)
- {
- if (TupleDescCompactAttr(tupleDesc, j)->attlen <= 0)
- {
- slow = true;
- break;
- }
- }
- }
+ startAttr = Min(tupleDesc->firstNonCachedOffsetAttr - 1, firstNullAttr);
+ off = TupleDescCompactAttr(tupleDesc, startAttr)->attcacheoff;
}
-
- if (!slow)
+ else
{
- int natts = tupleDesc->natts;
- int j = 1;
-
- /*
- * If we get here, we have a tuple with no nulls or var-widths up to
- * and including the target attribute, so we can use the cached offset
- * ... only we don't have it yet, or we'd not have got here. Since
- * it's cheap to compute offsets for fixed-width columns, we take the
- * opportunity to initialize the cached offsets for *all* the leading
- * fixed-width columns, in hope of avoiding future visits to this
- * routine.
- */
- TupleDescCompactAttr(tupleDesc, 0)->attcacheoff = 0;
+ /* Otherwise, start at the beginning... */
+ startAttr = 0;
+ off = 0;
+ }
- /* we might have set some offsets in the slow path previously */
- while (j < natts && TupleDescCompactAttr(tupleDesc, j)->attcacheoff > 0)
- j++;
+ /*
+ * Calculate 'off' up to the first NULL attr. We use two cheaper loops
+ * when the tuple has no variable-width columns. When variable-width
+ * columns exists, we use att_addlength_pointer() to move the offset
+ * beyond the current attribute.
+ */
+ if (IndexTupleHasVarwidths(tup))
+ {
+ /* Calculate the offset up until the first NULL */
+ for (i = startAttr; i < firstNullAttr; i++)
+ {
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- off = TupleDescCompactAttr(tupleDesc, j - 1)->attcacheoff +
- TupleDescCompactAttr(tupleDesc, j - 1)->attlen;
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+ off = att_addlength_pointer(off, cattr->attlen, tp + off);
+ }
- for (; j < natts; j++)
+ /* Calculate the offset for any remaining columns. */
+ for (; i < attnum; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, j);
+ Assert(hasnulls);
- if (att->attlen <= 0)
- break;
+ if (att_isnull(i, bp))
+ continue;
- off = att_nominal_alignby(off, att->attalignby);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- att->attcacheoff = off;
-
- off += att->attlen;
+ off = att_pointer_alignby(off,
+ cattr->attalignby,
+ cattr->attlen,
+ tp + off);
+ off = att_addlength_pointer(off, cattr->attlen, tp + off);
}
-
- Assert(j > attnum);
-
- off = TupleDescCompactAttr(tupleDesc, attnum)->attcacheoff;
}
else
{
- bool usecache = true;
- int i;
+ /* Handle tuples with only fixed-width attributes */
- /*
- * Now we know that we have to walk the tuple CAREFULLY. But we still
- * might be able to cache some offsets for next time.
- *
- * Note - This loop is a little tricky. For each non-null attribute,
- * we have to first account for alignment padding before the attr,
- * then advance over the attr based on its length. Nulls have no
- * storage and no alignment padding either. We can use/set
- * attcacheoff until we reach either a null or a var-width attribute.
- */
- off = 0;
- for (i = 0;; i++) /* loop exit is at "break" */
+ /* Calculate the offset up until the first NULL */
+ for (i = startAttr; i < firstNullAttr; i++)
{
- CompactAttribute *att = TupleDescCompactAttr(tupleDesc, i);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- if (IndexTupleHasNulls(tup) && att_isnull(i, bp))
- {
- usecache = false;
- continue; /* this cannot be the target att */
- }
-
- /* If we know the next offset, we can skip the rest */
- if (usecache && att->attcacheoff >= 0)
- off = att->attcacheoff;
- else if (att->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be
- * no pad bytes in any case: then the offset will be valid for
- * either an aligned or unaligned value.
- */
- if (usecache &&
- off == att_nominal_alignby(off, att->attalignby))
- att->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, att->attalignby, -1,
- tp + off);
- usecache = false;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, att->attalignby);
+ Assert(cattr->attlen > 0);
+ off = att_nominal_alignby(off, cattr->attalignby);
+ off += cattr->attlen;
+ }
- if (usecache)
- att->attcacheoff = off;
- }
+ /* Calculate the offset for any remaining columns. */
+ for (; i < attnum; i++)
+ {
+ Assert(hasnulls);
- if (i == attnum)
- break;
+ if (att_isnull(i, bp))
+ continue;
- off = att_addlength_pointer(off, att->attlen, tp + off);
+ cattr = TupleDescCompactAttr(tupleDesc, i);
- if (usecache && att->attlen <= 0)
- usecache = false;
+ Assert(cattr->attlen > 0);
+ off = att_nominal_alignby(off, cattr->attalignby);
+ off += cattr->attlen;
}
}
- return fetchatt(TupleDescCompactAttr(tupleDesc, attnum), tp + off);
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ off = att_pointer_alignby(off, cattr->attalignby,
+ cattr->attlen, tp + off);
+ return fetchatt(cattr, tp + off);
}
/*
@@ -480,63 +387,87 @@ index_deform_tuple_internal(TupleDesc tupleDescriptor,
Datum *values, bool *isnull,
char *tp, bits8 *bp, int hasnulls)
{
+ CompactAttribute *cattr;
int natts = tupleDescriptor->natts; /* number of atts to extract */
- int attnum;
- int off = 0; /* offset in tuple data */
- bool slow = false; /* can we use/set attcacheoff? */
+ int attnum = 0;
+ uint32 off = 0; /* offset in tuple data */
+ int firstNonCacheOffsetAttr;
+ int firstNullAttr;
/* Assert to protect callers who allocate fixed-size arrays */
Assert(natts <= INDEX_MAX_KEYS);
- for (attnum = 0; attnum < natts; attnum++)
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDescriptor->firstNonCachedOffsetAttr >= 0);
+
+ firstNonCacheOffsetAttr = Min(tupleDescriptor->firstNonCachedOffsetAttr, natts);
+
+ if (hasnulls)
+ {
+ firstNullAttr = first_null_attr(bp, natts);
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, firstNullAttr);
+ }
+ else
+ firstNullAttr = natts;
+
+ if (firstNonCacheOffsetAttr > 0)
{
- CompactAttribute *thisatt = TupleDescCompactAttr(tupleDescriptor, attnum);
+#ifdef USE_ASSERT_CHECKING
+ /* In Assert enabled builds, verify attcacheoff is correct */
+ off = 0;
+#endif
- if (hasnulls && att_isnull(attnum, bp))
+ do
{
- values[attnum] = (Datum) 0;
- isnull[attnum] = true;
- slow = true; /* can't use attcacheoff anymore */
- continue;
- }
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDescriptor, attnum);
- isnull[attnum] = false;
+#ifdef USE_ASSERT_CHECKING
+ off = att_nominal_alignby(off, cattr->attalignby);
+ Assert(off == cattr->attcacheoff);
+ off += cattr->attlen;
+#endif
- if (!slow && thisatt->attcacheoff >= 0)
- off = thisatt->attcacheoff;
- else if (thisatt->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be no
- * pad bytes in any case: then the offset will be valid for either
- * an aligned or unaligned value.
- */
- if (!slow &&
- off == att_nominal_alignby(off, thisatt->attalignby))
- thisatt->attcacheoff = off;
- else
- {
- off = att_pointer_alignby(off, thisatt->attalignby, -1,
- tp + off);
- slow = true;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- off = att_nominal_alignby(off, thisatt->attalignby);
+ values[attnum] = fetch_att_noerr(tp + cattr->attcacheoff, cattr->attbyval,
+ cattr->attlen);
+ } while (++attnum < firstNonCacheOffsetAttr);
- if (!slow)
- thisatt->attcacheoff = off;
- }
+ off = cattr->attcacheoff + cattr->attlen;
+ }
- values[attnum] = fetchatt(thisatt, tp + off);
+ for (; attnum < firstNullAttr; attnum++)
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDescriptor, attnum);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
+ }
- off = att_addlength_pointer(off, thisatt->attlen, tp + off);
+ for (; attnum < natts; attnum++)
+ {
+ Assert(hasnulls);
- if (thisatt->attlen <= 0)
- slow = true; /* can't use attcacheoff anymore */
+ if (att_isnull(attnum, bp))
+ {
+ values[attnum] = (Datum) 0;
+ isnull[attnum] = true;
+ continue;
+ }
+
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDescriptor, attnum);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ cattr->attlen,
+ cattr->attalignby);
}
}
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 2137385a833..c68561337d7 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -197,6 +197,10 @@ CreateTemplateTupleDesc(int natts)
desc->tdtypmod = -1;
desc->tdrefcount = -1; /* assume not reference-counted */
+ /* This will be set to the correct value by TupleDescFinalize() */
+ desc->firstNonCachedOffsetAttr = -1;
+ desc->firstNonGuaranteedAttr = -1;
+
return desc;
}
@@ -457,6 +461,9 @@ TupleDescCopy(TupleDesc dst, TupleDesc src)
* descriptor to another.
*
* !!! Constraints and defaults are not copied !!!
+ *
+ * The caller must take care of calling TupleDescFinalize() on 'dst' once all
+ * TupleDesc changes have been made.
*/
void
TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
@@ -489,6 +496,50 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
populate_compact_attribute(dst, dstAttno - 1);
}
+/*
+ * TupleDescFinalize
+ * Finalize the given TupleDesc. This must be called after the
+ * attributes arrays have been populated or adjusted by any code.
+ *
+ * Must be called after populate_compact_attribute() and before
+ * BlessTupleDesc().
+ */
+void
+TupleDescFinalize(TupleDesc tupdesc)
+{
+ int firstNonCachedOffsetAttr = 0;
+ int firstNonGuaranteedAttr = tupdesc->natts;
+ int off = 0;
+
+ for (int i = 0; i < tupdesc->natts; i++)
+ {
+ CompactAttribute *cattr = TupleDescCompactAttr(tupdesc, i);
+
+ /*
+ * Find the highest attnum which is guaranteed to exist in all tuples
+ * in the table. We currently only pay attention to byval attributes
+ * to allow additional optimizations during tuple deformation.
+ */
+ if (firstNonGuaranteedAttr == tupdesc->natts &&
+ (cattr->attnullability != ATTNULLABLE_VALID || !cattr->attbyval ||
+ cattr->atthasmissing || cattr->attisdropped || cattr->attlen <= 0))
+ firstNonGuaranteedAttr = i;
+
+ if (cattr->attlen <= 0)
+ break;
+
+ off = att_nominal_alignby(off, cattr->attalignby);
+
+ cattr->attcacheoff = off;
+
+ off += cattr->attlen;
+ firstNonCachedOffsetAttr = i + 1;
+ }
+
+ tupdesc->firstNonCachedOffsetAttr = firstNonCachedOffsetAttr;
+ tupdesc->firstNonGuaranteedAttr = firstNonGuaranteedAttr;
+}
+
/*
* Free a TupleDesc including all substructure
*/
diff --git a/src/backend/access/spgist/spgutils.c b/src/backend/access/spgist/spgutils.c
index b246e8127db..a4694bd8065 100644
--- a/src/backend/access/spgist/spgutils.c
+++ b/src/backend/access/spgist/spgutils.c
@@ -335,9 +335,6 @@ getSpGistTupleDesc(Relation index, SpGistTypeDesc *keyType)
/* We shouldn't need to bother with making these valid: */
att->attcompression = InvalidCompressionMethod;
att->attcollation = InvalidOid;
- /* In case we changed typlen, we'd better reset following offsets */
- for (int i = spgFirstIncludeColumn; i < outTupDesc->natts; i++)
- TupleDescCompactAttr(outTupDesc, i)->attcacheoff = -1;
populate_compact_attribute(outTupDesc, spgKeyColumn);
TupleDescFinalize(outTupDesc);
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index bfd3ebc601e..0b635486993 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -1944,7 +1944,7 @@ ExecPartitionCheckEmitError(ResultRelInfo *resultRelInfo,
*/
if (map != NULL)
slot = execute_attr_map_slot(map, slot,
- MakeTupleTableSlot(tupdesc, &TTSOpsVirtual));
+ MakeTupleTableSlot(tupdesc, &TTSOpsVirtual, 0));
modifiedCols = bms_union(ExecGetInsertedCols(rootrel, estate),
ExecGetUpdatedCols(rootrel, estate));
}
@@ -2060,7 +2060,7 @@ ExecConstraints(ResultRelInfo *resultRelInfo,
*/
if (map != NULL)
slot = execute_attr_map_slot(map, slot,
- MakeTupleTableSlot(tupdesc, &TTSOpsVirtual));
+ MakeTupleTableSlot(tupdesc, &TTSOpsVirtual, 0));
modifiedCols = bms_union(ExecGetInsertedCols(rootrel, estate),
ExecGetUpdatedCols(rootrel, estate));
rel = rootrel->ri_RelationDesc;
@@ -2196,7 +2196,7 @@ ReportNotNullViolationError(ResultRelInfo *resultRelInfo, TupleTableSlot *slot,
*/
if (map != NULL)
slot = execute_attr_map_slot(map, slot,
- MakeTupleTableSlot(tupdesc, &TTSOpsVirtual));
+ MakeTupleTableSlot(tupdesc, &TTSOpsVirtual, 0));
modifiedCols = bms_union(ExecGetInsertedCols(rootrel, estate),
ExecGetUpdatedCols(rootrel, estate));
rel = rootrel->ri_RelationDesc;
@@ -2304,7 +2304,7 @@ ExecWithCheckOptions(WCOKind kind, ResultRelInfo *resultRelInfo,
*/
if (map != NULL)
slot = execute_attr_map_slot(map, slot,
- MakeTupleTableSlot(tupdesc, &TTSOpsVirtual));
+ MakeTupleTableSlot(tupdesc, &TTSOpsVirtual, 0));
modifiedCols = bms_union(ExecGetInsertedCols(rootrel, estate),
ExecGetUpdatedCols(rootrel, estate));
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 07b248aa5f3..bfd2286ec2b 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -992,118 +992,6 @@ tts_buffer_heap_store_tuple(TupleTableSlot *slot, HeapTuple tuple,
}
}
-/*
- * slot_deform_heap_tuple_internal
- * An always inline helper function for use in slot_deform_heap_tuple to
- * allow the compiler to emit specialized versions of this function for
- * various combinations of "slow" and "hasnulls". For example, if a
- * given tuple has no nulls, then we needn't check "hasnulls" for every
- * attribute that we're deforming. The caller can just call this
- * function with hasnulls set to constant-false and have the compiler
- * remove the constant-false branches and emit more optimal code.
- *
- * Returns the next attnum to deform, which can be equal to natts when the
- * function manages to deform all requested attributes. *offp is an input and
- * output parameter which is the byte offset within the tuple to start deforming
- * from which, on return, gets set to the offset where the next attribute
- * should be deformed from. *slowp is set to true when subsequent deforming
- * of this tuple must use a version of this function with "slow" passed as
- * true.
- *
- * Callers cannot assume when we return "attnum" (i.e. all requested
- * attributes have been deformed) that slow mode isn't required for any
- * additional deforming as the final attribute may have caused a switch to
- * slow mode.
- */
-static pg_attribute_always_inline int
-slot_deform_heap_tuple_internal(TupleTableSlot *slot, HeapTuple tuple,
- int attnum, int natts, bool slow,
- bool hasnulls, uint32 *offp, bool *slowp)
-{
- TupleDesc tupleDesc = slot->tts_tupleDescriptor;
- Datum *values = slot->tts_values;
- bool *isnull = slot->tts_isnull;
- HeapTupleHeader tup = tuple->t_data;
- char *tp; /* ptr to tuple data */
- bits8 *bp = tup->t_bits; /* ptr to null bitmap in tuple */
- bool slownext = false;
-
- tp = (char *) tup + tup->t_hoff;
-
- for (; attnum < natts; attnum++)
- {
- CompactAttribute *thisatt = TupleDescCompactAttr(tupleDesc, attnum);
-
- if (hasnulls && att_isnull(attnum, bp))
- {
- values[attnum] = (Datum) 0;
- isnull[attnum] = true;
- if (!slow)
- {
- *slowp = true;
- return attnum + 1;
- }
- else
- continue;
- }
-
- isnull[attnum] = false;
-
- /* calculate the offset of this attribute */
- if (!slow && thisatt->attcacheoff >= 0)
- *offp = thisatt->attcacheoff;
- else if (thisatt->attlen == -1)
- {
- /*
- * We can only cache the offset for a varlena attribute if the
- * offset is already suitably aligned, so that there would be no
- * pad bytes in any case: then the offset will be valid for either
- * an aligned or unaligned value.
- */
- if (!slow && *offp == att_nominal_alignby(*offp, thisatt->attalignby))
- thisatt->attcacheoff = *offp;
- else
- {
- *offp = att_pointer_alignby(*offp,
- thisatt->attalignby,
- -1,
- tp + *offp);
-
- if (!slow)
- slownext = true;
- }
- }
- else
- {
- /* not varlena, so safe to use att_nominal_alignby */
- *offp = att_nominal_alignby(*offp, thisatt->attalignby);
-
- if (!slow)
- thisatt->attcacheoff = *offp;
- }
-
- values[attnum] = fetchatt(thisatt, tp + *offp);
-
- *offp = att_addlength_pointer(*offp, thisatt->attlen, tp + *offp);
-
- /* check if we need to switch to slow mode */
- if (!slow)
- {
- /*
- * We're unable to deform any further if the above code set
- * 'slownext', or if this isn't a fixed-width attribute.
- */
- if (slownext || thisatt->attlen <= 0)
- {
- *slowp = true;
- return attnum + 1;
- }
- }
- }
-
- return natts;
-}
-
/*
* slot_deform_heap_tuple
* Given a TupleTableSlot, extract data from the slot's physical tuple
@@ -1125,94 +1013,226 @@ static pg_attribute_always_inline void
slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int reqnatts)
{
- bool hasnulls = HeapTupleHasNulls(tuple);
- int attnum;
+ CompactAttribute *cattr;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+ HeapTupleHeader tup = tuple->t_data;
+ size_t attnum;
+ int firstNonCacheOffsetAttr;
+ int firstNonGuaranteedAttr;
+ int firstNullAttr;
int natts;
+ Datum *values;
+ bool *isnull;
+ char *tp; /* ptr to tuple data */
uint32 off; /* offset in tuple data */
- bool slow; /* can we use/set attcacheoff? */
- /* We can only fetch as many attributes as the tuple has. */
- natts = Min(HeapTupleHeaderGetNatts(tuple->t_data), reqnatts);
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
+
+ isnull = slot->tts_isnull;
/*
- * Check whether the first call for this tuple, and initialize or restore
- * loop state.
+ * Some callers may form and deform tuples prior to NOT NULL constraints
+ * being checked. Here we'd like to optimize the case where we only need
+ * to fetch attributes before or up to the point where the attribute is
+ * guaranteed to exist in the tuple. We rely on the slot flag being set
+ * correctly to only enable this optimization when it's valid to do so.
+ * This optimization allows us to save fetching the number of attributes
+ * from the tuple and saves the additional cost of handling non-byval
+ * attrs.
*/
+ firstNonGuaranteedAttr = Min(reqnatts, slot->tts_first_nonguaranteed);
+
+ firstNonCacheOffsetAttr = tupleDesc->firstNonCachedOffsetAttr;
+
+ if (HeapTupleHasNulls(tuple))
+ {
+ natts = HeapTupleHeaderGetNatts(tup);
+ tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits) +
+ BITMAPLEN(natts));
+
+ natts = Min(natts, reqnatts);
+ if (natts > firstNonGuaranteedAttr)
+ {
+ bits8 *bp = tup->t_bits;
+
+ /* Find the first NULL attr */
+ firstNullAttr = first_null_attr(bp, natts);
+
+ /*
+ * And populate the isnull array for all attributes being fetched
+ * from the tuple.
+ */
+ populate_isnull_array(bp, natts, isnull);
+ }
+ else
+ {
+ /* Otherwise all required columns are guaranteed to exist */
+ firstNullAttr = natts;
+ }
+ }
+ else
+ {
+ tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits));
+
+ /*
+ * We only need to look at the tuple's natts if we need more than the
+ * guaranteed number of columns
+ */
+ if (reqnatts > firstNonGuaranteedAttr)
+ natts = Min(HeapTupleHeaderGetNatts(tup), reqnatts);
+ else
+ {
+ /* No need to access the number of attributes in the tuple */
+ natts = reqnatts;
+ }
+
+ /* All attrs can be fetched without checking for NULLs */
+ firstNullAttr = natts;
+ }
+
attnum = slot->tts_nvalid;
+ values = slot->tts_values;
slot->tts_nvalid = reqnatts;
- if (attnum == 0)
+
+ /* Ensure we calculated tp correctly */
+ Assert(tp == (char *) tup + tup->t_hoff);
+
+ if (attnum < firstNonGuaranteedAttr)
{
- /* Start from the first attribute */
- off = 0;
- slow = false;
+ int attlen;
+
+ do
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ attlen = cattr->attlen;
+
+ /* We don't expect any non-byval types */
+ pg_assume(attlen > 0);
+ Assert(cattr->attbyval == true);
+
+ off = cattr->attcacheoff;
+ values[attnum] = fetch_att_noerr(tp + off, true, attlen);
+ attnum++;
+ } while (attnum < firstNonGuaranteedAttr);
+
+ off += attlen;
+
+ if (attnum == reqnatts)
+ goto done;
}
else
{
/* Restore state from previous execution */
off = *offp;
- slow = TTS_SLOW(slot);
+
+ /* We expect *offp to be set to 0 when attnum == 0 */
+ Assert(off == 0 || attnum > 0);
}
+ /* We can use attcacheoff up until the first NULL */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, firstNullAttr);
+
/*
- * If 'slow' isn't set, try deforming using deforming code that does not
- * contain any of the extra checks required for non-fixed offset
- * deforming. During deforming, if or when we find a NULL or a variable
- * length attribute, we'll switch to a deforming method which includes the
- * extra code required for non-fixed offset deforming, a.k.a slow mode.
- * Because this is performance critical, we inline
- * slot_deform_heap_tuple_internal passing the 'slow' and 'hasnull'
- * parameters as constants to allow the compiler to emit specialized code
- * with the known-const false comparisons and subsequent branches removed.
+ * Handle the portion of the tuple that we have cached the offset for up
+ * to the first NULL attribute. The offset is effectively fixed for
+ * these, so we can use the CompactAttribute's attcacheoff.
*/
- if (!slow)
+ if (attnum < firstNonCacheOffsetAttr)
{
- /* Tuple without any NULLs? We can skip doing any NULL checking */
- if (!hasnulls)
- attnum = slot_deform_heap_tuple_internal(slot,
- tuple,
- attnum,
- natts,
- false, /* slow */
- false, /* hasnulls */
- &off,
- &slow);
- else
- attnum = slot_deform_heap_tuple_internal(slot,
- tuple,
- attnum,
- natts,
- false, /* slow */
- true, /* hasnulls */
- &off,
- &slow);
+ int attlen;
+
+ do
+ {
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ attlen = cattr->attlen;
+
+ off = cattr->attcacheoff;
+ values[attnum] = fetch_att_noerr(tp + off,
+ cattr->attbyval,
+ attlen);
+ attnum++;
+ } while (attnum < firstNonCacheOffsetAttr);
+
+ /*
+ * Point the offset after the end of the last attribute with a cached
+ * offset. We expect the final cached offset attribute to have a
+ * fixed width, so just add the attlen to the attcacheoff
+ */
+ Assert(attlen > 0);
+ off += attlen;
}
- /* If there's still work to do then we must be in slow mode */
- if (attnum < natts)
+ /*
+ * Handle any portion of the tuple that doesn't have a fixed offset up
+ * until the first NULL attribute. This loop only differs from the one
+ * after it by the NULL checks.
+ */
+ for (; attnum < firstNullAttr; attnum++)
{
- /* XXX is it worth adding a separate call when hasnulls is false? */
- attnum = slot_deform_heap_tuple_internal(slot,
- tuple,
- attnum,
- natts,
- true, /* slow */
- hasnulls,
- &off,
- &slow);
+ int attlen;
+
+ isnull[attnum] = false;
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ attlen = cattr->attlen;
+
+ /*
+ * cstrings don't exist in heap tuples. Use pg_assume to instruct the
+ * compiler not to emit the cstring-related code in
+ * align_fetch_then_add().
+ */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ attlen,
+ cattr->attalignby);
}
/*
- * Save state for next execution
+ * Now handle any remaining attributes in the tuple up to the requested
+ * attnum. This time, include NULL checks as we're now at the first NULL
+ * attribute.
*/
- *offp = off;
- if (slow)
- slot->tts_flags |= TTS_FLAG_SLOW;
- else
- slot->tts_flags &= ~TTS_FLAG_SLOW;
+ for (; attnum < natts; attnum++)
+ {
+ int attlen;
+
+ if (isnull[attnum])
+ {
+ values[attnum] = (Datum) 0;
+ continue;
+ }
+
+ cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ attlen = cattr->attlen;
- /* Fetch any missing attrs and raise an error if reqnatts is invalid. */
+ /* As above, we don't expect cstrings */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ /* align 'off', fetch the datum, and increment off beyond the datum */
+ values[attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ attlen,
+ cattr->attalignby);
+ }
+
+ /* Fetch any missing attrs and raise an error if reqnatts is invalid */
if (unlikely(attnum < reqnatts))
+ {
+ *offp = off;
slot_getmissingattrs(slot, attnum, reqnatts);
+ return;
+ }
+done:
+
+ /* Save current offset for next execution */
+ *offp = off;
}
const TupleTableSlotOps TTSOpsVirtual = {
@@ -1307,7 +1327,7 @@ const TupleTableSlotOps TTSOpsBufferHeapTuple = {
*/
TupleTableSlot *
MakeTupleTableSlot(TupleDesc tupleDesc,
- const TupleTableSlotOps *tts_ops)
+ const TupleTableSlotOps *tts_ops, uint16 flags)
{
Size basesz,
allocsz;
@@ -1331,6 +1351,7 @@ MakeTupleTableSlot(TupleDesc tupleDesc,
*((const TupleTableSlotOps **) &slot->tts_ops) = tts_ops;
slot->type = T_TupleTableSlot;
slot->tts_flags |= TTS_FLAG_EMPTY;
+ slot->tts_flags |= flags;
if (tupleDesc != NULL)
slot->tts_flags |= TTS_FLAG_FIXED;
slot->tts_tupleDescriptor = tupleDesc;
@@ -1342,12 +1363,31 @@ MakeTupleTableSlot(TupleDesc tupleDesc,
slot->tts_values = (Datum *)
(((char *) slot)
+ MAXALIGN(basesz));
+
+ /*
+ * We round the size of tts_isnull up to the next highest multiple of
+ * 8. This is needed as populate_isnull_array() operates on 8
+ * elements at a time when converting a tuple's NULL bitmap into a
+ * boolean array.
+ */
slot->tts_isnull = (bool *)
(((char *) slot)
+ MAXALIGN(basesz)
- + MAXALIGN(tupleDesc->natts * sizeof(Datum)));
+ + TYPEALIGN(8, tupleDesc->natts * sizeof(Datum)));
PinTupleDesc(tupleDesc);
+
+ /*
+ * Precalculate the maximum guaranteed attribute that has to exist in
+ * every tuple which gets deformed into this slot. When the
+ * TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS flag is enabled, we simply take
+ * the precalculated value from the tupleDesc, otherwise the
+ * optimization is disabled, and we set the value to 0.
+ */
+ if ((flags & TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS) != 0)
+ slot->tts_first_nonguaranteed = tupleDesc->firstNonGuaranteedAttr;
+ else
+ slot->tts_first_nonguaranteed = 0;
}
/*
@@ -1366,9 +1406,9 @@ MakeTupleTableSlot(TupleDesc tupleDesc,
*/
TupleTableSlot *
ExecAllocTableSlot(List **tupleTable, TupleDesc desc,
- const TupleTableSlotOps *tts_ops)
+ const TupleTableSlotOps *tts_ops, uint16 flags)
{
- TupleTableSlot *slot = MakeTupleTableSlot(desc, tts_ops);
+ TupleTableSlot *slot = MakeTupleTableSlot(desc, tts_ops, flags);
*tupleTable = lappend(*tupleTable, slot);
@@ -1435,7 +1475,7 @@ TupleTableSlot *
MakeSingleTupleTableSlot(TupleDesc tupdesc,
const TupleTableSlotOps *tts_ops)
{
- TupleTableSlot *slot = MakeTupleTableSlot(tupdesc, tts_ops);
+ TupleTableSlot *slot = MakeTupleTableSlot(tupdesc, tts_ops, 0);
return slot;
}
@@ -1515,8 +1555,14 @@ ExecSetSlotDescriptor(TupleTableSlot *slot, /* slot to change */
*/
slot->tts_values = (Datum *)
MemoryContextAlloc(slot->tts_mcxt, tupdesc->natts * sizeof(Datum));
+
+ /*
+ * We round the size of tts_isnull up to the next highest multiple of 8.
+ * This is needed as populate_isnull_array() operates on 8 elements at a
+ * time when converting a tuple's NULL bitmap into a boolean array.
+ */
slot->tts_isnull = (bool *)
- MemoryContextAlloc(slot->tts_mcxt, tupdesc->natts * sizeof(bool));
+ MemoryContextAlloc(slot->tts_mcxt, TYPEALIGN(8, tupdesc->natts * sizeof(bool)));
}
/* --------------------------------
@@ -1978,7 +2024,7 @@ ExecInitResultSlot(PlanState *planstate, const TupleTableSlotOps *tts_ops)
TupleTableSlot *slot;
slot = ExecAllocTableSlot(&planstate->state->es_tupleTable,
- planstate->ps_ResultTupleDesc, tts_ops);
+ planstate->ps_ResultTupleDesc, tts_ops, 0);
planstate->ps_ResultTupleSlot = slot;
planstate->resultopsfixed = planstate->ps_ResultTupleDesc != NULL;
@@ -2006,10 +2052,11 @@ ExecInitResultTupleSlotTL(PlanState *planstate,
*/
void
ExecInitScanTupleSlot(EState *estate, ScanState *scanstate,
- TupleDesc tupledesc, const TupleTableSlotOps *tts_ops)
+ TupleDesc tupledesc, const TupleTableSlotOps *tts_ops,
+ uint16 flags)
{
scanstate->ss_ScanTupleSlot = ExecAllocTableSlot(&estate->es_tupleTable,
- tupledesc, tts_ops);
+ tupledesc, tts_ops, flags);
scanstate->ps.scandesc = tupledesc;
scanstate->ps.scanopsfixed = tupledesc != NULL;
scanstate->ps.scanops = tts_ops;
@@ -2029,7 +2076,7 @@ ExecInitExtraTupleSlot(EState *estate,
TupleDesc tupledesc,
const TupleTableSlotOps *tts_ops)
{
- return ExecAllocTableSlot(&estate->es_tupleTable, tupledesc, tts_ops);
+ return ExecAllocTableSlot(&estate->es_tupleTable, tupledesc, tts_ops, 0);
}
/* ----------------
@@ -2261,10 +2308,16 @@ ExecTypeSetColNames(TupleDesc typeInfo, List *namesList)
* This happens "for free" if the tupdesc came from a relcache entry, but
* not if we have manufactured a tupdesc for a transient RECORD datatype.
* In that case we have to notify typcache.c of the existence of the type.
+ *
+ * TupleDescFinalize() must be called on the TupleDesc before calling this
+ * function.
*/
TupleDesc
BlessTupleDesc(TupleDesc tupdesc)
{
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupdesc->firstNonCachedOffsetAttr >= 0);
+
if (tupdesc->tdtypeid == RECORDOID &&
tupdesc->tdtypmod < 0)
assign_record_type_typmod(tupdesc);
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index a7955e476f9..f62582859f9 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -711,7 +711,7 @@ ExecCreateScanSlotFromOuterPlan(EState *estate,
outerPlan = outerPlanState(scanstate);
tupDesc = ExecGetResultType(outerPlan);
- ExecInitScanTupleSlot(estate, scanstate, tupDesc, tts_ops);
+ ExecInitScanTupleSlot(estate, scanstate, tupDesc, tts_ops, 0);
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
index 7d487a165fa..c5c321b4f42 100644
--- a/src/backend/executor/nodeAgg.c
+++ b/src/backend/executor/nodeAgg.c
@@ -1682,7 +1682,7 @@ find_hash_columns(AggState *aggstate)
&perhash->hashfunctions);
perhash->hashslot =
ExecAllocTableSlot(&estate->es_tupleTable, hashDesc,
- &TTSOpsMinimalTuple);
+ &TTSOpsMinimalTuple, 0);
list_free(hashTlist);
bms_free(colnos);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index e0b6df64767..6f29954e84f 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -382,7 +382,10 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
*/
ExecInitScanTupleSlot(estate, &scanstate->ss,
RelationGetDescr(currentRelation),
- table_slot_callbacks(currentRelation));
+ table_slot_callbacks(currentRelation), 0);
+
+ scanstate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeCtescan.c b/src/backend/executor/nodeCtescan.c
index e6e476388e5..45b09d93b93 100644
--- a/src/backend/executor/nodeCtescan.c
+++ b/src/backend/executor/nodeCtescan.c
@@ -261,7 +261,7 @@ ExecInitCteScan(CteScan *node, EState *estate, int eflags)
*/
ExecInitScanTupleSlot(estate, &scanstate->ss,
ExecGetResultType(scanstate->cteplanstate),
- &TTSOpsMinimalTuple);
+ &TTSOpsMinimalTuple, 0);
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeCustom.c b/src/backend/executor/nodeCustom.c
index a9ad5af6a98..b7cc890cd20 100644
--- a/src/backend/executor/nodeCustom.c
+++ b/src/backend/executor/nodeCustom.c
@@ -79,14 +79,14 @@ ExecInitCustomScan(CustomScan *cscan, EState *estate, int eflags)
TupleDesc scan_tupdesc;
scan_tupdesc = ExecTypeFromTL(cscan->custom_scan_tlist);
- ExecInitScanTupleSlot(estate, &css->ss, scan_tupdesc, slotOps);
+ ExecInitScanTupleSlot(estate, &css->ss, scan_tupdesc, slotOps, 0);
/* Node's targetlist will contain Vars with varno = INDEX_VAR */
tlistvarno = INDEX_VAR;
}
else
{
ExecInitScanTupleSlot(estate, &css->ss, RelationGetDescr(scan_rel),
- slotOps);
+ slotOps, 0);
/* Node's targetlist will contain Vars with varno = scanrelid */
tlistvarno = scanrelid;
}
diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c
index 8721b67b7cc..6f0daddce07 100644
--- a/src/backend/executor/nodeForeignscan.c
+++ b/src/backend/executor/nodeForeignscan.c
@@ -191,7 +191,7 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
scan_tupdesc = ExecTypeFromTL(node->fdw_scan_tlist);
ExecInitScanTupleSlot(estate, &scanstate->ss, scan_tupdesc,
- &TTSOpsHeapTuple);
+ &TTSOpsHeapTuple, 0);
/* Node's targetlist will contain Vars with varno = INDEX_VAR */
tlistvarno = INDEX_VAR;
}
@@ -202,7 +202,7 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
/* don't trust FDWs to return tuples fulfilling NOT NULL constraints */
scan_tupdesc = CreateTupleDescCopy(RelationGetDescr(currentRelation));
ExecInitScanTupleSlot(estate, &scanstate->ss, scan_tupdesc,
- &TTSOpsHeapTuple);
+ &TTSOpsHeapTuple, 0);
/* Node's targetlist will contain Vars with varno = scanrelid */
tlistvarno = scanrelid;
}
diff --git a/src/backend/executor/nodeFunctionscan.c b/src/backend/executor/nodeFunctionscan.c
index feb82d64967..222741adf3b 100644
--- a/src/backend/executor/nodeFunctionscan.c
+++ b/src/backend/executor/nodeFunctionscan.c
@@ -494,7 +494,7 @@ ExecInitFunctionScan(FunctionScan *node, EState *estate, int eflags)
* Initialize scan slot and type.
*/
ExecInitScanTupleSlot(estate, &scanstate->ss, scan_tupdesc,
- &TTSOpsMinimalTuple);
+ &TTSOpsMinimalTuple, 0);
/*
* Initialize result slot, type and projection.
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index c2d09374517..144a57fde95 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -567,7 +567,10 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
*/
tupDesc = ExecTypeFromTL(node->indextlist);
ExecInitScanTupleSlot(estate, &indexstate->ss, tupDesc,
- &TTSOpsVirtual);
+ &TTSOpsVirtual, 0);
+
+ indexstate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
/*
* We need another slot, in a format that's suitable for the table AM, for
@@ -576,7 +579,7 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
indexstate->ioss_TableSlot =
ExecAllocTableSlot(&estate->es_tupleTable,
RelationGetDescr(currentRelation),
- table_slot_callbacks(currentRelation));
+ table_slot_callbacks(currentRelation), 0);
/*
* Initialize result type and projection info. The node's targetlist will
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index a616abff04c..e7bebb89517 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -938,7 +938,10 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
*/
ExecInitScanTupleSlot(estate, &indexstate->ss,
RelationGetDescr(currentRelation),
- table_slot_callbacks(currentRelation));
+ table_slot_callbacks(currentRelation), 0);
+
+ indexstate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeNamedtuplestorescan.c b/src/backend/executor/nodeNamedtuplestorescan.c
index fdfccc8169f..29d862a4001 100644
--- a/src/backend/executor/nodeNamedtuplestorescan.c
+++ b/src/backend/executor/nodeNamedtuplestorescan.c
@@ -137,7 +137,7 @@ ExecInitNamedTuplestoreScan(NamedTuplestoreScan *node, EState *estate, int eflag
* The scan tuple type is specified for the tuplestore.
*/
ExecInitScanTupleSlot(estate, &scanstate->ss, scanstate->tupdesc,
- &TTSOpsMinimalTuple);
+ &TTSOpsMinimalTuple, 0);
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c
index 1b0af70fd7a..a1da05ecc18 100644
--- a/src/backend/executor/nodeSamplescan.c
+++ b/src/backend/executor/nodeSamplescan.c
@@ -128,7 +128,11 @@ ExecInitSampleScan(SampleScan *node, EState *estate, int eflags)
/* and create slot with appropriate rowtype */
ExecInitScanTupleSlot(estate, &scanstate->ss,
RelationGetDescr(scanstate->ss.ss_currentRelation),
- table_slot_callbacks(scanstate->ss.ss_currentRelation));
+ table_slot_callbacks(scanstate->ss.ss_currentRelation),
+ 0);
+
+ scanstate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index af3c788ce8b..8f219f60a93 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -244,7 +244,8 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags)
/* and create slot with the appropriate rowtype */
ExecInitScanTupleSlot(estate, &scanstate->ss,
RelationGetDescr(scanstate->ss.ss_currentRelation),
- table_slot_callbacks(scanstate->ss.ss_currentRelation));
+ table_slot_callbacks(scanstate->ss.ss_currentRelation),
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS);
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeSubqueryscan.c b/src/backend/executor/nodeSubqueryscan.c
index 4fd6f6fb4a5..70914e8189c 100644
--- a/src/backend/executor/nodeSubqueryscan.c
+++ b/src/backend/executor/nodeSubqueryscan.c
@@ -130,7 +130,8 @@ ExecInitSubqueryScan(SubqueryScan *node, EState *estate, int eflags)
*/
ExecInitScanTupleSlot(estate, &subquerystate->ss,
ExecGetResultType(subquerystate->subplan),
- ExecGetResultSlotOps(subquerystate->subplan, NULL));
+ ExecGetResultSlotOps(subquerystate->subplan, NULL),
+ 0);
/*
* The slot used as the scantuple isn't the slot above (outside of EPQ),
diff --git a/src/backend/executor/nodeTableFuncscan.c b/src/backend/executor/nodeTableFuncscan.c
index 52070d147a4..769b9766542 100644
--- a/src/backend/executor/nodeTableFuncscan.c
+++ b/src/backend/executor/nodeTableFuncscan.c
@@ -148,7 +148,7 @@ ExecInitTableFuncScan(TableFuncScan *node, EState *estate, int eflags)
tf->colcollations);
/* and the corresponding scan slot */
ExecInitScanTupleSlot(estate, &scanstate->ss, tupdesc,
- &TTSOpsMinimalTuple);
+ &TTSOpsMinimalTuple, 0);
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index 503817da65b..decc3167ba7 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -394,7 +394,10 @@ ExecInitTidRangeScan(TidRangeScan *node, EState *estate, int eflags)
*/
ExecInitScanTupleSlot(estate, &tidrangestate->ss,
RelationGetDescr(currentRelation),
- table_slot_callbacks(currentRelation));
+ table_slot_callbacks(currentRelation), 0);
+
+ tidrangestate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeTidscan.c b/src/backend/executor/nodeTidscan.c
index 4eddb0828b5..26593b930af 100644
--- a/src/backend/executor/nodeTidscan.c
+++ b/src/backend/executor/nodeTidscan.c
@@ -536,7 +536,10 @@ ExecInitTidScan(TidScan *node, EState *estate, int eflags)
*/
ExecInitScanTupleSlot(estate, &tidstate->ss,
RelationGetDescr(currentRelation),
- table_slot_callbacks(currentRelation));
+ table_slot_callbacks(currentRelation), 0);
+
+ tidstate->ss.ss_ScanTupleSlot->tts_flags |=
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS;
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeValuesscan.c b/src/backend/executor/nodeValuesscan.c
index e663fb68cfc..effc896ea1c 100644
--- a/src/backend/executor/nodeValuesscan.c
+++ b/src/backend/executor/nodeValuesscan.c
@@ -247,7 +247,7 @@ ExecInitValuesScan(ValuesScan *node, EState *estate, int eflags)
* Get info about values list, initialize scan slot with it.
*/
tupdesc = ExecTypeFromExprList((List *) linitial(node->values_lists));
- ExecInitScanTupleSlot(estate, &scanstate->ss, tupdesc, &TTSOpsVirtual);
+ ExecInitScanTupleSlot(estate, &scanstate->ss, tupdesc, &TTSOpsVirtual, 0);
/*
* Initialize result type and projection.
diff --git a/src/backend/executor/nodeWorktablescan.c b/src/backend/executor/nodeWorktablescan.c
index 210cc44f911..66e904c7636 100644
--- a/src/backend/executor/nodeWorktablescan.c
+++ b/src/backend/executor/nodeWorktablescan.c
@@ -165,7 +165,7 @@ ExecInitWorkTableScan(WorkTableScan *node, EState *estate, int eflags)
scanstate->ss.ps.resultopsset = true;
scanstate->ss.ps.resultopsfixed = false;
- ExecInitScanTupleSlot(estate, &scanstate->ss, NULL, &TTSOpsMinimalTuple);
+ ExecInitScanTupleSlot(estate, &scanstate->ss, NULL, &TTSOpsMinimalTuple, 0);
/*
* initialize child expressions
diff --git a/src/backend/jit/llvm/llvmjit_deform.c b/src/backend/jit/llvm/llvmjit_deform.c
index 3eb087eb56b..12521e3e46a 100644
--- a/src/backend/jit/llvm/llvmjit_deform.c
+++ b/src/backend/jit/llvm/llvmjit_deform.c
@@ -62,7 +62,6 @@ slot_compile_deform(LLVMJitContext *context, TupleDesc desc,
LLVMValueRef v_tts_values;
LLVMValueRef v_tts_nulls;
LLVMValueRef v_slotoffp;
- LLVMValueRef v_flagsp;
LLVMValueRef v_nvalidp;
LLVMValueRef v_nvalid;
LLVMValueRef v_maxatt;
@@ -178,7 +177,6 @@ slot_compile_deform(LLVMJitContext *context, TupleDesc desc,
v_tts_nulls =
l_load_struct_gep(b, StructTupleTableSlot, v_slot, FIELDNO_TUPLETABLESLOT_ISNULL,
"tts_ISNULL");
- v_flagsp = l_struct_gep(b, StructTupleTableSlot, v_slot, FIELDNO_TUPLETABLESLOT_FLAGS, "");
v_nvalidp = l_struct_gep(b, StructTupleTableSlot, v_slot, FIELDNO_TUPLETABLESLOT_NVALID, "");
if (ops == &TTSOpsHeapTuple || ops == &TTSOpsBufferHeapTuple)
@@ -747,14 +745,10 @@ slot_compile_deform(LLVMJitContext *context, TupleDesc desc,
{
LLVMValueRef v_off = l_load(b, TypeSizeT, v_offp, "");
- LLVMValueRef v_flags;
LLVMBuildStore(b, l_int16_const(lc, natts), v_nvalidp);
v_off = LLVMBuildTrunc(b, v_off, LLVMInt32TypeInContext(lc), "");
LLVMBuildStore(b, v_off, v_slotoffp);
- v_flags = l_load(b, LLVMInt16TypeInContext(lc), v_flagsp, "tts_flags");
- v_flags = LLVMBuildOr(b, v_flags, l_int16_const(lc, TTS_FLAG_SLOW), "");
- LLVMBuildStore(b, v_flags, v_flagsp);
LLVMBuildRetVoid(b);
}
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 857ebf7d6fb..4ecfcbff7ab 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -1559,7 +1559,7 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
if (relentry->attrmap)
{
TupleTableSlot *slot = MakeTupleTableSlot(RelationGetDescr(targetrel),
- &TTSOpsVirtual);
+ &TTSOpsVirtual, 0);
old_slot = execute_attr_map_slot(relentry->attrmap, old_slot, slot);
}
@@ -1574,7 +1574,7 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
if (relentry->attrmap)
{
TupleTableSlot *slot = MakeTupleTableSlot(RelationGetDescr(targetrel),
- &TTSOpsVirtual);
+ &TTSOpsVirtual, 0);
new_slot = execute_attr_map_slot(relentry->attrmap, new_slot, slot);
}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index d27ac216e6d..597de687b45 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -666,14 +666,6 @@ RelationBuildTupleDesc(Relation relation)
elog(ERROR, "pg_attribute catalog is missing %d attribute(s) for relation OID %u",
need, RelationGetRelid(relation));
- /*
- * We can easily set the attcacheoff value for the first attribute: it
- * must be zero. This eliminates the need for special cases for attnum=1
- * that used to exist in fastgetattr() and index_getattr().
- */
- if (RelationGetNumberOfAttributes(relation) > 0)
- TupleDescCompactAttr(relation->rd_att, 0)->attcacheoff = 0;
-
/*
* Set up constraint/default info
*/
@@ -1985,8 +1977,6 @@ formrdesc(const char *relationName, Oid relationReltype,
populate_compact_attribute(relation->rd_att, i);
}
- /* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
- TupleDescCompactAttr(relation->rd_att, 0)->attcacheoff = 0;
TupleDescFinalize(relation->rd_att);
/* mark not-null status */
@@ -4446,8 +4436,6 @@ BuildHardcodedDescriptor(int natts, const FormData_pg_attribute *attrs)
populate_compact_attribute(result, i);
}
- /* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc */
- TupleDescCompactAttr(result, 0)->attcacheoff = 0;
TupleDescFinalize(result);
/* Note: we don't bother to set up a TupleConstr entry */
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 595413dbbc5..ad7bc013812 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -131,6 +131,19 @@ typedef struct CompactAttribute
* Any code making changes manually to and fields in the FormData_pg_attribute
* array must subsequently call populate_compact_attribute() to flush the
* changes out to the corresponding 'compact_attrs' element.
+ *
+ * firstNonCachedOffsetAttr stores the index into the compact_attrs array for
+ * the first attribute that we don't have a known attcacheoff for.
+ *
+ * firstNonGuaranteedAttr stores the index to into the compact_attrs array for
+ * the first attribute that is either NULLable, missing, or !attbyval. This
+ * can be used in locations as a guarantee that attributes before this will
+ * always exist in tuples. The !attbyval part isn't required for this, but
+ * including this allows various tuple deforming routines to forego any checks
+ * for !attbyval.
+ *
+ * Once a TupleDesc has been populated, before it is used for any purpose,
+ * TupleDescFinalize() must be called on it.
*/
typedef struct TupleDescData
{
@@ -138,6 +151,11 @@ typedef struct TupleDescData
Oid tdtypeid; /* composite type ID for tuple type */
int32 tdtypmod; /* typmod for tuple type */
int tdrefcount; /* reference count, or -1 if not counting */
+ int firstNonCachedOffsetAttr; /* index of the first att without
+ * an attcacheoff */
+ int firstNonGuaranteedAttr; /* index of the first nullable,
+ * missing, dropped, or !attbyval
+ * attribute. */
TupleConstr *constr; /* constraints, or NULL if none */
/* compact_attrs[N] is the compact metadata of Attribute Number N+1 */
CompactAttribute compact_attrs[FLEXIBLE_ARRAY_MEMBER];
@@ -195,7 +213,6 @@ extern TupleDesc CreateTupleDescTruncatedCopy(TupleDesc tupdesc, int natts);
extern TupleDesc CreateTupleDescCopyConstr(TupleDesc tupdesc);
-#define TupleDescFinalize(d) ((void) 0)
#define TupleDescSize(src) \
(offsetof(struct TupleDescData, compact_attrs) + \
(src)->natts * sizeof(CompactAttribute) + \
@@ -206,6 +223,7 @@ extern void TupleDescCopy(TupleDesc dst, TupleDesc src);
extern void TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
TupleDesc src, AttrNumber srcAttno);
+extern void TupleDescFinalize(TupleDesc tupdesc);
extern void FreeTupleDesc(TupleDesc tupdesc);
extern void IncrTupleDescRefCount(TupleDesc tupdesc);
diff --git a/src/include/access/tupmacs.h b/src/include/access/tupmacs.h
index d64c18b950b..87dbeb76618 100644
--- a/src/include/access/tupmacs.h
+++ b/src/include/access/tupmacs.h
@@ -15,7 +15,9 @@
#define TUPMACS_H
#include "catalog/pg_type_d.h" /* for TYPALIGN macros */
-
+#include "port/pg_bitutils.h"
+#include "port/pg_bswap.h"
+#include "varatt.h"
/*
* Check a tuple's null bitmap to determine whether the attribute is null.
@@ -28,6 +30,62 @@ att_isnull(int ATT, const bits8 *BITS)
return !(BITS[ATT >> 3] & (1 << (ATT & 0x07)));
}
+/*
+ * populate_isnull_array
+ * Transform a tuple's null bitmap into a boolean array.
+ *
+ * Caller must ensure that the isnull array is sized so it contains
+ * at least as many elements as there are bits in the 'bits' array.
+ * Callers should be aware that isnull is populated 8 elements at a time,
+ * effectively as if natts is rounded up to the next multiple of 8.
+ */
+static inline void
+populate_isnull_array(const bits8 *bits, int natts, bool *isnull)
+{
+ int nbytes = (natts + 7) >> 3;
+
+ /*
+ * Multiplying the inverted NULL bitmap byte by this value results in the
+ * lowest bit in each byte being set the same as each bit of the inverted
+ * byte. We perform this as 2 32-bit operations rather than a single
+ * 64-bit operation as multiplying by the required value to do this in
+ * 64-bits would result in overflowing a uint64 in some cases.
+ *
+ * XXX if we ever require BMI2 (-march=x86-64-v3), then this could be done
+ * more efficiently on most X86-64 CPUs with the PDEP instruction. Beware
+ * that some chips (e.g. AMD's Zen2) are horribly inefficient at PDEP.
+ */
+#define SPREAD_BITS_MULTIPLIER_32 0x204081U
+
+ for (int i = 0; i < nbytes; i++, isnull += 8)
+ {
+ uint64 isnull_8;
+ bits8 nullbyte = ~bits[i];
+
+ /* Convert the lower 4 bits of NULL bitmap word into a 64 bit int */
+ isnull_8 = (nullbyte & 0xf) * SPREAD_BITS_MULTIPLIER_32;
+
+ /*
+ * Convert the upper 4 bits of null bitmap word into a 64 bit int,
+ * shift into the upper 32 bit and bitwise-OR with the result of the
+ * lower 4 bits.
+ */
+ isnull_8 |= ((uint64) ((nullbyte >> 4) * SPREAD_BITS_MULTIPLIER_32)) << 32;
+
+ /* Mask out all other bits apart from the lowest bit of each byte. */
+ isnull_8 &= UINT64CONST(0x0101010101010101);
+
+#ifdef WORDS_BIGENDIAN
+
+ /*
+ * Fix byte order on big-endian machines before copying to the array.
+ */
+ isnull_8 = pg_bswap64(isnull_8);
+#endif
+ memcpy(isnull, &isnull_8, sizeof(uint64));
+ }
+}
+
#ifndef FRONTEND
/*
* Given an attbyval and an attlen from either a Form_pg_attribute or
@@ -69,6 +127,170 @@ fetch_att(const void *T, bool attbyval, int attlen)
else
return PointerGetDatum(T);
}
+
+/*
+ * Same, but no error checking for invalid attlens for byval types. This
+ * is safe to use when attlen comes from CompactAttribute as we validate the
+ * length when populating that struct.
+ */
+static inline Datum
+fetch_att_noerr(const void *T, bool attbyval, int attlen)
+{
+ if (attbyval)
+ {
+ switch (attlen)
+ {
+ case sizeof(int32):
+ return Int32GetDatum(*((const int32 *) T));
+ case sizeof(int16):
+ return Int16GetDatum(*((const int16 *) T));
+ case sizeof(char):
+ return CharGetDatum(*((const char *) T));
+ default:
+ Assert(attlen == sizeof(int64));
+ return Int64GetDatum(*((const int64 *) T));
+ }
+ }
+ else
+ return PointerGetDatum(T);
+}
+
+
+/*
+ * align_fetch_then_add
+ * Applies all the functionality of att_pointer_alignby(),
+ * fetch_att_noerr() and att_addlength_pointer(), resulting in the *off
+ * pointer to the perhaps unaligned number of bytes into 'tupptr', ready
+ * to deform the next attribute.
+ *
+ * tupptr: pointer to the beginning of the tuple, after the header and any
+ * NULL bitmask.
+ * off: offset in bytes for reading tuple data, possibly unaligned.
+ * attbyval, attlen and attalignby are values from CompactAttribute.
+ */
+static inline Datum
+align_fetch_then_add(const char *tupptr, uint32 *off, bool attbyval, int attlen,
+ uint8 attalignby)
+{
+ Datum res;
+
+ if (attlen > 0)
+ {
+ const char *offset_ptr;
+
+ *off = TYPEALIGN(attalignby, *off);
+ offset_ptr = tupptr + *off;
+ *off += attlen;
+ if (attbyval)
+ {
+ switch (attlen)
+ {
+ case sizeof(char):
+ return CharGetDatum(*((const char *) offset_ptr));
+ case sizeof(int16):
+ return Int16GetDatum(*((const int16 *) offset_ptr));
+ case sizeof(int32):
+ return Int32GetDatum(*((const int32 *) offset_ptr));
+ default:
+
+ /*
+ * populate_compact_attribute_internal() should have
+ * checked
+ */
+ Assert(attlen == sizeof(int64));
+ return Int64GetDatum(*((const int64 *) offset_ptr));
+ }
+ }
+ return PointerGetDatum(offset_ptr);
+ }
+ else if (attlen == -1)
+ {
+ if (!VARATT_IS_SHORT(tupptr + *off))
+ *off = TYPEALIGN(attalignby, *off);
+
+ res = PointerGetDatum(tupptr + *off);
+ *off += VARSIZE_ANY(DatumGetPointer(res));
+ return res;
+ }
+ else
+ {
+ Assert(attlen == -2);
+ *off = TYPEALIGN(attalignby, *off);
+ res = PointerGetDatum(tupptr + *off);
+ *off += strlen(tupptr + *off) + 1;
+ return res;
+ }
+}
+
+/*
+ * first_null_attr
+ * Inspect a NULL bitmap from a tuple and return the 0-based attnum of the
+ * first NULL attribute. Returns natts if no NULLs were found.
+ *
+ * This is coded to expect that 'bits' contains at least one 0 bit somewhere
+ * in the array, but not necessarily < natts. Note that natts may be passed
+ * as a value lower than the number of bits physically stored in the tuple's
+ * NULL bitmap, in which case we may not find a NULL and return natts.
+ *
+ * The reason we require at least one 0 bit somewhere in the NULL bitmap is
+ * that the for loop that checks 0xFF bytes would loop to the last byte in
+ * the array if all bytes were 0xFF, and the subsequent code that finds the
+ * right-most 0 bit would access the first byte beyond the bitmap. Provided
+ * we find a 0 bit before then, that won't happen. Since tuples which have no
+ * NULLs don't have a NULL bitmap, this function won't get called for that
+ * case.
+ */
+static inline int
+first_null_attr(const bits8 *bits, int natts)
+{
+ int nattByte = natts >> 3;
+ int bytenum;
+ int res;
+
+#ifdef USE_ASSERT_CHECKING
+ int firstnull_check = natts;
+
+ /* Do it the slow way and check we get the same answer. */
+ for (int i = 0; i < natts; i++)
+ {
+ if (att_isnull(i, bits))
+ {
+ firstnull_check = i;
+ break;
+ }
+ }
+#endif
+
+ /* Process all bytes up to just before the byte for the natts attribute */
+ for (bytenum = 0; bytenum < nattByte; bytenum++)
+ {
+ /* break if there's any NULL attrs (a 0 bit) */
+ if (bits[bytenum] != 0xFF)
+ break;
+ }
+
+ /*
+ * Look for the highest 0-bit in the 'bytenum' element. To do this, we
+ * promote the uint8 to uint32 before performing the bitwise NOT and
+ * looking for the first 1-bit. This works even when the byte is 0xFF, as
+ * the bitwise NOT of 0xFF in 32 bits is 0xFFFFFF00, in which case
+ * pg_rightmost_one_pos32() will return 8. We may end up with a value
+ * higher than natts here, but we'll fix that with the Min() below.
+ */
+ res = bytenum << 3;
+ res += pg_rightmost_one_pos32(~((uint32) bits[bytenum]));
+
+ /*
+ * Since we did no masking to mask out bits beyond the natt'th bit, we may
+ * have found a bit higher than natts, so we must cap res to natts
+ */
+ res = Min(res, natts);
+
+ /* Ensure we got the same answer as the att_isnull() loop got */
+ Assert(res == firstnull_check);
+
+ return res;
+}
#endif /* FRONTEND */
/*
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 82c442d23f8..b1820653506 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -597,7 +597,8 @@ extern void ExecInitResultTupleSlotTL(PlanState *planstate,
const TupleTableSlotOps *tts_ops);
extern void ExecInitScanTupleSlot(EState *estate, ScanState *scanstate,
TupleDesc tupledesc,
- const TupleTableSlotOps *tts_ops);
+ const TupleTableSlotOps *tts_ops,
+ uint16 flags);
extern TupleTableSlot *ExecInitExtraTupleSlot(EState *estate,
TupleDesc tupledesc,
const TupleTableSlotOps *tts_ops);
diff --git a/src/include/executor/tuptable.h b/src/include/executor/tuptable.h
index 3b09abbf99f..78558098fa3 100644
--- a/src/include/executor/tuptable.h
+++ b/src/include/executor/tuptable.h
@@ -84,9 +84,6 @@
* tts_values/tts_isnull are allocated either when the slot is created (when
* the descriptor is provided), or when a descriptor is assigned to the slot;
* they are of length equal to the descriptor's natts.
- *
- * The TTS_FLAG_SLOW flag is saved state for
- * slot_deform_heap_tuple, and should not be touched by any other code.
*----------
*/
@@ -98,9 +95,13 @@
#define TTS_FLAG_SHOULDFREE (1 << 2)
#define TTS_SHOULDFREE(slot) (((slot)->tts_flags & TTS_FLAG_SHOULDFREE) != 0)
-/* saved state for slot_deform_heap_tuple */
-#define TTS_FLAG_SLOW (1 << 3)
-#define TTS_SLOW(slot) (((slot)->tts_flags & TTS_FLAG_SLOW) != 0)
+/*
+ * true = slot's formed tuple guaranteed to not have NULLs in NOT NULLable
+ * columns.
+ */
+#define TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS (1 << 3)
+#define TTS_OBEYS_NOT_NULL_CONSTRAINTS(slot) \
+ (((slot)->tts_flags & TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS) != 0)
/* fixed tuple descriptor */
#define TTS_FLAG_FIXED (1 << 4)
@@ -123,7 +124,14 @@ typedef struct TupleTableSlot
#define FIELDNO_TUPLETABLESLOT_VALUES 5
Datum *tts_values; /* current per-attribute values */
#define FIELDNO_TUPLETABLESLOT_ISNULL 6
- bool *tts_isnull; /* current per-attribute isnull flags */
+ bool *tts_isnull; /* current per-attribute isnull flags. Array
+ * size must always be rounded up to the next
+ * multiple of 8 elements. */
+ int tts_first_nonguaranteed; /* The value from the TupleDesc's
+ * firstNonGuaranteedAttr, or 0
+ * when tts_flags does not contain
+ * TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS */
+
MemoryContext tts_mcxt; /* slot itself is in this context */
ItemPointerData tts_tid; /* stored tuple's tid */
Oid tts_tableOid; /* table oid of tuple */
@@ -313,9 +321,11 @@ typedef struct MinimalTupleTableSlot
/* in executor/execTuples.c */
extern TupleTableSlot *MakeTupleTableSlot(TupleDesc tupleDesc,
- const TupleTableSlotOps *tts_ops);
+ const TupleTableSlotOps *tts_ops,
+ uint16 flags);
extern TupleTableSlot *ExecAllocTableSlot(List **tupleTable, TupleDesc desc,
- const TupleTableSlotOps *tts_ops);
+ const TupleTableSlotOps *tts_ops,
+ uint16 flags);
extern void ExecResetTupleTable(List *tupleTable, bool shouldFree);
extern TupleTableSlot *MakeSingleTupleTableSlot(TupleDesc tupdesc,
const TupleTableSlotOps *tts_ops);
diff --git a/src/test/modules/deform_bench/deform_bench.c b/src/test/modules/deform_bench/deform_bench.c
index 7838f639bef..4f104989297 100644
--- a/src/test/modules/deform_bench/deform_bench.c
+++ b/src/test/modules/deform_bench/deform_bench.c
@@ -48,7 +48,9 @@ deform_bench(PG_FUNCTION_ARGS)
errmsg("only heap AM is supported")));
tupdesc = RelationGetDescr(rel);
- slot = MakeTupleTableSlot(tupdesc, &TTSOpsBufferHeapTuple);
+ slot = MakeTupleTableSlot(tupdesc,
+ &TTSOpsBufferHeapTuple,
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS);
scan = table_beginscan_strat(rel, GetActiveSnapshot(), 0, NULL, true, false);
/*
--
2.51.0
[text/plain] v13-0005-Reduce-size-of-CompactAttribute-struct-to-8-byte.patch (5.6K, 6-v13-0005-Reduce-size-of-CompactAttribute-struct-to-8-byte.patch)
download | inline diff:
From a66fa85a01d4c03c570b66afe16ab9b49044b448 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Mon, 23 Feb 2026 09:39:37 +1300
Subject: [PATCH v13 5/6] Reduce size of CompactAttribute struct to 8 bytes
Previously, this was 16 bytes. With the use of some bitflags and by
reducing the attcacheoff field size to a 16-bit type, we can halve the
size of the struct.
It's unlikely that caching the offsets for offsets larger than what will
fit in a 16-bit int will help much as the tuple is very likely to have
some non-fixed-width types anyway, the offsets of which we cannot cache.
---
src/backend/access/common/tupdesc.c | 10 ++++++++++
src/backend/executor/execTuples.c | 17 ++++++++++++-----
src/include/access/tupdesc.h | 16 ++++++++--------
3 files changed, 30 insertions(+), 13 deletions(-)
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index c68561337d7..71461ba6096 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -530,6 +530,16 @@ TupleDescFinalize(TupleDesc tupdesc)
off = att_nominal_alignby(off, cattr->attalignby);
+ /*
+ * attcacheoff is an int16, so don't try to cache any offsets larger
+ * than will fit in that type. Any attributes which are offset more
+ * than 2^15 are likely due to variable-length attributes. Since we
+ * don't cache offsets for or beyond variable-length attributes, using
+ * an int16 rather than an int32 here is unlikely to cost us anything.
+ */
+ if (off > PG_INT16_MAX)
+ break;
+
cattr->attcacheoff = off;
off += cattr->attlen;
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index bfd2286ec2b..d24c07dca9f 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -1013,6 +1013,7 @@ static pg_attribute_always_inline void
slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int reqnatts)
{
+ CompactAttribute *cattrs;
CompactAttribute *cattr;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
HeapTupleHeader tup = tuple->t_data;
@@ -1095,6 +1096,13 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
values = slot->tts_values;
slot->tts_nvalid = reqnatts;
+ /*
+ * We store the tupleDesc's CompactAttribute array in 'cattrs' as gcc
+ * seems to be unwilling to optimize accessing the CompactAttribute
+ * element efficiently when accessing it via TupleDescCompactAttr().
+ */
+ cattrs = tupleDesc->compact_attrs;
+
/* Ensure we calculated tp correctly */
Assert(tp == (char *) tup + tup->t_hoff);
@@ -1105,7 +1113,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
do
{
isnull[attnum] = false;
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
attlen = cattr->attlen;
/* We don't expect any non-byval types */
@@ -1146,9 +1154,8 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
do
{
isnull[attnum] = false;
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
attlen = cattr->attlen;
-
off = cattr->attcacheoff;
values[attnum] = fetch_att_noerr(tp + off,
cattr->attbyval,
@@ -1175,7 +1182,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int attlen;
isnull[attnum] = false;
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
attlen = cattr->attlen;
/*
@@ -1208,7 +1215,7 @@ slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
continue;
}
- cattr = TupleDescCompactAttr(tupleDesc, attnum);
+ cattr = &cattrs[attnum];
attlen = cattr->attlen;
/* As above, we don't expect cstrings */
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index ad7bc013812..e98036b58bf 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -55,7 +55,7 @@ typedef struct TupleConstr
* directly after the FormData_pg_attribute struct is populated or
* altered in any way.
*
- * Currently, this struct is 16 bytes. Any code changes which enlarge this
+ * Currently, this struct is 8 bytes. Any code changes which enlarge this
* struct should be considered very carefully.
*
* Code which must access a TupleDesc's attribute data should always make use
@@ -67,17 +67,17 @@ typedef struct TupleConstr
*/
typedef struct CompactAttribute
{
- int32 attcacheoff; /* fixed offset into tuple, if known, or -1 */
+ int16 attcacheoff; /* fixed offset into tuple, if known, or -1 */
int16 attlen; /* attr len in bytes or -1 = varlen, -2 =
* cstring */
bool attbyval; /* as FormData_pg_attribute.attbyval */
- bool attispackable; /* FormData_pg_attribute.attstorage !=
- * TYPSTORAGE_PLAIN */
- bool atthasmissing; /* as FormData_pg_attribute.atthasmissing */
- bool attisdropped; /* as FormData_pg_attribute.attisdropped */
- bool attgenerated; /* FormData_pg_attribute.attgenerated != '\0' */
- char attnullability; /* status of not-null constraint, see below */
uint8 attalignby; /* alignment requirement in bytes */
+ bool attispackable:1; /* FormData_pg_attribute.attstorage !=
+ * TYPSTORAGE_PLAIN */
+ bool atthasmissing:1; /* as FormData_pg_attribute.atthasmissing */
+ bool attisdropped:1; /* as FormData_pg_attribute.attisdropped */
+ bool attgenerated:1; /* FormData_pg_attribute.attgenerated != '\0' */
+ char attnullability; /* status of not-null constraint, see below */
} CompactAttribute;
/* Valid values for CompactAttribute->attnullability */
--
2.51.0
[text/plain] v13-0006-WIP-Introduce-selective-tuple-deforming.patch (47.0K, 7-v13-0006-WIP-Introduce-selective-tuple-deforming.patch)
download | inline diff:
From ca630369d8f299c4425817479a164a6e9b37cd72 Mon Sep 17 00:00:00 2001
From: David Rowley <[email protected]>
Date: Wed, 4 Mar 2026 16:55:09 +1300
Subject: [PATCH v13 6/6] WIP: Introduce selective tuple deforming
Up until now, we have always deformed every attribute of each tuple up
until the last attribute that we require. This did once make sense to
do as we often had to walk the tuple in order to determine the byte
offset to any attribute that we deform. Now, since we proactively
populate the CompactAttribute.attcacheoff, in some cases we might be in
a better position to only deform the attributes that we need to deform
by directly jumping to the cached byte offset and skipping deforming any
attributes which are not required. In some cases the savings can be
very large as it not only allows tts_values and tts_isnull for unneeded
attributes to be left unpopulated, but it might also mean we can skip
loading entire cachelines when the tuple is wide enough to span multiple
cachelines.
We don't want to exclusively always deform tuples this way as doing this
means paying attention to an additional array which states which attnums
we must deform. Looking at that array for a SELECT * query, which
requires us to deform all attributes, would add overhead. To support
this a new expression evaluation operator has been added called
EEOP_SCAN_SELECTSOME and each function which builds an ExprState now
accepts a variant function that allows the caller to specify which attnums
are required from the scan side. This puts it on the caller to decide
which type of deforming should be done. When the caller provides
the attnums, the expression will be built with EEOP_SCAN_SELECTSOME
rather than EEOP_SCAN_FETCHSOME. This currently does not interact well
with the physical tlist optimization. Currently it's the planner's job
to figure out which attributes are actually required.
TODO: JIT support
---
src/backend/executor/execExpr.c | 182 ++++++++-
src/backend/executor/execExprInterp.c | 13 +
src/backend/executor/execScan.c | 18 +
src/backend/executor/execTuples.c | 364 +++++++++++++++++-
src/backend/executor/execUtils.c | 47 ++-
src/backend/executor/nodeSeqscan.c | 8 +-
src/backend/optimizer/plan/createplan.c | 43 ++-
src/include/executor/execExpr.h | 24 +-
src/include/executor/executor.h | 19 +
src/include/executor/tuptable.h | 22 ++
src/include/nodes/plannodes.h | 8 +
.../deform_bench/deform_bench--1.0.sql | 4 +
src/test/modules/deform_bench/deform_bench.c | 98 +++++
13 files changed, 821 insertions(+), 29 deletions(-)
diff --git a/src/backend/executor/execExpr.c b/src/backend/executor/execExpr.c
index bd46b75e498..a9dd2842ea5 100644
--- a/src/backend/executor/execExpr.c
+++ b/src/backend/executor/execExpr.c
@@ -66,8 +66,18 @@ typedef struct ExprSetupInfo
AttrNumber last_new;
/* MULTIEXPR SubPlan nodes appearing in the expression: */
List *multiexpr_subplans;
+
+ /*
+ * Fetch only these attnums from the scan with EEOP_SCAN_SELECTSOME. Empty
+ * set means use EEOP_SCAN_FETCHSOME (i.e fetch all up until last_scan).
+ * The first user attribute is based at member 0. System attributes not
+ * represented.
+ */
+ Bitmapset *scan_attrs;
} ExprSetupInfo;
+static ExprState *ExecInitExprInternal(Expr *node, PlanState *parent,
+ Node *escontext, Bitmapset *scan_attrs);
static void ExecReadyExpr(ExprState *state);
static void ExecInitExprRec(Expr *node, ExprState *state,
Datum *resv, bool *resnull);
@@ -77,7 +87,8 @@ static void ExecInitFunc(ExprEvalStep *scratch, Expr *node, List *args,
static void ExecInitSubPlanExpr(SubPlan *subplan,
ExprState *state,
Datum *resv, bool *resnull);
-static void ExecCreateExprSetupSteps(ExprState *state, Node *node);
+static void ExecCreateExprSetupSteps(ExprState *state, Node *node,
+ Bitmapset *scan_attrs);
static void ExecPushExprSetupSteps(ExprState *state, ExprSetupInfo *info);
static bool expr_setup_walker(Node *node, ExprSetupInfo *info);
static bool ExecComputeSlotInfo(ExprState *state, ExprEvalStep *op);
@@ -142,7 +153,7 @@ static void ExecInitJsonCoercion(ExprState *state, JsonReturning *returning,
ExprState *
ExecInitExpr(Expr *node, PlanState *parent)
{
- return ExecInitExprWithContext(node, parent, NULL);
+ return ExecInitExprInternal(node, parent, NULL, NULL);
}
/*
@@ -161,6 +172,31 @@ ExecInitExpr(Expr *node, PlanState *parent)
*/
ExprState *
ExecInitExprWithContext(Expr *node, PlanState *parent, Node *escontext)
+{
+ return ExecInitExprInternal(node, parent, escontext, NULL);
+}
+
+/*
+ * ExecInitExprWithScanAttrs
+ * As ExecInitExpr but when 'scan_attrs' is set, use
+ * EEOP_SCAN_SELECTSOME instead of EEOP_SCAN_FETCHSOME to deform only
+ * the mentioned 'scan_attrs' from the scan tuple.
+ */
+ExprState *
+ExecInitExprWithScanAttrs(Expr *node, PlanState *parent,
+ Bitmapset *scan_attrs)
+{
+ return ExecInitExprInternal(node, parent, NULL, scan_attrs);
+}
+
+/*
+ * ExecInitExprInteral
+ * Internal version to implement ExecInitExpr, ExecInitExprWithContext
+ * and ExecInitExprWithScanAttrs.
+ */
+static ExprState *
+ExecInitExprInternal(Expr *node, PlanState *parent, Node *escontext,
+ Bitmapset *scan_attrs)
{
ExprState *state;
ExprEvalStep scratch = {0};
@@ -177,7 +213,7 @@ ExecInitExprWithContext(Expr *node, PlanState *parent, Node *escontext)
state->escontext = (ErrorSaveContext *) escontext;
/* Insert setup steps as needed */
- ExecCreateExprSetupSteps(state, (Node *) node);
+ ExecCreateExprSetupSteps(state, (Node *) node, scan_attrs);
/* Compile the expression proper */
ExecInitExprRec(node, state, &state->resvalue, &state->resnull);
@@ -214,7 +250,7 @@ ExecInitExprWithParams(Expr *node, ParamListInfo ext_params)
state->ext_params = ext_params;
/* Insert setup steps as needed */
- ExecCreateExprSetupSteps(state, (Node *) node);
+ ExecCreateExprSetupSteps(state, (Node *) node, NULL);
/* Compile the expression proper */
ExecInitExprRec(node, state, &state->resvalue, &state->resnull);
@@ -248,6 +284,19 @@ ExecInitExprWithParams(Expr *node, ParamListInfo ext_params)
*/
ExprState *
ExecInitQual(List *qual, PlanState *parent)
+{
+ return ExecInitQualWithScanAttrs(qual, parent, NULL);
+}
+
+/*
+ * ExecInitQualWithScanAttrs
+ * As ExecInitQual but when 'scan_attrs' is set, use
+ * EEOP_SCAN_SELECTSOME instead of EEOP_SCAN_FETCHSOME to deform only
+ * the mentioned 'scan_attrs' from the scan tuple.
+ */
+ExprState *
+ExecInitQualWithScanAttrs(List *qual, PlanState *parent,
+ Bitmapset *scan_attrs)
{
ExprState *state;
ExprEvalStep scratch = {0};
@@ -268,7 +317,7 @@ ExecInitQual(List *qual, PlanState *parent)
state->flags = EEO_FLAG_IS_QUAL;
/* Insert setup steps as needed */
- ExecCreateExprSetupSteps(state, (Node *) qual);
+ ExecCreateExprSetupSteps(state, (Node *) qual, scan_attrs);
/*
* ExecQual() needs to return false for an expression returning NULL. That
@@ -393,6 +442,28 @@ ExecBuildProjectionInfo(List *targetList,
TupleTableSlot *slot,
PlanState *parent,
TupleDesc inputDesc)
+{
+ return ExecBuildProjectionInfoWithScanAttrs(targetList,
+ econtext,
+ slot,
+ parent,
+ inputDesc,
+ NULL);
+}
+
+/*
+ * ExecBuildProjectionInfoWithScanAttrs
+ * As ExecBuildProjectionInfo but when 'scan_attrs' is set, use
+ * EEOP_SCAN_SELECTSOME instead of EEOP_SCAN_FETCHSOME to deform only
+ * the mentioned 'scan_attrs' from the scan tuple.
+ */
+ProjectionInfo *
+ExecBuildProjectionInfoWithScanAttrs(List *targetList,
+ ExprContext *econtext,
+ TupleTableSlot *slot,
+ PlanState *parent,
+ TupleDesc inputDesc,
+ Bitmapset *scan_attrs)
{
ProjectionInfo *projInfo = makeNode(ProjectionInfo);
ExprState *state;
@@ -410,7 +481,7 @@ ExecBuildProjectionInfo(List *targetList,
state->resultslot = slot;
/* Insert setup steps as needed */
- ExecCreateExprSetupSteps(state, (Node *) targetList);
+ ExecCreateExprSetupSteps(state, (Node *) targetList, scan_attrs);
/* Now compile each tlist column */
foreach(lc, targetList)
@@ -2904,11 +2975,19 @@ ExecInitSubPlanExpr(SubPlan *subplan,
/*
* Add expression steps performing setup that's needed before any of the
* main execution of the expression.
+ *
+ * 'scan_attrs' may be given an empty set, in which case deforming the scan
+ * tuple is done via EEOP_SCAN_FETCHSOME, which fetches every attribute from
+ * the scan tuple up until the maximum attribute used by this expression.
+ * When 'scan_attrs' is set, EEOP_SCAN_SELECTSOME is used to only fetch the
+ * attributes mentioned. Callers must create a unioned set of the attributes
+ * needed from the scan for all expressions using the given slot so that we
+ * incrementally fetch the attributes required by all ExprStates.
*/
static void
-ExecCreateExprSetupSteps(ExprState *state, Node *node)
+ExecCreateExprSetupSteps(ExprState *state, Node *node, Bitmapset *scan_attrs)
{
- ExprSetupInfo info = {0, 0, 0, 0, 0, NIL};
+ ExprSetupInfo info = {0, 0, 0, 0, 0, NIL, scan_attrs};
/* Prescan to find out what we need. */
expr_setup_walker(node, &info);
@@ -2956,11 +3035,75 @@ ExecPushExprSetupSteps(ExprState *state, ExprSetupInfo *info)
}
if (info->last_scan > 0)
{
- scratch.opcode = EEOP_SCAN_FETCHSOME;
- scratch.d.fetch.last_var = info->last_scan;
- scratch.d.fetch.fixed = false;
- scratch.d.fetch.kind = NULL;
- scratch.d.fetch.known_desc = NULL;
+ /*
+ * We have two operators for fetching attributes out of a tuple during
+ * scans. EEOP_SCAN_FETCHSOME deforms all attributes in the tuple up
+ * to the 'last_scan' attnum. This isn't ideal in some cases, as we
+ * may only need a few attributes, and those might be deep into the
+ * tuple. EEOP_SCAN_SELECTSOME is an operator that fetches only the
+ * required attributes from the tuple. When the attcacheoff for these
+ * attributes is known and no NULLs exist in the tuple prior to the
+ * required attributes, then this can be a very fast operation.
+ * EEOP_SCAN_FETCHSOME is still supported as many cases require all
+ * attributes, and EEOP_SCAN_FETCHSOME can do this more efficiently.
+ */
+ if (bms_is_empty(info->scan_attrs))
+ {
+ scratch.opcode = EEOP_SCAN_FETCHSOME;
+ scratch.d.fetch.last_var = info->last_scan;
+ scratch.d.fetch.fixed = false;
+ scratch.d.fetch.kind = NULL;
+ scratch.d.fetch.known_desc = NULL;
+ }
+ else
+ {
+ int nattrs = bms_num_members(info->scan_attrs);
+ AttrNumber *atts;
+ int a;
+ int i;
+
+ scratch.opcode = EEOP_SCAN_SELECTSOME;
+ scratch.d.fetch.last_var = info->last_scan;
+ scratch.d.fetch.fixed = false;
+ scratch.d.fetch.natts = nattrs;
+ scratch.d.fetch.kind = NULL;
+ scratch.d.fetch.known_desc = NULL;
+
+ /*
+ * Allocate these two arrays as a single allocation. The
+ * req_attnums array needs 1 element for each attnum that's being
+ * selected, plus a sentinel attnum which we set to the
+ * 'last_scan' attnum so that we correctly terminate each of the
+ * loops during selective deformation before walking off the end
+ * of the array.
+ */
+ atts = palloc_array(AttrNumber, nattrs + 1 + info->last_scan + 1);
+
+ scratch.d.fetch.req_attnums = atts;
+ scratch.d.fetch.next_req_attnums_index = &atts[nattrs + 1];
+
+ /* Store each attnum in the Bitmapset into the req_attnum array */
+ a = -1;
+ i = 0;
+ while ((a = bms_next_member(info->scan_attrs, a)) >= 0)
+ scratch.d.fetch.req_attnums[i++] = a;
+
+ /* install sentinel */
+ scratch.d.fetch.req_attnums[nattrs] = info->last_scan;
+
+ /*
+ * Populate the next_req_attnums_index array. This allows the
+ * deforming function to refind the position in the
+ * next_req_attnums_index array from tts_nvalid.
+ */
+ a = 0;
+ for (i = 0; i <= info->last_scan; i++)
+ {
+ scratch.d.fetch.next_req_attnums_index[i] = a;
+ if (bms_is_member(i, info->scan_attrs))
+ a++;
+ }
+ }
if (ExecComputeSlotInfo(state, &scratch))
ExprEvalPushStep(state, &scratch);
}
@@ -3033,6 +3176,13 @@ expr_setup_walker(Node *node, ExprSetupInfo *info)
switch (variable->varreturningtype)
{
case VAR_RETURNING_DEFAULT:
+
+ /*
+ * scan_attrs must contain a member for this attnum or
+ * be completely empty
+ */
+ Assert(attnum < 0 || bms_is_empty(info->scan_attrs) ||
+ bms_is_member(attnum - 1, info->scan_attrs));
info->last_scan = Max(info->last_scan, attnum);
break;
case VAR_RETURNING_OLD:
@@ -3099,7 +3249,8 @@ ExecComputeSlotInfo(ExprState *state, ExprEvalStep *op)
opcode == EEOP_OUTER_FETCHSOME ||
opcode == EEOP_SCAN_FETCHSOME ||
opcode == EEOP_OLD_FETCHSOME ||
- opcode == EEOP_NEW_FETCHSOME);
+ opcode == EEOP_NEW_FETCHSOME ||
+ opcode == EEOP_SCAN_SELECTSOME);
if (op->d.fetch.known_desc != NULL)
{
@@ -3152,6 +3303,7 @@ ExecComputeSlotInfo(ExprState *state, ExprEvalStep *op)
}
}
else if (opcode == EEOP_SCAN_FETCHSOME ||
+ opcode == EEOP_SCAN_SELECTSOME ||
opcode == EEOP_OLD_FETCHSOME ||
opcode == EEOP_NEW_FETCHSOME)
{
@@ -4344,7 +4496,7 @@ ExecBuildHash32Expr(TupleDesc desc, const TupleTableSlotOps *ops,
state->parent = parent;
/* Insert setup steps as needed. */
- ExecCreateExprSetupSteps(state, (Node *) hash_exprs);
+ ExecCreateExprSetupSteps(state, (Node *) hash_exprs, NULL);
/*
* Make a place to store intermediate hash values between subsequent
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 61ff5ddc74c..76965826f83 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -479,6 +479,7 @@ ExecInterpExpr(ExprState *state, ExprContext *econtext, bool *isnull)
&&CASE_EEOP_SCAN_FETCHSOME,
&&CASE_EEOP_OLD_FETCHSOME,
&&CASE_EEOP_NEW_FETCHSOME,
+ &&CASE_EEOP_SCAN_SELECTSOME,
&&CASE_EEOP_INNER_VAR,
&&CASE_EEOP_OUTER_VAR,
&&CASE_EEOP_SCAN_VAR,
@@ -676,6 +677,18 @@ ExecInterpExpr(ExprState *state, ExprContext *econtext, bool *isnull)
EEO_NEXT();
}
+ EEO_CASE(EEOP_SCAN_SELECTSOME)
+ {
+ CheckOpSlotCompatibility(op, scanslot);
+
+ slot_selectattrs(scanslot,
+ op->d.fetch.last_var,
+ op->d.fetch.req_attnums,
+ op->d.fetch.next_req_attnums_index);
+
+ EEO_NEXT();
+ }
+
EEO_CASE(EEOP_INNER_VAR)
{
int attnum = op->d.var.attnum;
diff --git a/src/backend/executor/execScan.c b/src/backend/executor/execScan.c
index 9f68be17b99..525af11aa08 100644
--- a/src/backend/executor/execScan.c
+++ b/src/backend/executor/execScan.c
@@ -86,6 +86,24 @@ ExecAssignScanProjectionInfo(ScanState *node)
ExecConditionalAssignProjectionInfo(&node->ps, tupdesc, scan->scanrelid);
}
+/*
+ * ExecAssignScanProjectionInfoWithScanAttrs
+ * As ExecAssignScanProjectionInfo but when 'scan_attrs' is set, use
+ * EEOP_SCAN_SELECTSOME instead of EEOP_SCAN_FETCHSOME to deform only
+ * the mentioned 'scan_attrs' from the scan tuple.
+ */
+void
+ExecAssignScanProjectionInfoWithScanAttrs(ScanState *node,
+ Bitmapset *scan_attrs)
+{
+ Scan *scan = (Scan *) node->ps.plan;
+ TupleDesc tupdesc = node->ss_ScanTupleSlot->tts_tupleDescriptor;
+
+ ExecConditionalAssignProjectionInfoWithScanAttrs(&node->ps, tupdesc,
+ scan->scanrelid,
+ scan_attrs);
+}
+
/*
* ExecAssignScanProjectionInfoWithVarno
* As above, but caller can specify varno expected in Vars in the tlist.
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index d24c07dca9f..a65b4ca2ee6 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -74,6 +74,12 @@ static TupleDesc ExecTypeFromTLInternal(List *targetList,
bool skipjunk);
static pg_attribute_always_inline void slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
int reqnatts);
+static pg_attribute_always_inline void slot_selectively_deform_heap_tuple(TupleTableSlot *slot,
+ HeapTuple tuple,
+ uint32 *offp,
+ int last_attnum,
+ AttrNumber *attnums,
+ AttrNumber *attnum_map);
static inline void tts_buffer_heap_store_tuple(TupleTableSlot *slot,
HeapTuple tuple,
Buffer buffer,
@@ -129,7 +135,22 @@ tts_virtual_clear(TupleTableSlot *slot)
static void
tts_virtual_getsomeattrs(TupleTableSlot *slot, int natts)
{
- elog(ERROR, "getsomeattrs is not required to be called on a virtual tuple table slot");
+ elog(ERROR,
+ "%s is not required to be called on a virtual tuple table slot",
+ "getsomeattrs");
+}
+
+/*
+ * VirtualTupleTableSlots always have fully populated tts_values and
+ * tts_isnull arrays. So this function should never be called.
+ */
+static void
+tts_virtual_selectattrs(TupleTableSlot *slot, int last_attnum,
+ AttrNumber *attnums, AttrNumber *attnum_map)
+{
+ elog(ERROR,
+ "%s is not required to be called on a virtual tuple table slot",
+ "selectattrs");
}
/*
@@ -352,6 +373,22 @@ tts_heap_getsomeattrs(TupleTableSlot *slot, int natts)
slot_deform_heap_tuple(slot, hslot->tuple, &hslot->off, natts);
}
+static void
+tts_heap_selectattrs(TupleTableSlot *slot, int last_attnum,
+ AttrNumber *attnums, AttrNumber *attnum_map)
+{
+ HeapTupleTableSlot *hslot = (HeapTupleTableSlot *) slot;
+
+ Assert(!TTS_EMPTY(slot));
+
+ slot_selectively_deform_heap_tuple(slot,
+ hslot->tuple,
+ &hslot->off,
+ last_attnum,
+ attnums,
+ attnum_map);
+}
+
static Datum
tts_heap_getsysattr(TupleTableSlot *slot, int attnum, bool *isnull)
{
@@ -550,6 +587,22 @@ tts_minimal_getsomeattrs(TupleTableSlot *slot, int natts)
slot_deform_heap_tuple(slot, mslot->tuple, &mslot->off, natts);
}
+static void
+tts_minimal_selectattrs(TupleTableSlot *slot, int last_attnum,
+ AttrNumber *attnums, AttrNumber *attnum_map)
+{
+ MinimalTupleTableSlot *mslot = (MinimalTupleTableSlot *) slot;
+
+ Assert(!TTS_EMPTY(slot));
+
+ slot_selectively_deform_heap_tuple(slot,
+ mslot->tuple,
+ &mslot->off,
+ last_attnum,
+ attnums,
+ attnum_map);
+}
+
/*
* MinimalTupleTableSlots never provide system attributes. We generally
* shouldn't get here, but provide a user-friendly message if we do.
@@ -757,6 +810,23 @@ tts_buffer_heap_getsomeattrs(TupleTableSlot *slot, int natts)
slot_deform_heap_tuple(slot, bslot->base.tuple, &bslot->base.off, natts);
}
+static void
+tts_buffer_heap_selectattrs(TupleTableSlot *slot, int last_attnum,
+ AttrNumber *attnums, AttrNumber *attnum_map)
+{
+ BufferHeapTupleTableSlot *bslot = (BufferHeapTupleTableSlot *) slot;
+
+ Assert(!TTS_EMPTY(slot));
+
+ slot_selectively_deform_heap_tuple(slot,
+ bslot->base.tuple,
+ &bslot->base.off,
+ last_attnum,
+ attnums,
+ attnum_map);
+}
+
+
static Datum
tts_buffer_heap_getsysattr(TupleTableSlot *slot, int attnum, bool *isnull)
{
@@ -1242,12 +1312,301 @@ done:
*offp = off;
}
+/*
+ * slot_selectively_deform_heap_tuple
+ * Deform attributes of 'tuple' into the Datum/isnull arrays in 'slot'.
+ * Unlike slot_deform_heap_tuple, which deforms every attribute up to the
+ * given attribute number, here we deform only the attribute numbers
+ * mentioned in the 'attnums' array. When only a few attributes are
+ * required, this can be more efficient. When the attributes have a
+ * known attcacheoff and it's valid to use that, then this version can be
+ * much more efficient than slot_deform_heap_tuple when only a small
+ * number of the total attributes are required.
+ */
+static pg_attribute_always_inline void
+slot_selectively_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple,
+ uint32 *offp, int last_attnum,
+ AttrNumber *attnums,
+ AttrNumber *attnum_map)
+{
+ CompactAttribute *cattrs;
+ CompactAttribute *cattr;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+ HeapTupleHeader tup = tuple->t_data;
+ size_t attnum;
+ int attnums_idx;
+ int firstNonCacheOffsetAttr;
+ int firstNonGuaranteedAttr;
+ int firstNullAttr;
+ int natts;
+ Datum *values;
+ bool *isnull;
+ char *tp; /* ptr to tuple data */
+ uint32 off; /* offset in tuple data */
+ int off_attnum; /* the attnum that 'off' points to */
+
+ /* Did someone forget to call TupleDescFinalize()? */
+ Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);
+
+ isnull = slot->tts_isnull;
+
+ /*
+ * Some callers may form and deform tuples prior to NOT NULL constraints
+ * being checked. Here we'd like to optimize the case where we only need
+ * to fetch attributes before or up to the point where the attribute is
+ * guaranteed to exist in the tuple. We rely on the slot flag being set
+ * correctly to only enable this optimization when it's valid to do so.
+ * This optimization allows us to save fetching the number of attributes
+ * from the tuple and saves the additional cost of handling non-byval
+ * attrs.
+ */
+ firstNonGuaranteedAttr = Min(last_attnum, slot->tts_first_nonguaranteed);
+ firstNonCacheOffsetAttr = tupleDesc->firstNonCachedOffsetAttr;
+
+ if (HeapTupleHasNulls(tuple))
+ {
+ natts = HeapTupleHeaderGetNatts(tup);
+ tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits) +
+ BITMAPLEN(natts));
+
+ natts = Min(natts, last_attnum);
+ if (natts > firstNonGuaranteedAttr)
+ {
+ bits8 *bp = tup->t_bits;
+
+ /* Find the first NULL attr */
+ firstNullAttr = first_null_attr(bp, natts);
+
+ /*
+ * And populate the isnull array for all attributes up to the last
+ * attribute we're deforming. When not using attcacheoff, we need
+ * to know if an attribute is NULL even when we're not deforming
+ * it, so that we can skip over it when calculating the offset to
+ * attributes that we are deforming.
+ */
+ populate_isnull_array(bp, natts, isnull);
+ }
+ else
+ {
+ /* Otherwise all required columns are guaranteed to exist */
+ firstNullAttr = natts;
+ }
+ }
+ else
+ {
+ tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits));
+
+ /*
+ * We only need to look at the tuple's natts if we need more than the
+ * guaranteed number of columns
+ */
+ if (last_attnum > firstNonGuaranteedAttr)
+ natts = Min(HeapTupleHeaderGetNatts(tup), last_attnum);
+ else
+ {
+ /* No need to access the number of attributes in the tuple */
+ natts = last_attnum;
+ }
+
+ /* All attrs can be fetched without checking for NULLs */
+ firstNullAttr = natts;
+ }
+
+ attnums_idx = attnum_map[slot->tts_nvalid];
+ attnum = attnums[attnums_idx];
+ values = slot->tts_values;
+
+ /*
+ * We store the tupleDesc's CompactAttribute array in 'cattrs' as gcc
+ * seems to be unwilling to optimize accessing the CompactAttribute
+ * element efficiently when accessing it via TupleDescCompactAttr().
+ */
+ cattrs = tupleDesc->compact_attrs;
+
+ /* Ensure we calculated tp correctly */
+ Assert(tp == (char *) tup + tup->t_hoff);
+
+ if (attnum < firstNonGuaranteedAttr)
+ {
+ int attlen;
+
+ /*
+ * We use a do/while loop as the if condition above guarantees at
+ * least one loop and confirms to the compiler that 'attlen' and 'off'
+ * get initialized, which some compilers are not clever enough to
+ * figure out if we were to use a for loop.
+ */
+ do
+ {
+ isnull[attnum] = false;
+ cattr = &cattrs[attnum];
+ attlen = cattr->attlen;
+
+ /* We don't expect any non-byval types */
+ pg_assume(attlen > 0);
+ Assert(cattr->attbyval == true);
+
+ off = cattr->attcacheoff;
+ values[attnum] = fetch_att_noerr(tp + off, true, attlen);
+ attnum = attnums[++attnums_idx];
+ } while (attnum < firstNonGuaranteedAttr);
+
+ off += attlen;
+
+ if (attnum == last_attnum)
+ goto done;
+ }
+
+ /* We can use attcacheoff up until the first NULL */
+ firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, firstNullAttr);
+
+ /*
+ * Handle the portion of the tuple that we have cached the offset for up
+ * to the first NULL attribute. The offset is effectively fixed for
+ * these, so we can use the CompactAttribute's attcacheoff.
+ */
+ if (attnum < firstNonCacheOffsetAttr)
+ {
+ int attlen;
+
+ do
+ {
+ isnull[attnum] = false;
+ cattr = &cattrs[attnum];
+ attlen = cattr->attlen;
+ off = cattr->attcacheoff;
+ values[attnum] = fetch_att_noerr(tp + off, cattr->attbyval,
+ attlen);
+ attnum = attnums[++attnums_idx];
+ } while (attnum < firstNonCacheOffsetAttr);
+
+ off += attlen;
+ Assert(attlen > 0);
+
+ if (attnum == last_attnum)
+ goto done;
+ }
+
+
+ if (slot->tts_nvalid >= firstNonCacheOffsetAttr)
+ {
+ /* Restore state from previous execution */
+ off_attnum = slot->tts_nvalid;
+ off = *offp;
+ }
+ else
+ {
+ off_attnum = firstNonCacheOffsetAttr - 1;
+ off = cattrs[off_attnum].attcacheoff;
+ }
+
+ /*
+ * We no longer have the ability to use attcacheoff, so we must look
+ * through all attributes from this point on. For attributes that we are
+ * not selecting, we only calculate the offset to skip them, and don't do
+ * the actual fetch. Here we loop up to the first NULL attribute.
+ */
+ for (; off_attnum < firstNullAttr; off_attnum++)
+ {
+ int attlen;
+
+ cattr = &cattrs[off_attnum];
+ attlen = cattr->attlen;
+
+ /*
+ * cstrings don't exist in heap tuples. Use pg_assume to instruct the
+ * compiler not to emit the cstring-related code in
+ * align_fetch_then_add().
+ */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ off = att_pointer_alignby(off, cattr->attalignby, attlen, tp + off);
+
+ /*
+ * If this is an attribute we want, do the fetch and then move attnum
+ * to the next attribute we want.
+ */
+ if (off_attnum == attnum)
+ {
+ isnull[off_attnum] = false;
+ values[off_attnum] =
+ fetch_att_noerr(tp + off, cattr->attbyval,
+ attlen);
+ attnum = attnums[++attnums_idx];
+
+ }
+ /* Move offset beyond this attribute */
+ off = att_addlength_pointer(off, attlen, tp + off);
+ }
+
+ /*
+ * Now handle any remaining attributes in the tuple up to the requested
+ * attnum. This time, include NULL checks as we're now at the first NULL
+ * attribute.
+ */
+ for (; off_attnum < natts; off_attnum++)
+ {
+ int attlen;
+
+ cattr = &cattrs[off_attnum];
+ attlen = cattr->attlen;
+
+ /* As above, we don't expect cstrings */
+ pg_assume(attlen > 0 || attlen == -1);
+
+ /* Is this an attribute we're selecting? */
+ if (off_attnum == attnum)
+ {
+ attnum = attnums[++attnums_idx];
+
+ if (isnull[off_attnum])
+ {
+ values[off_attnum] = (Datum) 0;
+ continue;
+ }
+
+ /*
+ * align 'off', fetch the datum, and increment off beyond the
+ * datum
+ */
+ values[off_attnum] = align_fetch_then_add(tp,
+ &off,
+ cattr->attbyval,
+ attlen,
+ cattr->attalignby);
+ }
+ else if (!isnull[off_attnum])
+ {
+ /* We don't want this attribute, move beyond it */
+ off = att_pointer_alignby(off, cattr->attalignby, attlen, tp + off);
+ off = att_addlength_pointer(off, attlen, tp + off);
+ }
+
+ }
+
+ /* Fetch any missing attrs and raise an error if reqnatts is invalid */
+ if (unlikely(attnum < last_attnum))
+ {
+ *offp = off;
+ /* XXX worth doing this selectively too? */
+ slot_getmissingattrs(slot, attnum, last_attnum);
+ slot->tts_nvalid = last_attnum;
+ return;
+ }
+done:
+
+ slot->tts_nvalid = last_attnum;
+ /* Save current offset for next execution */
+ *offp = off;
+}
+
const TupleTableSlotOps TTSOpsVirtual = {
.base_slot_size = sizeof(VirtualTupleTableSlot),
.init = tts_virtual_init,
.release = tts_virtual_release,
.clear = tts_virtual_clear,
.getsomeattrs = tts_virtual_getsomeattrs,
+ .selectattrs = tts_virtual_selectattrs,
.getsysattr = tts_virtual_getsysattr,
.materialize = tts_virtual_materialize,
.is_current_xact_tuple = tts_virtual_is_current_xact_tuple,
@@ -1269,6 +1628,7 @@ const TupleTableSlotOps TTSOpsHeapTuple = {
.release = tts_heap_release,
.clear = tts_heap_clear,
.getsomeattrs = tts_heap_getsomeattrs,
+ .selectattrs = tts_heap_selectattrs,
.getsysattr = tts_heap_getsysattr,
.is_current_xact_tuple = tts_heap_is_current_xact_tuple,
.materialize = tts_heap_materialize,
@@ -1287,6 +1647,7 @@ const TupleTableSlotOps TTSOpsMinimalTuple = {
.release = tts_minimal_release,
.clear = tts_minimal_clear,
.getsomeattrs = tts_minimal_getsomeattrs,
+ .selectattrs = tts_minimal_selectattrs,
.getsysattr = tts_minimal_getsysattr,
.is_current_xact_tuple = tts_minimal_is_current_xact_tuple,
.materialize = tts_minimal_materialize,
@@ -1305,6 +1666,7 @@ const TupleTableSlotOps TTSOpsBufferHeapTuple = {
.release = tts_buffer_heap_release,
.clear = tts_buffer_heap_clear,
.getsomeattrs = tts_buffer_heap_getsomeattrs,
+ .selectattrs = tts_buffer_heap_selectattrs,
.getsysattr = tts_buffer_heap_getsysattr,
.is_current_xact_tuple = tts_buffer_is_current_xact_tuple,
.materialize = tts_buffer_heap_materialize,
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index f62582859f9..252e8306631 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -580,8 +580,7 @@ ExecGetCommonChildSlotOps(PlanState *ps)
* ----------------
*/
void
-ExecAssignProjectionInfo(PlanState *planstate,
- TupleDesc inputDesc)
+ExecAssignProjectionInfo(PlanState *planstate, TupleDesc inputDesc)
{
planstate->ps_ProjInfo =
ExecBuildProjectionInfo(planstate->plan->targetlist,
@@ -591,6 +590,28 @@ ExecAssignProjectionInfo(PlanState *planstate,
inputDesc);
}
+/* ----------------
+ * ExecAssignProjectionInfoWithScanAttrs
+ *
+ * As ExecAssignScanProjectionInfo but when 'scan_attrs' is set, use
+ * EEOP_SCAN_SELECTSOME instead of EEOP_SCAN_FETCHSOME to deform only the
+ * mentioned 'scan_attrs' from the scan tuple.
+ * ----------------
+*/
+void
+ExecAssignProjectionInfoWithScanAttrs(PlanState *planstate,
+ TupleDesc inputDesc,
+ Bitmapset *scan_attrs)
+{
+ planstate->ps_ProjInfo =
+ ExecBuildProjectionInfoWithScanAttrs(planstate->plan->targetlist,
+ planstate->ps_ExprContext,
+ planstate->ps_ResultTupleSlot,
+ planstate,
+ inputDesc,
+ scan_attrs);
+}
+
/* ----------------
* ExecConditionalAssignProjectionInfo
@@ -602,6 +623,26 @@ ExecAssignProjectionInfo(PlanState *planstate,
void
ExecConditionalAssignProjectionInfo(PlanState *planstate, TupleDesc inputDesc,
int varno)
+{
+ ExecConditionalAssignProjectionInfoWithScanAttrs(planstate,
+ inputDesc,
+ varno,
+ NULL);
+}
+
+/* ----------------
+ * ExecConditionalAssignProjectionInfoWithScanAttrs
+ *
+ * As ExecConditionalAssignProjectionInfo but when 'scan_attrs' is set, use
+ * EEOP_SCAN_SELECTSOME instead of EEOP_SCAN_FETCHSOME to deform only the
+ * mentioned 'scan_attrs' from the scan tuple.
+ * ----------------
+*/
+void
+ExecConditionalAssignProjectionInfoWithScanAttrs(PlanState *planstate,
+ TupleDesc inputDesc,
+ int varno,
+ Bitmapset *scan_attrs)
{
if (tlist_matches_tupdesc(planstate,
planstate->plan->targetlist,
@@ -622,7 +663,7 @@ ExecConditionalAssignProjectionInfo(PlanState *planstate, TupleDesc inputDesc,
planstate->resultopsfixed = true;
planstate->resultopsset = true;
}
- ExecAssignProjectionInfo(planstate, inputDesc);
+ ExecAssignProjectionInfoWithScanAttrs(planstate, inputDesc, scan_attrs);
}
}
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 8f219f60a93..41de367832c 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -251,13 +251,15 @@ ExecInitSeqScan(SeqScan *node, EState *estate, int eflags)
* Initialize result type and projection.
*/
ExecInitResultTypeTL(&scanstate->ss.ps);
- ExecAssignScanProjectionInfo(&scanstate->ss);
+ ExecAssignScanProjectionInfoWithScanAttrs(&scanstate->ss,
+ node->scan.scan_varattnos);
/*
* initialize child expressions
*/
- scanstate->ss.ps.qual =
- ExecInitQual(node->scan.plan.qual, (PlanState *) scanstate);
+ scanstate->ss.ps.qual = ExecInitQualWithScanAttrs(node->scan.plan.qual,
+ (PlanState *) scanstate,
+ node->scan.scan_varattnos);
/*
* When EvalPlanQual() is not in use, assign ExecProcNode for this node
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 50b0e10308b..4522ac4d4c0 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -118,7 +118,8 @@ static ModifyTable *create_modifytable_plan(PlannerInfo *root, ModifyTablePath *
static Limit *create_limit_plan(PlannerInfo *root, LimitPath *best_path,
int flags);
static SeqScan *create_seqscan_plan(PlannerInfo *root, Path *best_path,
- List *tlist, List *scan_clauses);
+ List *tlist, List *scan_clauses,
+ Bitmapset *tlist_varattnos);
static SampleScan *create_samplescan_plan(PlannerInfo *root, Path *best_path,
List *tlist, List *scan_clauses);
static Scan *create_indexscan_plan(PlannerInfo *root, IndexPath *best_path,
@@ -178,7 +179,8 @@ static void label_sort_with_costsize(PlannerInfo *root, Sort *plan,
double limit_tuples);
static void label_incrementalsort_with_costsize(PlannerInfo *root, IncrementalSort *plan,
List *pathkeys, double limit_tuples);
-static SeqScan *make_seqscan(List *qptlist, List *qpqual, Index scanrelid);
+static SeqScan *make_seqscan(List *qptlist, List *qpqual, Index scanrelid,
+ Bitmapset *scan_varattnos);
static SampleScan *make_samplescan(List *qptlist, List *qpqual, Index scanrelid,
TableSampleClause *tsc);
static IndexScan *make_indexscan(List *qptlist, List *qpqual, Index scanrelid,
@@ -550,6 +552,7 @@ create_plan_recurse(PlannerInfo *root, Path *best_path, int flags)
static Plan *
create_scan_plan(PlannerInfo *root, Path *best_path, int flags)
{
+ Bitmapset *tlist_varattnos = NULL;
RelOptInfo *rel = best_path->parent;
List *scan_clauses;
List *gating_clauses;
@@ -579,6 +582,14 @@ create_scan_plan(PlannerInfo *root, Path *best_path, int flags)
break;
}
+ /*
+ * Figure out which attributes we need from the scan before applying the
+ * physical tlist optimization.
+ */
+ pull_varattnos((Node *) best_path->pathtarget->exprs,
+ rel->relid,
+ &tlist_varattnos);
+
/*
* If this is a parameterized scan, we also need to enforce all the join
* clauses available from the outer relation(s).
@@ -672,7 +683,8 @@ create_scan_plan(PlannerInfo *root, Path *best_path, int flags)
plan = (Plan *) create_seqscan_plan(root,
best_path,
tlist,
- scan_clauses);
+ scan_clauses,
+ tlist_varattnos);
break;
case T_SampleScan:
@@ -2752,10 +2764,13 @@ create_limit_plan(PlannerInfo *root, LimitPath *best_path, int flags)
*/
static SeqScan *
create_seqscan_plan(PlannerInfo *root, Path *best_path,
- List *tlist, List *scan_clauses)
+ List *tlist, List *scan_clauses, Bitmapset *tlist_varattnos)
{
SeqScan *scan_plan;
Index scan_relid = best_path->parent->relid;
+ Bitmapset *scan_varattnos = tlist_varattnos;
+ Bitmapset *non_sys_attrs = NULL;
+ int i;
/* it should be a base rel... */
Assert(scan_relid > 0);
@@ -2767,6 +2782,19 @@ create_seqscan_plan(PlannerInfo *root, Path *best_path,
/* Reduce RestrictInfo list to bare expressions; ignore pseudoconstants */
scan_clauses = extract_actual_clauses(scan_clauses, false);
+ /* Pull varattnos from WHERE clause Vars */
+ pull_varattnos((Node *) scan_clauses, scan_relid, &scan_varattnos);
+
+ /* Don't set these when whole-row var is present */
+ if (!bms_is_member(0 - FirstLowInvalidHeapAttributeNumber, scan_varattnos))
+ {
+ /* XXX invent bms_right_shift_members()? */
+ i = 0 - FirstLowInvalidHeapAttributeNumber;
+ while ((i = bms_next_member(scan_varattnos, i)) >= 0)
+ non_sys_attrs = bms_add_member(non_sys_attrs,
+ i - 1 + FirstLowInvalidHeapAttributeNumber);
+ }
+
/* Replace any outer-relation variables with nestloop params */
if (best_path->param_info)
{
@@ -2776,7 +2804,8 @@ create_seqscan_plan(PlannerInfo *root, Path *best_path,
scan_plan = make_seqscan(tlist,
scan_clauses,
- scan_relid);
+ scan_relid,
+ non_sys_attrs);
copy_generic_path_info(&scan_plan->scan.plan, best_path);
@@ -5487,7 +5516,8 @@ bitmap_subplan_mark_shared(Plan *plan)
static SeqScan *
make_seqscan(List *qptlist,
List *qpqual,
- Index scanrelid)
+ Index scanrelid,
+ Bitmapset *scan_varattnos)
{
SeqScan *node = makeNode(SeqScan);
Plan *plan = &node->scan.plan;
@@ -5497,6 +5527,7 @@ make_seqscan(List *qptlist,
plan->lefttree = NULL;
plan->righttree = NULL;
node->scan.scanrelid = scanrelid;
+ node->scan.scan_varattnos = scan_varattnos;
return node;
}
diff --git a/src/include/executor/execExpr.h b/src/include/executor/execExpr.h
index aa9b361fa31..f29d9dd799b 100644
--- a/src/include/executor/execExpr.h
+++ b/src/include/executor/execExpr.h
@@ -78,6 +78,9 @@ typedef enum ExprEvalOp
EEOP_OLD_FETCHSOME,
EEOP_NEW_FETCHSOME,
+ /* apply slot_selectattrs on the corresponding tuple slot */
+ EEOP_SCAN_SELECTSOME,
+
/* compute non-system Var value */
EEOP_INNER_VAR,
EEOP_OUTER_VAR,
@@ -318,15 +321,34 @@ typedef struct ExprEvalStep
*/
union
{
- /* for EEOP_INNER/OUTER/SCAN/OLD/NEW_FETCHSOME */
+ /*
+ * for EEOP_INNER/OUTER/SCAN/OLD/NEW_FETCHSOME and
+ * EEOP_SCAN_SELECTSOME
+ */
struct
{
/* attribute number up to which to fetch (inclusive) */
int last_var;
/* will the type of slot be the same for every invocation */
bool fixed;
+ /* Number of elements in req_attnums array. XXX needed? */
+ AttrNumber natts;
+
+ /* One element for each attnum to select, ordered by attnum */
+ AttrNumber *req_attnums;
+
+ /*
+ * Provides mapping of 0-based attnums back to the index of the
+ * req_attnums array that deforming should continue from. This
+ * allows us to re-find the element of req_attnums using the
+ * slot's tts_nvalid so that we can continue deforming from the
+ * last defromed attribute.
+ */
+ AttrNumber *next_req_attnums_index;
+
/* tuple descriptor, if known */
TupleDesc known_desc;
+
/* type of slot, can only be relied upon if fixed is set */
const TupleTableSlotOps *kind;
} fetch;
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index b1820653506..ea86c3822ee 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -325,8 +325,12 @@ ExecProcNode(PlanState *node)
*/
extern ExprState *ExecInitExpr(Expr *node, PlanState *parent);
extern ExprState *ExecInitExprWithContext(Expr *node, PlanState *parent, Node *escontext);
+extern ExprState *ExecInitExprWithScanAttrs(Expr *node, PlanState *parent,
+ Bitmapset *scan_attrs);
extern ExprState *ExecInitExprWithParams(Expr *node, ParamListInfo ext_params);
extern ExprState *ExecInitQual(List *qual, PlanState *parent);
+extern ExprState *ExecInitQualWithScanAttrs(List *qual, PlanState *parent,
+ Bitmapset *scan_attrs);
extern ExprState *ExecInitCheck(List *qual, PlanState *parent);
extern List *ExecInitExprList(List *nodes, PlanState *parent);
extern ExprState *ExecBuildAggTrans(AggState *aggstate, struct AggStatePerPhaseData *phase,
@@ -365,6 +369,12 @@ extern ProjectionInfo *ExecBuildProjectionInfo(List *targetList,
TupleTableSlot *slot,
PlanState *parent,
TupleDesc inputDesc);
+extern ProjectionInfo *ExecBuildProjectionInfoWithScanAttrs(List *targetList,
+ ExprContext *econtext,
+ TupleTableSlot *slot,
+ PlanState *parent,
+ TupleDesc inputDesc,
+ Bitmapset *scan_attrs);
extern ProjectionInfo *ExecBuildUpdateProjection(List *targetList,
bool evalTargetList,
List *targetColnos,
@@ -584,6 +594,8 @@ typedef bool (*ExecScanRecheckMtd) (ScanState *node, TupleTableSlot *slot);
extern TupleTableSlot *ExecScan(ScanState *node, ExecScanAccessMtd accessMtd,
ExecScanRecheckMtd recheckMtd);
extern void ExecAssignScanProjectionInfo(ScanState *node);
+extern void ExecAssignScanProjectionInfoWithScanAttrs(ScanState *node,
+ Bitmapset *scan_attrs);
extern void ExecAssignScanProjectionInfoWithVarno(ScanState *node, int varno);
extern void ExecScanReScan(ScanState *node);
@@ -680,8 +692,15 @@ extern const TupleTableSlotOps *ExecGetCommonSlotOps(PlanState **planstates,
extern const TupleTableSlotOps *ExecGetCommonChildSlotOps(PlanState *ps);
extern void ExecAssignProjectionInfo(PlanState *planstate,
TupleDesc inputDesc);
+extern void ExecAssignProjectionInfoWithScanAttrs(PlanState *planstate,
+ TupleDesc inputDesc,
+ Bitmapset *scan_attrs);
extern void ExecConditionalAssignProjectionInfo(PlanState *planstate,
TupleDesc inputDesc, int varno);
+extern void ExecConditionalAssignProjectionInfoWithScanAttrs(PlanState *planstate,
+ TupleDesc inputDesc,
+ int varno,
+ Bitmapset *scan_attrs);
extern void ExecAssignScanType(ScanState *scanstate, TupleDesc tupDesc);
extern void ExecCreateScanSlotFromOuterPlan(EState *estate,
ScanState *scanstate,
diff --git a/src/include/executor/tuptable.h b/src/include/executor/tuptable.h
index 78558098fa3..cc2c5a257d0 100644
--- a/src/include/executor/tuptable.h
+++ b/src/include/executor/tuptable.h
@@ -168,6 +168,16 @@ struct TupleTableSlotOps
*/
void (*getsomeattrs) (TupleTableSlot *slot, int natts);
+ /*
+ * Populate the tts_values and tts_isnull elements of the given slot with
+ * the values of the corresponding attribute from the tuple stored in the
+ * slot. Populate up as far as last_attnum and store each attribute
+ * mentioned in the attnums array. Use attnum_map to determine the
+ * starting element in the attnums array from the slot's tts_nvalid.
+ */
+ void (*selectattrs) (TupleTableSlot *slot, int last_attnum,
+ AttrNumber *attnums, AttrNumber *attnum_map);
+
/*
* Returns value of the given system attribute as a datum and sets isnull
* to false, if it's not NULL. Throws an error if the slot type does not
@@ -374,6 +384,18 @@ slot_getsomeattrs(TupleTableSlot *slot, int attnum)
slot->tts_ops->getsomeattrs(slot, attnum);
}
+static inline void
+slot_selectattrs(TupleTableSlot *slot, int last_attnum, AttrNumber *attnums,
+ AttrNumber *attnum_map)
+{
+ /*
+ * Populate slot only attributes mentioned in the attnums array, up to
+ * 'last_attnum', if it's not already
+ */
+ if (slot->tts_nvalid < last_attnum)
+ slot->tts_ops->selectattrs(slot, last_attnum, attnums, attnum_map);
+}
+
/*
* slot_getallattrs
* This function forces all the entries of the slot's Datum/isnull
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 8c9321aab8c..08dcf02b8bb 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -540,6 +540,14 @@ typedef struct Scan
Plan plan;
/* relid is index into the range table */
Index scanrelid;
+
+ /*
+ * All varattnos that are required from the scanrelid. Does not include
+ * any added due to the physical tlist optimization or system attributes
+ * or whole-row attributes. User attributes are 0 based, i.e attnum==1 is
+ * member 0.
+ */
+ Bitmapset *scan_varattnos;
} Scan;
/* ----------------
diff --git a/src/test/modules/deform_bench/deform_bench--1.0.sql b/src/test/modules/deform_bench/deform_bench--1.0.sql
index 492b71dba3b..4a419fde35c 100644
--- a/src/test/modules/deform_bench/deform_bench--1.0.sql
+++ b/src/test/modules/deform_bench/deform_bench--1.0.sql
@@ -6,3 +6,7 @@
CREATE FUNCTION deform_bench(tableoid Oid, attnum int[]) RETURNS FLOAT
AS 'MODULE_PATHNAME', 'deform_bench'
LANGUAGE C VOLATILE STRICT;
+
+CREATE FUNCTION deform_bench_select(tableoid Oid, attnum int[]) RETURNS FLOAT
+AS 'MODULE_PATHNAME', 'deform_bench_select'
+LANGUAGE C VOLATILE STRICT;
diff --git a/src/test/modules/deform_bench/deform_bench.c b/src/test/modules/deform_bench/deform_bench.c
index 4f104989297..f929d952596 100644
--- a/src/test/modules/deform_bench/deform_bench.c
+++ b/src/test/modules/deform_bench/deform_bench.c
@@ -16,6 +16,7 @@
#include "catalog/pg_type_d.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "nodes/bitmapset.h"
#include "utils/array.h"
#include "utils/arrayaccess.h"
#include "utils/builtins.h"
@@ -23,6 +24,7 @@
PG_MODULE_MAGIC;
PG_FUNCTION_INFO_V1(deform_bench);
+PG_FUNCTION_INFO_V1(deform_bench_select);
Datum
deform_bench(PG_FUNCTION_ARGS)
@@ -104,6 +106,102 @@ deform_bench(PG_FUNCTION_ARGS)
relation_close(rel, AccessShareLock);
+ /* Returns the number of milliseconds to run the test */
+ PG_RETURN_FLOAT8((double) (end - start) / (CLOCKS_PER_SEC / 1000));
+}
+
+Datum
+deform_bench_select(PG_FUNCTION_ARGS)
+{
+ Oid tableoid = PG_GETARG_OID(0);
+ ArrayType *array = PG_GETARG_ARRAYTYPE_P(1);
+ TableScanDesc scan;
+ Relation rel;
+ TupleDesc tupdesc;
+ TupleTableSlot *slot;
+ Datum *elem_datums = NULL;
+ bool *elem_nulls = NULL;
+ int elem_count;
+ int i;
+ int attnum;
+ int last_attnum;
+ AttrNumber *attnums;
+ Bitmapset *attrs = NULL;
+ clock_t start,
+ end;
+
+ rel = relation_open(tableoid, AccessShareLock);
+
+ if (rel->rd_rel->relam != HEAP_TABLE_AM_OID)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("only heap AM is supported")));
+
+ tupdesc = RelationGetDescr(rel);
+ slot = MakeTupleTableSlot(tupdesc,
+ &TTSOpsBufferHeapTuple,
+ TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS);
+ scan = table_beginscan_strat(rel, GetActiveSnapshot(), 0, NULL, true, false);
+
+ /*
+ * The array is used to allow callers to specify which attributes to
+ * deform. E.g: '{1,10}'::int[] would deform only attnum=1 and attnum=10.
+ *
+ * You'll get an ERROR if you pass an attnum that does not exist. NULL
+ * elements are ignored.
+ */
+ deconstruct_array(array,
+ INT4OID,
+ sizeof(int32),
+ true,
+ 'i',
+ &elem_datums,
+ &elem_nulls,
+ &elem_count);
+
+ for (i = 0; i < elem_count; i++)
+ {
+ /* Make a NULL element mean all attributes */
+ if (elem_nulls[i])
+ continue;
+
+ attnum = DatumGetInt32(elem_datums[i]);
+
+ if (attnum <= 0)
+ elog(ERROR, "only user attributes can be deformed by deform_bench_select");
+ if (attnum >= 0xffff)
+ elog(ERROR, "invalid attnum %d", attnum);
+
+ attrs = bms_add_member(attrs, attnum - 1);
+ }
+
+ attnums = palloc_array(AttrNumber, bms_num_members(attrs));
+
+ attnum = -1;
+ i = 0;
+ while ((attnum = bms_next_member(attrs, attnum)) >= 0)
+ attnums[i++] = (AttrNumber) attnum;
+
+ last_attnum = attnums[i - 1];
+
+ start = clock();
+
+ while (heap_getnextslot(scan, ForwardScanDirection, slot))
+ {
+ AttrNumber attnum_map = 0;
+ CHECK_FOR_INTERRUPTS();
+
+ /* Pass in a faked up attnum_map. tts_nvalid will always be 0 */
+ slot_selectattrs(slot, last_attnum, attnums, &attnum_map);
+ }
+
+ end = clock();
+
+ ExecDropSingleTupleTableSlot(slot);
+ table_endscan(scan);
+ relation_close(rel, AccessShareLock);
+
+
/* Returns the number of milliseconds to run the test */
PG_RETURN_FLOAT8((double) (end - start) / (CLOCKS_PER_SEC / 1000));
}
--
2.51.0
[image/gif] master_v_v13-0001-0006.gif (119.8K, 8-master_v_v13-0001-0006.gif)
download | view image
[image/gif] master_v_v13-0001-0005.gif (157.7K, 9-master_v_v13-0001-0005.gif)
download | view image
^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: More speedups for tuple deformation
@ 2026-03-15 13:50 Junwang Zhao <[email protected]>
parent: David Rowley <[email protected]>
1 sibling, 0 replies; 30+ messages in thread
From: Junwang Zhao @ 2026-03-15 13:50 UTC (permalink / raw)
To: David Rowley <[email protected]>; +Cc: Andres Freund <[email protected]>; John Naylor <[email protected]>; Chao Li <[email protected]>; PostgreSQL Developers <[email protected]>
On Fri, Mar 13, 2026 at 8:19 PM David Rowley <[email protected]> wrote:
>
> Thanks for having a look.
>
>
> On Sun, 8 Mar 2026 at 05:36, Junwang Zhao <[email protected]> wrote:
> > I have some comments on v12-0004.
> >
> > 1.
> >
> > + off += cattr->attlen;
> > + firstNonCachedOffsetAttr = i + 1;
> > + }
> > +
> > + tupdesc->firstNonCachedOffsetAttr = firstNonCachedOffsetAttr;
> > + tupdesc->firstNonGuaranteedAttr = firstNonGuaranteedAttr;
> > +}
> >
> > The firstNonCachedOffsetAttr seems to be the first variable width
> > attribute, but it seems that the offset of this attribute can be cached,
> > for example, in a table defined as (int, text), the offset of
> > firstNonCachedOffsetAttr should be 4, is that correct?
>
> Yes.
>
> > If TupleDescFinalize records the offset firstNonCachedOffsetAttr,
> > it might save one iterator of the deforming loop. For example,
> > add something like the following after the above mentioned code.
> >
> > if (firstNonCachedOffsetAttr < tupdesc->natts)
> > {
> > cattr = TupleDescCompactAttr(tupdesc, firstNonCachedOffsetAttr);
> > cattr->attcacheoff = off;
> > }
>
> The problem is that short varlenas have 1 byte alignment and normal
> varlenas have 4 byte alignment. It might be possible to do something
> if the previous column is 4-byte aligned and has a length of 4 or 8,
> since that means the offset must also be 1-byte aligned. The main
> reason I don't want to do this is that the only positive is that
> *maybe* 1 extra column can be deformed with a fixed offset. The
> drawback is that the following code *has* to use
> att_addlength_pointer(), *regardless* instead of "off += attlen;".
> This means more deforming code and more complexity in
> TupleDescFinalize(). I'd rather not do this.
That explains, thanks.
>
> > 2.
> >
> > in slot_deform_heap_tuple, there are multiple statements setting
> > firstNonCacheOffsetAttr,
> >
> > + firstNonCacheOffsetAttr = tupleDesc->firstNonCachedOffsetAttr;
> >
> > + /* We can only use any cached offsets until the first NULL attr */
> > + firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr,
> > + firstNullAttr);
> >
> > + /* We can only fetch as many attributes as the tuple has. */
> > + firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, natts);
> >
> > Based on the logic, it seems the second one could be moved
> > to the third position, and the third one could then be safely
> > removed?
>
> Yeah. Well spotted. I've done that in the attached.
>
> I've also modified the 0006 patch to add a new deform_bench_select()
> function which allows the benchmark to call the new selective deform
> function. See the attached graphs comparing master to v13-0001-0005
> and master to v13-0001-0006. It's good to see that there's still quite
> a large speedup even from the tests that don't have an attcacheoff for
> the column being deformed. Tests 1 and 5 do have a attcacheoff for the
> column deformed, so they're a good bit faster again. To get the
> 0001-0006 results, I used the deform_test_run.sh script from [1] and
> modified it to call deform_bench_select() instead of deform_bench().
>
> I also noticed that when building with older gcc versions, I was
> getting warnings about attlen and 'off' not being initialised. I ended
> up switching back to the do/while loops to fix that rather than adding
> needless initialisation, which would add overhead. 1 loop is
> guaranteed, and the older compiler is not clever enough to work that
> out.
>
> David
>
> [1] https://postgr.es/m/CAApHDvo1i-ycAcWnK3L7ZASTuM8mW46kvRqMaUHD46HSuJmx7A@mail.gmail.com
--
Regards
Junwang Zhao
^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: More speedups for tuple deformation
@ 2026-03-16 07:01 Tender Wang <[email protected]>
parent: David Rowley <[email protected]>
1 sibling, 2 replies; 30+ messages in thread
From: Tender Wang @ 2026-03-16 07:01 UTC (permalink / raw)
To: David Rowley <[email protected]>; +Cc: Andres Freund <[email protected]>; John Naylor <[email protected]>; PostgreSQL Developers <[email protected]>
Hi David,
After c456e391138, I found a crash.
How to reproduce:
psql (19devel)
Type "help" for help.
postgres=# CREATE TABLE t0 (
c0 double precision,
c1 int4range
);
INSERT INTO public.t0 VALUES (0.013077308, '[160364071,527006655)');
INSERT INTO public.t0 VALUES (0.2488792, NULL);
INSERT INTO public.t0 VALUES ('Infinity', '[-477016999,529396556)');
INSERT INTO public.t0 VALUES (0.2687647, '[-1356493040,71686839)');
INSERT INTO public.t0 VALUES (-2019956220, NULL);
INSERT INTO public.t0 VALUES (NULL, '[-1645797683,476242219)');
INSERT INTO public.t0 VALUES (0.64798653, NULL);
INSERT INTO public.t0 VALUES (0.70265657, NULL);
INSERT INTO public.t0 VALUES (0.31205487, NULL);
INSERT INTO public.t0 VALUES (0.43003803, '[-1973503943,635641598)');
INSERT INTO public.t0 VALUES (0.7500011, '[516760684,870974126)');
INSERT INTO public.t0 VALUES (0, '[-1666589420,-456062869)');
INSERT INTO public.t0 VALUES (NULL, '[-1328295821,1687052919)');
INSERT INTO public.t0 VALUES (NULL, '[-81427373,1340586611)');
INSERT INTO public.t0 VALUES (NULL, '[1469060470,1895771979)');
CREATE TABLE t2 (
c0 bigint,
c1 bigint CONSTRAINT t1_c1_not_null NOT NULL
);
INSERT INTO public.t2 VALUES (NULL, 23);
INSERT INTO public.t2 VALUES (622289345, 15);
CREATE TABLE t4 (
c0 boolean,
c1 boolean,
CONSTRAINT t4_c0_check CHECK (c0)
);
INSERT INTO public.t4 VALUES (true, true);
SELECT * FROM t2, t4 RIGHT OUTER JOIN t0 ON t4.c0 WHERE t4.c1 ORDER BY
t4.c0, t0.c1;
CREATE TABLE
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
CREATE TABLE
INSERT 0 1
INSERT 0 1
CREATE TABLE
INSERT 0 1
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
The connection to the server was lost. Attempting reset: Succeeded.
postgres=# \q
git bisect shows c456e391138 is the first bad commit.
Please take a look.
--
Thanks,
Tender Wang
^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: More speedups for tuple deformation
@ 2026-03-16 09:17 David Rowley <[email protected]>
parent: Tender Wang <[email protected]>
1 sibling, 2 replies; 30+ messages in thread
From: David Rowley @ 2026-03-16 09:17 UTC (permalink / raw)
To: Tender Wang <[email protected]>; +Cc: Andres Freund <[email protected]>; John Naylor <[email protected]>; PostgreSQL Developers <[email protected]>
On Mon, 16 Mar 2026 at 20:01, Tender Wang <[email protected]> wrote:
> SELECT * FROM t2, t4 RIGHT OUTER JOIN t0 ON t4.c0 WHERE t4.c1 ORDER BY
> server closed the connection unexpectedly
Thanks. Looks like I didn't get the startAttr logic correct in
nocachegetattr(). Starting by using the attcacheoff of the first NULL
attribute isn't valid. It should be the attribute prior to that one.
I'm just verifying some code locally now.
David
^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: More speedups for tuple deformation
@ 2026-03-16 16:37 Junwang Zhao <[email protected]>
parent: David Rowley <[email protected]>
1 sibling, 0 replies; 30+ messages in thread
From: Junwang Zhao @ 2026-03-16 16:37 UTC (permalink / raw)
To: David Rowley <[email protected]>; +Cc: Tender Wang <[email protected]>; Andres Freund <[email protected]>; John Naylor <[email protected]>; PostgreSQL Developers <[email protected]>
Hi David and Tender,
On Mon, Mar 16, 2026 at 5:17 PM David Rowley <[email protected]> wrote:
>
> On Mon, 16 Mar 2026 at 20:01, Tender Wang <[email protected]> wrote:
> > SELECT * FROM t2, t4 RIGHT OUTER JOIN t0 ON t4.c0 WHERE t4.c1 ORDER BY
>
> > server closed the connection unexpectedly
>
> Thanks. Looks like I didn't get the startAttr logic correct in
> nocachegetattr(). Starting by using the attcacheoff of the first NULL
> attribute isn't valid. It should be the attribute prior to that one.
>
> I'm just verifying some code locally now.
>
> David
>
>
The following case is more simpler:
drop table if exists ty;
create table ty(c0 int not null, c1 double precision, c2 int4range);
insert into ty values (1, 1.0, '[1469060470,1895771979)');
insert into ty values(2, null, '[-1973503943,635641598)');
select * from ty order by c2;
In this case, firstNonCachedOffsetAttr is 2 and firstNullAttr is 1.
If we start from 1, the cached offset becomes 8 due to double's
alignby, and deforming int4range from offset 8 will lead to corrupted data.
Therefore, as David said, we should start from the attribute prior
to that one. PFA is a trivial fix, I think we should add the test but
I haven't found a proper regress test file for it.
--
Regards
Junwang Zhao
Attachments:
[application/octet-stream] 0001-Fix-startAttr-computation-for-nocache-attribute-fetc.patch (1.7K, 2-0001-Fix-startAttr-computation-for-nocache-attribute-fetc.patch)
download | inline diff:
From d4b36daa6ddefae4e3548886f337d11ce284e692 Mon Sep 17 00:00:00 2001
From: Junwang Zhao <[email protected]>
Date: Tue, 17 Mar 2026 00:26:49 +0800
Subject: [PATCH] Fix startAttr computation for nocache attribute fetch
Adjust nocache[heap|index]_getattr() to base the starting attcacheoff
on the attribute before the first NULL, ensuring cached offsets are
valid.
---
src/backend/access/common/heaptuple.c | 2 +-
src/backend/access/common/indextuple.c | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 26f0c3bb2c4..31f64b0a31a 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -541,7 +541,7 @@ nocachegetattr(HeapTuple tup,
* Start at the highest attcacheoff attribute with no NULLs in prior
* attributes.
*/
- startAttr = Min(tupleDesc->firstNonCachedOffsetAttr - 1, firstNullAttr);
+ startAttr = Min(tupleDesc->firstNonCachedOffsetAttr - 1, Max(0, firstNullAttr - 1));
off = TupleDescCompactAttr(tupleDesc, startAttr)->attcacheoff;
}
else
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 6ba09932ba6..7d0cf9b3ba7 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -267,7 +267,7 @@ nocache_index_getattr(IndexTuple tup,
* Start at the highest attcacheoff attribute with no NULLs in prior
* attributes.
*/
- startAttr = Min(tupleDesc->firstNonCachedOffsetAttr - 1, firstNullAttr);
+ startAttr = Min(tupleDesc->firstNonCachedOffsetAttr - 1, Max(0, firstNullAttr - 1));
off = TupleDescCompactAttr(tupleDesc, startAttr)->attcacheoff;
}
else
--
2.41.0
^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: More speedups for tuple deformation
@ 2026-03-16 21:15 David Rowley <[email protected]>
parent: David Rowley <[email protected]>
1 sibling, 1 reply; 30+ messages in thread
From: David Rowley @ 2026-03-16 21:15 UTC (permalink / raw)
To: Tender Wang <[email protected]>; +Cc: Andres Freund <[email protected]>; John Naylor <[email protected]>; PostgreSQL Developers <[email protected]>
On Mon, 16 Mar 2026 at 22:17, David Rowley <[email protected]> wrote:
>
> On Mon, 16 Mar 2026 at 20:01, Tender Wang <[email protected]> wrote:
> > SELECT * FROM t2, t4 RIGHT OUTER JOIN t0 ON t4.c0 WHERE t4.c1 ORDER BY
>
> > server closed the connection unexpectedly
>
> Thanks. Looks like I didn't get the startAttr logic correct in
> nocachegetattr(). Starting by using the attcacheoff of the first NULL
> attribute isn't valid. It should be the attribute prior to that one.
>
> I'm just verifying some code locally now.
Now pushed. Some concerns about the lack of exercise in the tests for
fastgetattr() for user tables. I ended up testing by injecting the
following into slot_deform_heap_tuple()
- attnum = slot->tts_nvalid;
+ attnum2 = attnum = slot->tts_nvalid;
[...]
+ for (attnum = attnum2; attnum < natts; attnum++)
+ {
+ Datum v = values[attnum];
+ bool b = isnull[attnum];
+ values[attnum] = fastgetattr(tuple, attnum + 1, tupleDesc,
&isnull[attnum]);
+ Assert(v == values[attnum]);
+ Assert(b == isnull[attnum]);
+ }
+
/* Fetch any missing attrs and raise an error if reqnatts is invalid */
if (unlikely(attnum < reqnatts))
Still considering the best way to get a bit more coverage more permanently.
David
^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: More speedups for tuple deformation
@ 2026-03-19 12:39 Tender Wang <[email protected]>
parent: David Rowley <[email protected]>
0 siblings, 1 reply; 30+ messages in thread
From: Tender Wang @ 2026-03-19 12:39 UTC (permalink / raw)
To: David Rowley <[email protected]>; +Cc: Andres Freund <[email protected]>; John Naylor <[email protected]>; PostgreSQL Developers <[email protected]>
Hi David,
David Rowley <[email protected]> 于2026年3月17日周二 05:15写道：
>
> On Mon, 16 Mar 2026 at 22:17, David Rowley <[email protected]> wrote:
> >
> > On Mon, 16 Mar 2026 at 20:01, Tender Wang <[email protected]> wrote:
> > > SELECT * FROM t2, t4 RIGHT OUTER JOIN t0 ON t4.c0 WHERE t4.c1 ORDER BY
> >
> > > server closed the connection unexpectedly
> >
> > Thanks. Looks like I didn't get the startAttr logic correct in
> > nocachegetattr(). Starting by using the attcacheoff of the first NULL
> > attribute isn't valid. It should be the attribute prior to that one.
> >
> > I'm just verifying some code locally now.
>
> Now pushed. Some concerns about the lack of exercise in the tests for
> fastgetattr() for user tables. I ended up testing by injecting the
> following into slot_deform_heap_tuple()
I encountered a crash when doing SQLSmith on TPCH (s=1) test.
How to reproduce:
1. prepare TPCH s =1 data.
2. Execute the following query
postgres=# update public.region set
r_comment = public.region.r_comment
returning
5 as c0,
pg_catalog.bittypmodout(
cast(public.region.r_regionkey as int4)) as c1;
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
The connection to the server was lost. Attempting reset: Succeeded.
(gdb) bt
#0 __pthread_kill_implementation (no_tid=0, signo=6,
threadid=140696004203456) at ./nptl/pthread_kill.c:44
#1 __pthread_kill_internal (signo=6, threadid=140696004203456) at
./nptl/pthread_kill.c:78
#2 __GI___pthread_kill (threadid=140696004203456,
signo=signo@entry=6) at ./nptl/pthread_kill.c:89
#3 0x00007ff659450476 in __GI_raise (sig=sig@entry=6) at
../sysdeps/posix/raise.c:26
#4 0x00007ff6594367f3 in __GI_abort () at ./stdlib/abort.c:79
#5 0x000055d8ae8cad94 in ExceptionalCondition
(conditionName=0x55d8aeac5beb "attlen > 0 || attlen == -1",
fileName=0x55d8aeac5a18 "execTuples.c", lineNumber=1197) at
assert.c:65
#6 0x000055d8ae3afa2e in slot_deform_heap_tuple (slot=0x55d8cabf1a58,
tuple=0x55d8cabf1ab0, offp=0x55d8cabf1ac8, reqnatts=2) at
execTuples.c:1197
#7 0x000055d8ae3ae853 in tts_minimal_getsomeattrs
(slot=0x55d8cabf1a58, natts=2) at execTuples.c:550
#8 0x000055d8ae06ac3b in slot_getsomeattrs (slot=0x55d8cabf1a58,
attnum=2) at ../../../../src/include/executor/tuptable.h:380
#9 0x000055d8ae06ac62 in slot_getallattrs (slot=0x55d8cabf1a58) at
../../../../src/include/executor/tuptable.h:392
#10 0x000055d8ae06b5f5 in printtup (slot=0x55d8cabf1a58,
self=0x55d8cab11828) at printtup.c:319
#11 0x000055d8ae6bfd82 in RunFromStore (portal=0x55d8cab860a0,
direction=ForwardScanDirection, count=0, dest=0x55d8cab11828) at
pquery.c:1089
#12 0x000055d8ae6bf88b in PortalRunSelect (portal=0x55d8cab860a0,
forward=true, count=0, dest=0x55d8cab11828) at pquery.c:912
#13 0x000055d8ae6bf4ea in PortalRun (portal=0x55d8cab860a0,
count=9223372036854775807, isTopLevel=true, dest=0x55d8cab11828,
altdest=0x55d8cab11828, qc=0x7ffe1b92e4f0) at pquery.c:760
#14 0x000055d8ae6b7c2b in exec_simple_query (
query_string=0x55d8caadb4b0 "update public.region set \n
r_comment = public.region.r_comment\n returning \n 5
as c0, \n pg_catalog.bittypmodout(\n", ' ' <repeats 12
times>, "cast(public.region.r_regionkey as int4)) a"...)
at postgres.c:1278
#15 0x000055d8ae6bd47b in PostgresMain (dbname=0x55d8cab218d0
"postgres", username=0x55d8cab218b8 "ubuntu") at postgres.c:4810
#16 0x000055d8ae6b3068 in BackendMain (startup_data=0x7ffe1b92e7a0,
startup_data_len=24) at backend_startup.c:124
#17 0x000055d8ae5a17a6 in postmaster_child_launch
(child_type=B_BACKEND, child_slot=2, startup_data=0x7ffe1b92e7a0,
startup_data_len=24, client_sock=0x7ffe1b92e800) at
launch_backend.c:268
#18 0x000055d8ae5a8290 in BackendStartup (client_sock=0x7ffe1b92e800)
at postmaster.c:3606
#19 0x000055d8ae5a57d2 in ServerLoop () at postmaster.c:1713
#20 0x000055d8ae5a509c in PostmasterMain (argc=3, argv=0x55d8caad5b30)
at postmaster.c:1403
#21 0x000055d8ae430b37 in main (argc=3, argv=0x55d8caad5b30) at main.c:231
(gdb) f 6
#6 0x000055d8ae3afa2e in slot_deform_heap_tuple (slot=0x55d8cabf1a58,
tuple=0x55d8cabf1ab0, offp=0x55d8cabf1ac8, reqnatts=2) at
execTuples.c:1197
1197 pg_assume(attlen > 0 || attlen == -1);
(gdb) p attlen
$1 = -2
Please take a look.
--
Thanks,
Tender Wang
^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: More speedups for tuple deformation
@ 2026-03-19 12:56 David Rowley <[email protected]>
parent: Tender Wang <[email protected]>
0 siblings, 1 reply; 30+ messages in thread
From: David Rowley @ 2026-03-19 12:56 UTC (permalink / raw)
To: Tender Wang <[email protected]>; +Cc: Andres Freund <[email protected]>; John Naylor <[email protected]>; PostgreSQL Developers <[email protected]>
On Fri, 20 Mar 2026, 01:39 Tender Wang, <[email protected]> wrote:
> Hi David,
>
> David Rowley <[email protected]> 于2026年3月17日周二 05:15写道：
> >
> > On Mon, 16 Mar 2026 at 22:17, David Rowley <[email protected]> wrote:
> > >
> > > On Mon, 16 Mar 2026 at 20:01, Tender Wang <[email protected]> wrote:
> postgres=# update public.region set
> r_comment = public.region.r_comment
> returning
> 5 as c0,
> pg_catalog.bittypmodout(
> cast(public.region.r_regionkey as int4)) as c1;
> server closed the connection unexpectedly
> This probably means the server terminated abnormally
> before or while processing the request.
> The connection to the server was lost. Attempting reset: Succeeded.
>
> (gdb) bt
> 1197 pg_assume(attlen > 0 || attlen == -1);
> (gdb) p attlen
> $1 = -2
>
Thanks for the report. I'll look in detail in the morning when I'm at my
computer again. I guess i'll need to add an extra parameter (that will be
constant folded away during the inlining) to the deformed function to
specify if cstrings can exist in the tuple, which seemingly needs to be
true when deforming minimal tuples. I'd rather not lose that optimisation
with heap tuples.
David
^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: More speedups for tuple deformation
@ 2026-03-20 01:18 David Rowley <[email protected]>
parent: David Rowley <[email protected]>
0 siblings, 0 replies; 30+ messages in thread
From: David Rowley @ 2026-03-20 01:18 UTC (permalink / raw)
To: Tender Wang <[email protected]>; +Cc: Andres Freund <[email protected]>; John Naylor <[email protected]>; PostgreSQL Developers <[email protected]>
On Fri, 20 Mar 2026 at 01:56, David Rowley <[email protected]> wrote:
>
> On Fri, 20 Mar 2026, 01:39 Tender Wang, <[email protected]> wrote:
>> 1197 pg_assume(attlen > 0 || attlen == -1);
>> (gdb) p attlen
>> $1 = -2
>
>
> Thanks for the report. I'll look in detail in the morning when I'm at my computer again. I guess i'll need to add an extra parameter (that will be constant folded away during the inlining) to the deformed function to specify if cstrings can exist in the tuple, which seemingly needs to be true when deforming minimal tuples. I'd rather not lose that optimisation with heap tuples.
I've pushed a fix for this. A more simple recreator was:
with cte as materialized (select relname::cstring as relname from
pg_class) select relname from cte;
David
^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: More speedups for tuple deformation
@ 2026-04-01 02:00 Alexander Lakhin <[email protected]>
parent: Tender Wang <[email protected]>
1 sibling, 1 reply; 30+ messages in thread
From: Alexander Lakhin @ 2026-04-01 02:00 UTC (permalink / raw)
To: David Rowley <[email protected]>; +Cc: Tender Wang <[email protected]>; Andres Freund <[email protected]>; John Naylor <[email protected]>; PostgreSQL Developers <[email protected]>
Hello David,
16.03.2026 09:01, Tender Wang wrote:
> Hi David,
>
> After c456e391138, I found a crash.
I've also found an assertion failure:
SELECT JSON_ARRAY(VALUES (('', '')));
triggers:
TRAP: failed Assert("attlen > 0 || attlen == -1"), File: "heaptuple.c", Line: 598, PID: 2582495
Could you have a look, please?
Best regards,
Alexander
^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: More speedups for tuple deformation
@ 2026-04-01 03:35 David Rowley <[email protected]>
parent: Alexander Lakhin <[email protected]>
0 siblings, 2 replies; 30+ messages in thread
From: David Rowley @ 2026-04-01 03:35 UTC (permalink / raw)
To: Alexander Lakhin <[email protected]>; +Cc: Tender Wang <[email protected]>; Andres Freund <[email protected]>; John Naylor <[email protected]>; PostgreSQL Developers <[email protected]>
On Wed, 1 Apr 2026 at 15:00, Alexander Lakhin <[email protected]> wrote:
> I've also found an assertion failure:
> SELECT JSON_ARRAY(VALUES (('', '')));
>
> triggers:
> TRAP: failed Assert("attlen > 0 || attlen == -1"), File: "heaptuple.c", Line: 598, PID: 2582495
>
> Could you have a look, please?
Thanks. Looking...
David
^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: More speedups for tuple deformation
@ 2026-04-01 05:13 Junwang Zhao <[email protected]>
parent: David Rowley <[email protected]>
1 sibling, 1 reply; 30+ messages in thread
From: Junwang Zhao @ 2026-04-01 05:13 UTC (permalink / raw)
To: David Rowley <[email protected]>; +Cc: Alexander Lakhin <[email protected]>; Tender Wang <[email protected]>; Andres Freund <[email protected]>; John Naylor <[email protected]>; PostgreSQL Developers <[email protected]>
On Wed, Apr 1, 2026 at 11:35 AM David Rowley <[email protected]> wrote:
>
> On Wed, 1 Apr 2026 at 15:00, Alexander Lakhin <[email protected]> wrote:
> > I've also found an assertion failure:
> > SELECT JSON_ARRAY(VALUES (('', '')));
> >
> > triggers:
> > TRAP: failed Assert("attlen > 0 || attlen == -1"), File: "heaptuple.c", Line: 598, PID: 2582495
> >
> > Could you have a look, please?
>
> Thanks. Looking...
The parser transforms '' (or any cstring) into Const with UNKNOWNOID.
When building the tuple descriptor in ExecTypeFromExprList,
TupleDescInitEntry uses UNKNOWNOID to populate the attribute entry,
thus the failure in nocachegetattr.
Therefore, the assertion should be addressed by removing the pg_assume,
since this case can occur.
Just trying to learn through analysis, might be incorrect. Let's wait for
David's fix.
>
> David
>
>
--
Regards
Junwang Zhao
^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: More speedups for tuple deformation
@ 2026-04-01 13:19 jian he <[email protected]>
parent: Junwang Zhao <[email protected]>
0 siblings, 0 replies; 30+ messages in thread
From: jian he @ 2026-04-01 13:19 UTC (permalink / raw)
To: Junwang Zhao <[email protected]>; +Cc: David Rowley <[email protected]>; Alexander Lakhin <[email protected]>; Tender Wang <[email protected]>; Andres Freund <[email protected]>; John Naylor <[email protected]>; PostgreSQL Developers <[email protected]>
On Wed, Apr 1, 2026 at 1:13 PM Junwang Zhao <[email protected]> wrote:
>
> The parser transforms '' (or any cstring) into Const with UNKNOWNOID.
> When building the tuple descriptor in ExecTypeFromExprList,
> TupleDescInitEntry uses UNKNOWNOID to populate the attribute entry,
> thus the failure in nocachegetattr.
>
> Therefore, the assertion should be addressed by removing the pg_assume,
> since this case can occur.
>
> Just trying to learn through analysis, might be incorrect. Let's wait for
> David's fix.
>
HI.
You can also trigger it via:
SELECT row_to_json(('', ''));
In transformRowExpr, we can coerce these UNKNOWNOID Const to TEXTOID Const.
But I guess it's harder to guarantee that comment "cstrings don't
exist in heap tuples." in nocachegetattr to be true.
--
jian
https://www.enterprisedb.com/
^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: More speedups for tuple deformation
@ 2026-04-02 01:12 David Rowley <[email protected]>
parent: David Rowley <[email protected]>
1 sibling, 0 replies; 30+ messages in thread
From: David Rowley @ 2026-04-02 01:12 UTC (permalink / raw)
To: Alexander Lakhin <[email protected]>; +Cc: Tender Wang <[email protected]>; Andres Freund <[email protected]>; John Naylor <[email protected]>; PostgreSQL Developers <[email protected]>
On Wed, 1 Apr 2026 at 16:35, David Rowley <[email protected]> wrote:
>
> On Wed, 1 Apr 2026 at 15:00, Alexander Lakhin <[email protected]> wrote:
> > I've also found an assertion failure:
> > SELECT JSON_ARRAY(VALUES (('', '')));
> >
> > triggers:
> > TRAP: failed Assert("attlen > 0 || attlen == -1"), File: "heaptuple.c", Line: 598, PID: 2582495
> >
> > Could you have a look, please?
>
> Thanks. Looking...
I've pushed a fix for this. Thanks for the report.
David
^ permalink raw reply [nested|flat] 30+ messages in thread
end of thread, other threads:[~2026-04-02 01:12 UTC | newest]
Thread overview: 30+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-02-03 00:33 Re: More speedups for tuple deformation Andres Freund <[email protected]>
2026-02-03 05:24 ` John Naylor <[email protected]>
2026-02-03 14:41 ` Andres Freund <[email protected]>
2026-02-24 02:23 ` David Rowley <[email protected]>
2026-02-24 08:45 ` Amit Langote <[email protected]>
2026-02-24 12:33 ` David Rowley <[email protected]>
2026-02-24 14:39 ` Andres Freund <[email protected]>
2026-02-25 00:59 ` David Rowley <[email protected]>
2026-02-25 18:05 ` Andres Freund <[email protected]>
2026-02-25 20:29 ` Andres Freund <[email protected]>
2026-03-01 13:10 ` David Rowley <[email protected]>
2026-03-06 04:09 ` David Rowley <[email protected]>
2026-03-07 16:36 ` Junwang Zhao <[email protected]>
2026-03-13 12:18 ` David Rowley <[email protected]>
2026-03-15 13:50 ` Junwang Zhao <[email protected]>
2026-03-16 07:01 ` Tender Wang <[email protected]>
2026-03-16 09:17 ` David Rowley <[email protected]>
2026-03-16 16:37 ` Junwang Zhao <[email protected]>
2026-03-16 21:15 ` David Rowley <[email protected]>
2026-03-19 12:39 ` Tender Wang <[email protected]>
2026-03-19 12:56 ` David Rowley <[email protected]>
2026-03-20 01:18 ` David Rowley <[email protected]>
2026-04-01 02:00 ` Alexander Lakhin <[email protected]>
2026-04-01 03:35 ` David Rowley <[email protected]>
2026-04-01 05:13 ` Junwang Zhao <[email protected]>
2026-04-01 13:19 ` jian he <[email protected]>
2026-04-02 01:12 ` David Rowley <[email protected]>
2026-02-24 18:33 ` Zsolt Parragi <[email protected]>
2026-02-25 00:39 ` David Rowley <[email protected]>
2026-02-25 07:03 ` John Naylor <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox