pg.ddx.io pgsql-committers@postgresql.org mailing list archive
help / color / mirror / Atom feedpgsql: Introduce CompactAttribute array in TupleDesc
5+ messages / 3 participants
[nested] [flat]
Commits from this thread:
2024-12-03 4171c44c9b Revert "Introduce CompactAttribute array in TupleDesc"
* pgsql: Introduce CompactAttribute array in TupleDesc
@ 2024-12-03 03:51 David Rowley <drowley@postgresql.org>
0 siblings, 1 reply; 5+ messages in thread
From: David Rowley @ 2024-12-03 03:51 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Introduce CompactAttribute array in TupleDesc
The new compact_attrs array stores a few select fields from
FormData_pg_attribute in a more compact way, using only 16 bytes per
column instead of the 104 bytes that FormData_pg_attribute uses. Using
CompactAttribute allows performance-critical operations such as tuple
deformation to be performed without looking at the FormData_pg_attribute
element in TupleDesc which means fewer cacheline accesses. With this
change, NAMEDATALEN could be increased with a much smaller negative impact
on performance.
For some workloads, tuple deformation can be the most CPU intensive part
of processing the query. Some testing with 16 columns on a table
where the first column is variable length showed around a 10% increase in
transactions per second for an OLAP type query performing aggregation on
the 16th column. However, in certain cases, the increases were much
higher, up to ~25% on one AMD Zen4 machine.
This also makes pg_attribute.attcacheoff redundant. A follow-on commit
will remove it, thus shrinking the FormData_pg_attribute struct by 4
bytes.
Author: David Rowley
Discussion: https://postgr.es/m/CAApHDvrBztXP3yx=NKNmo3xwFAFhEdyPnvrDg3=M0RhDs+4vYw@mail.gmail.com
Reviewed-by: Andres Freund, Victor Yegorov
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/d28dff3f6cd6a7562fb2c211ac0fb74a33ffd032
Modified Files
--------------
src/backend/access/common/heaptuple.c | 53 ++++++++++---------
src/backend/access/common/indextuple.c | 26 ++++-----
src/backend/access/common/tupdesc.c | 70 ++++++++++++++++++++----
src/backend/access/spgist/spgutils.c | 4 +-
src/backend/catalog/index.c | 2 +
src/backend/commands/tablecmds.c | 4 ++
src/backend/executor/execTuples.c | 4 +-
src/backend/utils/cache/relcache.c | 21 +++++---
src/backend/utils/cache/typcache.c | 13 +++++
src/include/access/htup_details.h | 4 +-
src/include/access/itup.h | 10 ++--
src/include/access/tupdesc.h | 97 ++++++++++++++++++++++++++++++++--
src/include/access/tupmacs.h | 5 +-
src/tools/pgindent/typedefs.list | 1 +
14 files changed, 246 insertions(+), 68 deletions(-)
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: pgsql: Introduce CompactAttribute array in TupleDesc
@ 2024-12-03 04:10 David Rowley <dgrowleyml@gmail.com>
parent: David Rowley <drowley@postgresql.org>
0 siblings, 1 reply; 5+ messages in thread
From: David Rowley @ 2024-12-03 04:10 UTC (permalink / raw)
To: David Rowley <drowley@postgresql.org>; +Cc: pgsql-committers@lists.postgresql.org
On Tue, 3 Dec 2024 at 16:52, David Rowley <drowley@postgresql.org> wrote:
>
> Introduce CompactAttribute array in TupleDesc
I am looking at the buildfarm. It's fairly apparent that I've broken
something. I'm planning on reverting this shortly.
David
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: pgsql: Introduce CompactAttribute array in TupleDesc
@ 2024-12-03 04:55 Tom Lane <tgl@sss.pgh.pa.us>
parent: David Rowley <dgrowleyml@gmail.com>
0 siblings, 1 reply; 5+ messages in thread
From: Tom Lane @ 2024-12-03 04:55 UTC (permalink / raw)
To: David Rowley <dgrowleyml@gmail.com>; +Cc: pgsql-committers@lists.postgresql.org
David Rowley <dgrowleyml@gmail.com> writes:
> I am looking at the buildfarm. It's fairly apparent that I've broken
> something. I'm planning on reverting this shortly.
Just a gut-level guess, but I'm wondering about something like
uninitialized fields that happen to work as long as the memory
was zero to start with. A run under valgrind might offer insight.
regards, tom lane
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: pgsql: Introduce CompactAttribute array in TupleDesc
@ 2024-12-04 12:45 David Rowley <dgrowleyml@gmail.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 0 replies; 5+ messages in thread
From: David Rowley @ 2024-12-04 12:45 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-committers@lists.postgresql.org
On Tue, 3 Dec 2024 at 17:55, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Just a gut-level guess, but I'm wondering about something like
> uninitialized fields that happen to work as long as the memory
> was zero to start with. A run under valgrind might offer insight.
Thanks. I've written about the cause of this in [1].
David
[1] https://postgr.es/m/CAApHDvq%3DGXX%2Bf75AqSGX4JGvtCAXydsOWzNdOSnQNjxwtnG-Aw%40mail.gmail.com
^ permalink raw reply [nested|flat] 5+ messages in thread
* pgsql: Introduce CompactAttribute array in TupleDesc, take 2
@ 2024-12-20 09:32 David Rowley <drowley@postgresql.org>
0 siblings, 0 replies; 5+ messages in thread
From: David Rowley @ 2024-12-20 09:32 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Introduce CompactAttribute array in TupleDesc, take 2
The new compact_attrs array stores a few select fields from
FormData_pg_attribute in a more compact way, using only 16 bytes per
column instead of the 104 bytes that FormData_pg_attribute uses. Using
CompactAttribute allows performance-critical operations such as tuple
deformation to be performed without looking at the FormData_pg_attribute
element in TupleDesc which means fewer cacheline accesses.
For some workloads, tuple deformation can be the most CPU intensive part
of processing the query. Some testing with 16 columns on a table
where the first column is variable length showed around a 10% increase in
transactions per second for an OLAP type query performing aggregation on
the 16th column. However, in certain cases, the increases were much
higher, up to ~25% on one AMD Zen4 machine.
This also makes pg_attribute.attcacheoff redundant. A follow-on commit
will remove it, thus shrinking the FormData_pg_attribute struct by 4
bytes.
Author: David Rowley
Reviewed-by: Andres Freund, Victor Yegorov
Discussion: https://postgr.es/m/CAApHDvrBztXP3yx=NKNmo3xwFAFhEdyPnvrDg3=M0RhDs+4vYw@mail.gmail.com
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/5983a4cffc31640fda6643f10146a5b72b203eaa
Modified Files
--------------
contrib/amcheck/verify_heapam.c | 4 +-
contrib/pageinspect/gistfuncs.c | 4 +-
contrib/pageinspect/heapfuncs.c | 4 +-
contrib/postgres_fdw/postgres_fdw.c | 6 +-
src/backend/access/brin/brin_inclusion.c | 8 +-
src/backend/access/brin/brin_tuple.c | 2 +-
src/backend/access/common/attmap.c | 8 +-
src/backend/access/common/heaptuple.c | 53 +++++++------
src/backend/access/common/indextuple.c | 30 ++++---
src/backend/access/common/tupdesc.c | 118 ++++++++++++++++++++++++----
src/backend/access/gin/ginbulk.c | 4 +-
src/backend/access/gin/ginget.c | 4 +-
src/backend/access/gist/gist.c | 5 +-
src/backend/access/gist/gistbuild.c | 6 +-
src/backend/access/heap/heapam.c | 8 +-
src/backend/access/heap/heapam_handler.c | 2 +-
src/backend/access/heap/heaptoast.c | 6 +-
src/backend/access/nbtree/nbtutils.c | 4 +-
src/backend/access/spgist/spgdoinsert.c | 2 +-
src/backend/access/spgist/spgutils.c | 4 +-
src/backend/access/table/toast_helper.c | 2 +-
src/backend/catalog/index.c | 2 +
src/backend/commands/copy.c | 6 +-
src/backend/commands/tablecmds.c | 4 +
src/backend/executor/execExpr.c | 4 +-
src/backend/executor/execExprInterp.c | 6 +-
src/backend/executor/execJunk.c | 2 +-
src/backend/executor/execTuples.c | 8 +-
src/backend/executor/functions.c | 2 +-
src/backend/executor/nodeMemoize.c | 8 +-
src/backend/executor/nodeModifyTable.c | 4 +-
src/backend/executor/nodeValuesscan.c | 4 +-
src/backend/executor/tstoreReceiver.c | 4 +-
src/backend/jit/llvm/llvmjit_deform.c | 4 +-
src/backend/optimizer/util/plancat.c | 4 +-
src/backend/replication/pgoutput/pgoutput.c | 2 +-
src/backend/utils/adt/expandedrecord.c | 10 +--
src/backend/utils/adt/ri_triggers.c | 2 +-
src/backend/utils/cache/relcache.c | 24 ++++--
src/include/access/htup_details.h | 4 +-
src/include/access/itup.h | 10 ++-
src/include/access/tupdesc.h | 116 +++++++++++++++++++++++++--
src/include/access/tupmacs.h | 13 +--
src/tools/pgindent/typedefs.list | 1 +
44 files changed, 374 insertions(+), 154 deletions(-)
^ permalink raw reply [nested|flat] 5+ messages in thread
end of thread, other threads:[~2024-12-20 09:32 UTC | newest]
Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2024-12-03 03:51 pgsql: Introduce CompactAttribute array in TupleDesc David Rowley <drowley@postgresql.org>
2024-12-03 04:10 ` David Rowley <dgrowleyml@gmail.com>
2024-12-03 04:55 ` Tom Lane <tgl@sss.pgh.pa.us>
2024-12-04 12:45 ` David Rowley <dgrowleyml@gmail.com>
2024-12-20 09:32 pgsql: Introduce CompactAttribute array in TupleDesc, take 2 David Rowley <drowley@postgresql.org>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox