From: David Rowley <drowley@postgresql.org>
To: pgsql-committers@lists.postgresql.org
Subject: pgsql: Optimize WindowAgg's use of tuplestores
Date: Thu, 05 Sep 2024 04:18:44 +0000
Message-ID: <E1sm3xE-000C8E-4E@gemulon.postgresql.org> (raw)
Optimize WindowAgg's use of tuplestores
When WindowAgg finished one partition of a PARTITION BY, it previously
would call tuplestore_end() to purge all the stored tuples before again
calling tuplestore_begin_heap() and carefully setting up all of the
tuplestore read pointers exactly as required for the given frameOptions.
Since the frameOptions don't change between partitions, this part does
not make much sense. For queries that had very few rows per partition,
the overhead of this was very large.
It seems much better to create the tuplestore and the read pointers once
and simply call tuplestore_clear() at the end of each partition.
tuplestore_clear() moves all of the read pointers back to the start
position and deletes all the previously stored tuples.
A simple test query with 1 million partitions and 1 tuple per partition
has been shown to run around 40% faster than without this change. The
additional effort seems to have mostly been spent in malloc/free.
Making this work required adding a new bool field to WindowAggState
which had the unfortunate effect of being the 9th bool field in a group
resulting in the struct being enlarged. Here we shuffle the fields
around a little so that the two bool fields for runcondition relating
stuff fit into existing padding. Also, move the "runcondition" field to
be near those. This frees up enough space with the other bool fields so
that the newly added one fits into the padding bytes. This was done to
address a very small but apparent performance regression with queries
containing a large number of rows per partition.
Reviewed-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Reviewed-by: Tatsuo Ishii <ishii@postgresql.org>
Discussion: https://postgr.es/m/CAHoyFK9n-QCXKTUWT_xxtXninSMEv%2BgbJN66-y6prM3f4WkEHw%40mail.gmail.com
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/908a968612f9ed61911d8ca0a185b262b82f1269
Modified Files
--------------
src/backend/executor/nodeWindowAgg.c | 162 +++++++++++++++++++++++------------
src/include/nodes/execnodes.h | 22 ++---
2 files changed, 118 insertions(+), 66 deletions(-)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-committers@postgresql.org
Cc: drowley@postgresql.org, pgsql-committers@lists.postgresql.org
Subject: Re: pgsql: Optimize WindowAgg's use of tuplestores
In-Reply-To: <E1sm3xE-000C8E-4E@gemulon.postgresql.org>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox