From: Tomas Vondra <tomas.vondra@postgresql.org>
To: pgsql-committers@lists.postgresql.org
Subject: pgsql: Allow parallel CREATE INDEX for BRIN indexes
Date: Fri, 08 Dec 2023 17:16:27 +0000
Message-ID: <E1rBeSg-009EOz-PJ@gemulon.postgresql.org> (raw)
Allow parallel CREATE INDEX for BRIN indexes
Allow using multiple worker processes to build BRIN index, which until
now was supported only for BTREE indexes. For large tables this often
results in significant speedup when the build is CPU-bound.
The work is split in a simple way - each worker builds BRIN summaries on
a subset of the table, determined by the regular parallel scan used to
read the data, and feeds them into a shared tuplesort which sorts them
by blkno (start of the range). The leader then reads this sorted stream
of ranges, merges duplicates (which may happen if the parallel scan does
not align with BRIN pages_per_range), and adds the resulting ranges into
the index.
The number of duplicate results produced by workers (requiring merging
in the leader process) should be fairly small, thanks to how parallel
scans assign chunks to workers. The likelihood of duplicate results may
increase for higher pages_per_range values, but then there are fewer
page ranges in total. In any case, we expect the merging to be much
cheaper than summarization, so this should be a win.
Most of the parallelism infrastructure is a simplified copy of the code
used by BTREE indexes, omitting the parts irrelevant for BRIN indexes
(e.g. uniqueness checks).
This also introduces a new index AM flag amcanbuildparallel, determining
whether to attempt to start parallel workers for the index build.
Original patch by me, with reviews and substantial reworks by Matthias
van de Meent, certainly enough to make him a co-author.
Author: Tomas Vondra, Matthias van de Meent
Reviewed-by: Matthias van de Meent
Discussion: https://postgr.es/m/c2ee7d69-ce17-43f2-d1a0-9811edbda6e6%40enterprisedb.com
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/b437571714707bc6466abde1a0af5e69aaade09c
Modified Files
--------------
contrib/bloom/blutils.c | 1 +
doc/src/sgml/indexam.sgml | 7 +
src/backend/access/brin/brin.c | 886 ++++++++++++++++++++++-
src/backend/access/gin/ginutil.c | 1 +
src/backend/access/gist/gist.c | 1 +
src/backend/access/hash/hash.c | 1 +
src/backend/access/nbtree/nbtree.c | 1 +
src/backend/access/spgist/spgutils.c | 1 +
src/backend/access/transam/parallel.c | 4 +
src/backend/catalog/index.c | 2 +-
src/backend/utils/sort/tuplesortvariants.c | 207 ++++++
src/include/access/amapi.h | 2 +
src/include/access/brin.h | 3 +
src/include/utils/tuplesort.h | 11 +
src/test/modules/dummy_index_am/dummy_index_am.c | 1 +
src/tools/pgindent/typedefs.list | 5 +
16 files changed, 1118 insertions(+), 16 deletions(-)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-committers@postgresql.org
Cc: tomas.vondra@postgresql.org, pgsql-committers@lists.postgresql.org
Subject: Re: pgsql: Allow parallel CREATE INDEX for BRIN indexes
In-Reply-To: <E1rBeSg-009EOz-PJ@gemulon.postgresql.org>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox