pg.ddx.io pgsql-docs@postgresql.org mailing list archive
help / color / mirror / Atom feedParallel index build for BRIN
7+ messages / 2 participants
[nested] [flat]
Commits from this thread:
2024-12-17 8cd44db42a 42eae257cf Update comments about index parallel builds
2024-12-16 0cc7da4e23 f5dec4e861 doc: Mention BRIN indexes support parallel builds
* Parallel index build for BRIN
@ 2024-11-05 09:12 Egor Rogov <e.rogov@postgrespro.ru>
0 siblings, 1 reply; 7+ messages in thread
From: Egor Rogov @ 2024-11-05 09:12 UTC (permalink / raw)
To: pgsql-docs@lists.postgresql.org
Hi,
Commit b4375717 introduced parallel CREATE INDEX for BRIN. I've noticed
that a couple of documentation pages need to be updated accordingly. A
small patch is attached.
Thanks,
Egor
Attachments:
0001-Doc-parallel-CREATE-INDEX-is-allowed-for-BRIN.patch (text/plain, 2.0K) download
From 50dd542f113b915d52f47347efb6d7835bc6bb40 Mon Sep 17 00:00:00 2001
From: Egor Rogov <e.rogov@postgrespro.ru>
Date: Tue, 5 Nov 2024 11:52:15 +0300
Subject: [PATCH] Doc: parallel CREATE INDEX is allowed for BRIN
Commit b4375717 introduced parallel index build for BRIN,
but missed updating the documentation about it.
---
doc/src/sgml/config.sgml | 2 +-
doc/src/sgml/ref/create_index.sgml | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index d54f9049569..b5b1580dee7 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -2835,7 +2835,7 @@ include_dir 'conf.d'
Sets the maximum number of parallel workers that can be
started by a single utility command. Currently, the parallel
utility commands that support the use of parallel workers are
- <command>CREATE INDEX</command> only when building a B-tree index,
+ <command>CREATE INDEX</command> when building a B-tree or BRIN index,
and <command>VACUUM</command> without <literal>FULL</literal>
option. Parallel workers are taken from the pool of processes
established by <xref linkend="guc-max-worker-processes"/>, limited
diff --git a/doc/src/sgml/ref/create_index.sgml b/doc/src/sgml/ref/create_index.sgml
index 621bc0e253c..208389e8006 100644
--- a/doc/src/sgml/ref/create_index.sgml
+++ b/doc/src/sgml/ref/create_index.sgml
@@ -808,7 +808,7 @@ Indexes:
leveraging multiple CPUs in order to process the table rows faster.
This feature is known as <firstterm>parallel index
build</firstterm>. For index methods that support building indexes
- in parallel (currently, only B-tree),
+ in parallel (currently, B-tree and BRIN),
<varname>maintenance_work_mem</varname> specifies the maximum
amount of memory that can be used by each index build operation as
a whole, regardless of how many worker processes were started.
--
2.37.1 (Apple Git-137.1)
* Re: Parallel index build for BRIN
@ 2024-11-17 08:28 Egor Rogov <e.rogov@postgrespro.ru>
parent: Egor Rogov <e.rogov@postgrespro.ru>
0 siblings, 1 reply; 7+ messages in thread
From: Egor Rogov @ 2024-11-17 08:28 UTC (permalink / raw)
To: pgsql-docs@lists.postgresql.org
Hi everyone,
Thisthread doesn'tseem to have attractedattention, so let me try again.
Two documentation pages claim that B-tree is the only access method that
supports parallel building, which is no longer true. I propose to fix it
in a way like this:
inline patch 1: doc/src/sgml/config.sgml (+1/-1 lines) download inline-1.patch
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index d54f9049569..b5b1580dee7 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -2835,7 +2835,7 @@ include_dir 'conf.d'
Sets the maximum number of parallel workers that can be
started by a single utility command. Currently, the parallel
utility commands that support the use of parallel workers are
- <command>CREATE INDEX</command> only when building a B-tree index,
+ <command>CREATE INDEX</command> when building a B-tree or BRIN
inline patch 2: doc/src/sgml/ref/create_index.sgml (+1/-1 lines) download inline-2.patch
--- a/doc/src/sgml/ref/create_index.sgml
+++ b/doc/src/sgml/ref/create_index.sgml
@@ -808,7 +808,7 @@ Indexes:
leveraging multiple CPUs in order to process the table rows faster.
This feature is known as <firstterm>parallel index
build</firstterm>. For index methods that support building indexes
- in parallel (currently, only B-tree),
+ in parallel (currently, B-tree and BRIN),
<varname>maintenance_work_mem</varname> specifies the maximum
amount of memory that can be used by each index build operation as
a whole, regardless of how many worker processes were started.
* Re: Parallel index build for BRIN
@ 2024-12-08 15:00 Egor Rogov <e.rogov@postgrespro.ru>
parent: Egor Rogov <e.rogov@postgrespro.ru>
0 siblings, 1 reply; 7+ messages in thread
From: Egor Rogov @ 2024-12-08 15:00 UTC (permalink / raw)
To: pgsql-docs@lists.postgresql.org
Hi,
On 17.11.2024 11:28, Egor Rogov wrote:
> Hi everyone,
>
> This thread doesn't seem to have attracted attention, so let me try
> again. Two documentation pages claim that B-tree is the only access
> method that supports parallel building, which is no longer true. I
> propose to fix it in a way like this:
>
> diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
> index d54f9049569..b5b1580dee7 100644
> --- a/doc/src/sgml/config.sgml
> +++ b/doc/src/sgml/config.sgml
> @@ -2835,7 +2835,7 @@ include_dir 'conf.d'
> Sets the maximum number of parallel workers that can be
> started by a single utility command. Currently, the parallel
> utility commands that support the use of parallel workers are
> - <command>CREATE INDEX</command> only when building a B-tree
> index,
> + <command>CREATE INDEX</command> when building a B-tree or
> BRIN index,
> and <command>VACUUM</command> without <literal>FULL</literal>
> option. Parallel workers are taken from the pool of processes
> established by <xref linkend="guc-max-worker-processes"/>,
> limited
> diff --git a/doc/src/sgml/ref/create_index.sgml
> b/doc/src/sgml/ref/create_index.sgml
> index 621bc0e253c..208389e8006 100644
> --- a/doc/src/sgml/ref/create_index.sgml
> +++ b/doc/src/sgml/ref/create_index.sgml
> @@ -808,7 +808,7 @@ Indexes:
> leveraging multiple CPUs in order to process the table rows faster.
> This feature is known as <firstterm>parallel index
> build</firstterm>. For index methods that support building indexes
> - in parallel (currently, only B-tree),
> + in parallel (currently, B-tree and BRIN),
> <varname>maintenance_work_mem</varname> specifies the maximum
> amount of memory that can be used by each index build operation as
> a whole, regardless of how many worker processes were started.
I've spotted another mention of B-tree being the only AM that supports
parallel builds: comment in src/backend/catalog/index.c. As this mention
is not visible to the users, I'd propose removing it altogether rather
than fixing it. Updated patch is attached.
>
> Thanks,
> Egor
>
>
> On 05.11.2024 12:12, Egor Rogov wrote:
>> Hi,
>>
>> Commit b4375717 introduced parallel CREATE INDEX for BRIN. I've
>> noticed that a couple of documentation pages need to be updated
>> accordingly. A small patch is attached.
>>
>> Thanks,
>> Egor
Attachments:
0001-Doc-and-comment-parallel-CREATE-INDEX-is-allowed-for.patch (text/plain, 2.6K) download
From 625fa1fbdce1746285d00dd26c632845ab775a51 Mon Sep 17 00:00:00 2001
From: Egor Rogov <e.rogov@postgrespro.ru>
Date: Sun, 8 Dec 2024 17:45:19 +0300
Subject: [PATCH] Doc and comment: parallel CREATE INDEX is allowed for BRIN
Commit b4375717 introduced parallel index build for BRIN,
but missed updating the documentation and code comment about it.
---
doc/src/sgml/config.sgml | 2 +-
doc/src/sgml/ref/create_index.sgml | 2 +-
src/backend/catalog/index.c | 3 +--
3 files changed, 3 insertions(+), 4 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index e0c8325a39c..24bd504c213 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -2843,7 +2843,7 @@ include_dir 'conf.d'
Sets the maximum number of parallel workers that can be
started by a single utility command. Currently, the parallel
utility commands that support the use of parallel workers are
- <command>CREATE INDEX</command> only when building a B-tree index,
+ <command>CREATE INDEX</command> when building a B-tree or BRIN index,
and <command>VACUUM</command> without <literal>FULL</literal>
option. Parallel workers are taken from the pool of processes
established by <xref linkend="guc-max-worker-processes"/>, limited
diff --git a/doc/src/sgml/ref/create_index.sgml b/doc/src/sgml/ref/create_index.sgml
index 621bc0e253c..208389e8006 100644
--- a/doc/src/sgml/ref/create_index.sgml
+++ b/doc/src/sgml/ref/create_index.sgml
@@ -808,7 +808,7 @@ Indexes:
leveraging multiple CPUs in order to process the table rows faster.
This feature is known as <firstterm>parallel index
build</firstterm>. For index methods that support building indexes
- in parallel (currently, only B-tree),
+ in parallel (currently, B-tree and BRIN),
<varname>maintenance_work_mem</varname> specifies the maximum
amount of memory that can be used by each index build operation as
a whole, regardless of how many worker processes were started.
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1c3a9e06d37..fc1c2a8d026 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -2988,8 +2988,7 @@ index_build(Relation heapRelation,
Assert(PointerIsValid(indexRelation->rd_indam->ambuildempty));
/*
- * Determine worker process details for parallel CREATE INDEX. Currently,
- * only btree has support for parallel builds.
+ * Determine worker process details for parallel CREATE INDEX.
*
* Note that planner considers parallel safety for us.
*/
--
2.37.1 (Apple Git-137.1)
* Re: Parallel index build for BRIN
@ 2024-12-09 18:54 Tomas Vondra <tomas@vondra.me>
parent: Egor Rogov <e.rogov@postgrespro.ru>
0 siblings, 1 reply; 7+ messages in thread
From: Tomas Vondra @ 2024-12-09 18:54 UTC (permalink / raw)
To: Egor Rogov <e.rogov@postgrespro.ru>; pgsql-docs@lists.postgresql.org
On 12/8/24 16:00, Egor Rogov wrote:
> Hi,
>
>
> On 17.11.2024 11:28, Egor Rogov wrote:
>> Hi everyone,
>>
>> This thread doesn't seem to have attracted attention, so let me try
>> again. Two documentation pages claim that B-tree is the only access
>> method that supports parallel building, which is no longer true. I
>> propose to fix it in a way like this:
>>
>> diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
>> index d54f9049569..b5b1580dee7 100644
>> --- a/doc/src/sgml/config.sgml
>> +++ b/doc/src/sgml/config.sgml
>> @@ -2835,7 +2835,7 @@ include_dir 'conf.d'
>> Sets the maximum number of parallel workers that can be
>> started by a single utility command. Currently, the parallel
>> utility commands that support the use of parallel workers are
>> - <command>CREATE INDEX</command> only when building a B-tree
>> index,
>> + <command>CREATE INDEX</command> when building a B-tree or
>> BRIN index,
>> and <command>VACUUM</command> without <literal>FULL</literal>
>> option. Parallel workers are taken from the pool of processes
>> established by <xref linkend="guc-max-worker-processes"/>,
>> limited
>> diff --git a/doc/src/sgml/ref/create_index.sgml b/doc/src/sgml/ref/
>> create_index.sgml
>> index 621bc0e253c..208389e8006 100644
>> --- a/doc/src/sgml/ref/create_index.sgml
>> +++ b/doc/src/sgml/ref/create_index.sgml
>> @@ -808,7 +808,7 @@ Indexes:
>> leveraging multiple CPUs in order to process the table rows faster.
>> This feature is known as <firstterm>parallel index
>> build</firstterm>. For index methods that support building indexes
>> - in parallel (currently, only B-tree),
>> + in parallel (currently, B-tree and BRIN),
>> <varname>maintenance_work_mem</varname> specifies the maximum
>> amount of memory that can be used by each index build operation as
>> a whole, regardless of how many worker processes were started.
>
>
> I've spotted another mention of B-tree being the only AM that supports
> parallel builds: comment in src/backend/catalog/index.c. As this mention
> is not visible to the users, I'd propose removing it altogether rather
> than fixing it. Updated patch is attached.
>
Thanks for noticing this and the patches. You're right, this should have
been updated with the BRIN parallel builds. I'll get this committed
sometime the week.
regards
--
Tomas Vondra
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Parallel index build for BRIN
@ 2024-12-16 18:24 Tomas Vondra <tomas@vondra.me>
parent: Tomas Vondra <tomas@vondra.me>
0 siblings, 1 reply; 7+ messages in thread
From: Tomas Vondra @ 2024-12-16 18:24 UTC (permalink / raw)
To: Egor Rogov <e.rogov@postgrespro.ru>; pgsql-docs@lists.postgresql.org
On 12/9/24 19:54, Tomas Vondra wrote:
> On 12/8/24 16:00, Egor Rogov wrote:
>> Hi,
>>
>> ...
>>
>> I've spotted another mention of B-tree being the only AM that supports
>> parallel builds: comment in src/backend/catalog/index.c. As this mention
>> is not visible to the users, I'd propose removing it altogether rather
>> than fixing it. Updated patch is attached.
>>
>
> Thanks for noticing this and the patches. You're right, this should have
> been updated with the BRIN parallel builds. I'll get this committed
> sometime the week.
>
I've pushed the doc fix, and backpatched it to PG 17.
Thanks for the patience!
--
Tomas Vondra
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Parallel index build for BRIN
@ 2024-12-17 06:40 Egor Rogov <e.rogov@postgrespro.ru>
parent: Tomas Vondra <tomas@vondra.me>
0 siblings, 1 reply; 7+ messages in thread
From: Egor Rogov @ 2024-12-17 06:40 UTC (permalink / raw)
To: Tomas Vondra <tomas@vondra.me>; pgsql-docs@lists.postgresql.org
On 16.12.2024 21:24, Tomas Vondra wrote:
> On 12/9/24 19:54, Tomas Vondra wrote:
>> On 12/8/24 16:00, Egor Rogov wrote:
>>> Hi,
>>>
>>> ...
>>>
>>> I've spotted another mention of B-tree being the only AM that supports
>>> parallel builds: comment in src/backend/catalog/index.c. As this mention
>>> is not visible to the users, I'd propose removing it altogether rather
>>> than fixing it. Updated patch is attached.
>>>
>> Thanks for noticing this and the patches. You're right, this should have
>> been updated with the BRIN parallel builds. I'll get this committed
>> sometime the week.
>>
> I've pushed the doc fix, and backpatched it to PG 17.
Thanks so much, Tomas!
Please note that the comment in src/backend/catalog/index.c remains
unchanged:
inline patch 1: src/backend/catalog/index.c (+0/-0 lines) download inline-1.patch
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -2988,8 +2988,7 @@ index_build(Relation heapRelation,
* Re: Parallel index build for BRIN
@ 2024-12-17 14:52 Tomas Vondra <tomas@vondra.me>
parent: Egor Rogov <e.rogov@postgrespro.ru>
0 siblings, 0 replies; 7+ messages in thread
From: Tomas Vondra @ 2024-12-17 14:52 UTC (permalink / raw)
To: Egor Rogov <e.rogov@postgrespro.ru>; pgsql-docs@lists.postgresql.org
On 12/17/24 07:40, Egor Rogov wrote:
> On 16.12.2024 21:24, Tomas Vondra wrote:
>
>> On 12/9/24 19:54, Tomas Vondra wrote:
>>> On 12/8/24 16:00, Egor Rogov wrote:
>>>> Hi,
>>>>
>>>> ...
>>>>
>>>> I've spotted another mention of B-tree being the only AM that supports
>>>> parallel builds: comment in src/backend/catalog/index.c. As this
>>>> mention
>>>> is not visible to the users, I'd propose removing it altogether rather
>>>> than fixing it. Updated patch is attached.
>>>>
>>> Thanks for noticing this and the patches. You're right, this should have
>>> been updated with the BRIN parallel builds. I'll get this committed
>>> sometime the week.
>>>
>> I've pushed the doc fix, and backpatched it to PG 17.
>
>
> Thanks so much, Tomas!
>
> Please note that the comment in src/backend/catalog/index.c remains
> unchanged:
>
>
> --- a/src/backend/catalog/index.c
> +++ b/src/backend/catalog/index.c
> @@ -2988,8 +2988,7 @@ index_build(Relation heapRelation,
> Assert(PointerIsValid(indexRelation->rd_indam->ambuildempty));
>
> /*
> - * Determine worker process details for parallel CREATE INDEX.
> Currently,
> - * only btree has support for parallel builds.
> + * Determine worker process details for parallel CREATE INDEX.
> *
> * Note that planner considers parallel safety for us.
> */
>
Oh, thanks for noticing that. I'm not sure why I missed that, I must
have used the first patch by mistake.
Anyway, I found another obsolete comment in planner.c, so I fixed that
too and pushed.
Thanks
--
Tomas Vondra
^ permalink raw reply [nested|flat] 7+ messages in thread
end of thread, other threads:[~2024-12-17 14:52 UTC | newest]
Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2024-11-05 09:12 Parallel index build for BRIN Egor Rogov <e.rogov@postgrespro.ru>
2024-11-17 08:28 ` Egor Rogov <e.rogov@postgrespro.ru>
2024-12-08 15:00 ` Egor Rogov <e.rogov@postgrespro.ru>
2024-12-09 18:54 ` Tomas Vondra <tomas@vondra.me>
2024-12-16 18:24 ` Tomas Vondra <tomas@vondra.me>
2024-12-17 06:40 ` Egor Rogov <e.rogov@postgrespro.ru>
2024-12-17 14:52 ` Tomas Vondra <tomas@vondra.me>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox