From: Tomas Vondra <tomas.vondra@enterprisedb.com>
To: Andy Fan <zhihuifan1213@163.com>
Cc: pgsql-hackers@lists.postgresql.org
Subject: Re: Parallel CREATE INDEX for GIN indexes
Date: Mon, 13 May 2024 11:19:08 +0200
Message-ID: <f902c19e-efcf-45b3-9d1e-3669658ec6ff@enterprisedb.com> (raw)
In-Reply-To: <87y18ektdn.fsf@163.com>
References: <6ab4003f-a8b8-4d75-a67f-f25ad98582dc@enterprisedb.com>
<87pltvmgdm.fsf@163.com>
<3b721981-6fa3-4698-a9b6-70b2d8e8fa3b@enterprisedb.com>
<87y18ektdn.fsf@163.com>
On 5/13/24 10:19, Andy Fan wrote:
>
> Tomas Vondra <tomas.vondra@enterprisedb.com> writes:
>
>> ...
>>
>> I don't understand the question. The blocks are distributed to workers
>> by the parallel table scan, and it certainly does not do that block by
>> block. But even it it did, that's not a problem for this code.
>
> OK, I get ParallelBlockTableScanWorkerData.phsw_chunk_size is designed
> for this.
>
>> The problem is that if the scan wraps around, then one of the TID lists
>> for a given worker will have the min TID and max TID, so it will overlap
>> with every other TID list for the same key in that worker. And when the
>> worker does the merging, this list will force a "full" merge sort for
>> all TID lists (for that key), which is very expensive.
>
> OK.
>
> Thanks for all the answers, they are pretty instructive!
>
Thanks for the questions, it forces me to articulate the arguments more
clearly. I guess it'd be good to put some of this into a README or at
least a comment at the beginning of gininsert.c or somewhere close.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: tomas.vondra@enterprisedb.com, zhihuifan1213@163.com, pgsql-hackers@lists.postgresql.org
Subject: Re: Parallel CREATE INDEX for GIN indexes
In-Reply-To: <f902c19e-efcf-45b3-9d1e-3669658ec6ff@enterprisedb.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox