From: Tom Lane <tgl@sss.pgh.pa.us>
To: Alvaro Herrera <alvherre@commandprompt.com>
Cc: Teodor Sigaev <teodor@sigaev.ru>
Cc: Pgsql Hackers <pgsql-hackers@postgresql.org>
Subject: Re: [PATCHES] GIN improvements
Date: Wed, 23 Jul 2008 17:26:49 -0400
Message-ID: <13239.1216848409@sss.pgh.pa.us> (raw)
In-Reply-To: <20080723204430.GD3580@alvh.no-ip.org>
References: <483FD205.4090904@sigaev.ru>
<486BC501.6060605@sigaev.ru>
<27880.1215810535@sss.pgh.pa.us>
<487A6E0C.3090600@sigaev.ru>
<8993.1216843701@sss.pgh.pa.us>
<20080723204430.GD3580@alvh.no-ip.org>
Alvaro Herrera <alvherre@commandprompt.com> writes:
> Tom Lane wrote:
>> It's a mess:
> These are rather severe problems. Maybe there's a better solution, but
> perhaps it would be good enough to lock out concurrent access to the
> index while the bulkinsert procedure is working.
Ugh...
The idea I was toying with was to not allow GIN scans to "stop" on
pending-insertion pages; rather, they should suck out all the matching
tuple IDs into backend-local memory as fast as they can, and then return
the TIDs to the caller one at a time from that internal array. Then,
when the scan is later visiting the main part of the index, it could
check each matching TID against that array to see if it'd already
returned the TID. (So it might be an idea to sort the TID array after
gathering it, to make those subsequent checks fast via binary search.)
This would cost in backend-local memory, of course, but hopefully not
very much. The advantages are the elimination of the deadlock risk
from scan-blocks-insertcleanup-blocks-insert, and fixing the race
condition when a TID previously seen in the pending list is moved to
the main index. There were still a number of locking issues to fix
but I think they're all relatively easy to deal with.
regards, tom lane
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-patches@postgresql.org
Cc: tgl@sss.pgh.pa.us, alvherre@commandprompt.com, teodor@sigaev.ru, pgsql-hackers@postgresql.org
Subject: Re: [PATCHES] GIN improvements
In-Reply-To: <13239.1216848409@sss.pgh.pa.us>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox