From: Teodor Sigaev <teodor@sigaev.ru>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Pgsql Hackers <pgsql-hackers@postgresql.org>
Subject: Re: [PATCHES] GIN improvements
Date: Fri, 31 Oct 2008 16:27:22 +0300
Message-ID: <490B07BA.6030109@sigaev.ru> (raw)
In-Reply-To: <4888A594.6010102@sigaev.ru>
References: <483FD205.4090904@sigaev.ru>
<486BC501.6060605@sigaev.ru>
<27880.1215810535@sss.pgh.pa.us>
<487A6E0C.3090600@sigaev.ru>
<8993.1216843701@sss.pgh.pa.us>
<4888A594.6010102@sigaev.ru>
Reworked version of fast insertion patch for GIN.
>> * shiftList does LockBufferForCleanup, which means that it can be blocked
>> for an indefinitely long time by a concurrent scan, and since it's
>> holding
>> exclusive lock on metapage no new scans or insertions can start
>> meanwhile.
>> This is not only horrid from a performance standpoint but it very
>> probably
>> can result in deadlocks --- which will be deadlocks on LWLocks and thus
>> not even detected by the system.
> Ops, I see possible scenario: UPDATE tbl SET gin_indexed_field = ...
> where gin_indexed_field .... with concurrent shiftList. Will fix. Thank
> you.
Fixed, see below
>> * GIN index scans release lock and pin on one pending-list page before
>> acquiring pin and lock on the next, which means there's a race condition:
>> shiftList could visit and delete the next page before we get to it,
>> because there's a window where we're holding no buffer lock at all.
> Agree, will fix.
Fixed
>> * There is a bigger race condition, which is that after a scan has
>> returned a tuple from a pending page, vacuum could move the index entry
>> into the main index structure, and then that same scan could return that
>> same index entry a second time. This is a no-no, and I don't see any
>> easy
>> fix.
Fixed. TIDBitmap is used for that and for preventing deadlock mentioned above.
TIDBitmap is used for collectiong matched tuples from pending pages and after
that it used as filter for results from regular GIN's scan.
Patch extends TIDBitmap interface by 2 functions:
bool tbm_check_tuple(TIDBitmap *tbm, const ItemPointer tid);
returns true if tid already exists in bitmap
bool tbm_has_lossy(TIDBitmap *tbm);
returns true if bitmap becomes lossy
Also, sequential scan on pending page is replaced to binary search for
performance. It's not a big win but it might improve performance.
--
Teodor Sigaev E-mail: teodor@sigaev.ru
WWW: http://www.sigaev.ru/Attachments:
fast_insert_gin-0.15.gz (application/x-tar, 17.6K) download
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-patches@postgresql.org
Cc: teodor@sigaev.ru, tgl@sss.pgh.pa.us, pgsql-hackers@postgresql.org
Subject: Re: [PATCHES] GIN improvements
In-Reply-To: <490B07BA.6030109@sigaev.ru>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox