Message-ID: <52DDD51E.2080408@fuzzy.cz>
Date: Tue, 21 Jan 2014 03:02:06 +0100
From: Tomas Vondra <tv@fuzzy.cz>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
 rv:24.0) Gecko/20100101 Thunderbird/24.1.1
MIME-Version: 1.0
To: Heikki Linnakangas <hlinnakangas@vmware.com>,
 Alexander Korotkov <aekorotkov@gmail.com>
CC: pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: GIN improvements part 1: additional information
References: 
 <CAPpHfdutJEC9+6vWX579afZKzdHKCT2wzDkAEom1XmiXEwF3pw@mail.gmail.com>
 <CAPpHfdvKC9HE_D+BYBUpxqyWSE4UD_=ykZgfSxkAv4P3LBFQjg@mail.gmail.com>
 <52A62771.3070800@vmware.com>
 <CAPpHfdu2nXxxNCmtMBBiS+3CRdgF8AHCLt2w9Z4s2kKisJSPuA@mail.gmail.com>
 <52AEE445.1060206@vmware.com>
 <CAPpHfdvK3eZr8iQJtp9ax9k80MKRKkPKgqcc0SB1bn+_KVq-Xg@mail.gmail.com>
 <52AF837C.3040201@vmware.com> <52B2F5B0.1030809@vmware.com>
 <52B49C34.8060300@vmware.com>
 <CA+HiwqGO91OEE0ZBa0q2RqFEKm5dy5VzXrf7uoYObw9BCKSEVA@mail.gmail.com>
 <CAPpHfduBDxQ7Tw5bZCCgxmJzNHDRG6eELAuseuCEvKt=dw0Yfw@mail.gmail.com>
 <52D0A947.6050706@fuzzy.cz>
 <CAPpHfdskbPbjWYJhkd-FkZCKEUZd03PRAw05NyT0Hd-jxWOyfg@mail.gmail.com>
 <52D478F2.6070005@fuzzy.cz> <52D5E1BC.3000208@fuzzy.cz>
 <CAPpHfdv9Pheu5atEEUk75f_S1nf6vCoRqge2yUQ1v3xgZ4UP3g@mail.gmail.com>
 <CAPpHfds0A03zwmYdhKUg6s01Vq1p30n01=3c-tct264QTpV--Q@mail.gmail.com>
 <52D9789E.3030202@vmware.com>
 <CAPpHfdsYTRWuR5YkadvB_G=JiBRLsYRAL7CJkpPiz3AitvhD6g@mail.gmail .com>
 <52DD6B4B.3050806@vmwar e.com>
In-Reply-To: <52DD6B4B.3050806@vmware.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org

On 20.1.2014 19:30, Heikki Linnakangas wrote:
> 
> Attached is a yet another version, with more bugs fixed and more 
> comments added and updated. I would appreciate some heavy-testing of 
> this patch now. If you could re-run the tests you've been using,
> that could be great. I've tested the WAL replay by replicating GIN
> operations over streaming replication. That doesn't guarantee it's
> correct, but it's a good smoke test.

I gave it a try - the OOM error seems to be gone, but now get this

   PANIC:  cannot insert duplicate items to GIN index page

This only happens when building the index incrementally (i.e. using a
sequence of INSERT statements into a table with GIN index). When I
create a new index on a table (already containing the same dataset) it
works just fine.

Also, I tried to reproduce the issue by running a simple plpgsql loop
(instead of a complex python script):

DO LANGUAGE plpgsql $$
DECLARE
    r tsvector;
BEGIN
    FOR r IN SELECT body_tsvector FROM data_table LOOP
        INSERT INTO idx_table (body_tsvector) VALUES (r);
    END LOOP;
END$$;

where data_table is the table with imported data (the same data I
mentioned in the post about OOM errors), and index_table is an empty
table with a GIN index. And indeed it fails, but only if I run the block
in multiple sessions in parallel.

regards
Tomas


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers