pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Heikki Linnakangas <hlinnakangas@vmware.com>
To: Tomas Vondra <tv@fuzzy.cz>
Cc: Alexander Korotkov <aekorotkov@gmail.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: GIN improvements part 1: additional information
Date: Tue, 21 Jan 2014 11:35:17 +0200
Message-ID: <52DE3F55.80103@vmware.com> (raw)
In-Reply-To: <52DDD51E.2080408@fuzzy.cz>
References: <CAPpHfdutJEC9+6vWX579afZKzdHKCT2wzDkAEom1XmiXEwF3pw@mail.gmail.com>
	<52A62771.3070800@vmware.com>
	<CAPpHfdu2nXxxNCmtMBBiS+3CRdgF8AHCLt2w9Z4s2kKisJSPuA@mail.gmail.com>
	<52AEE445.1060206@vmware.com>
	<CAPpHfdvK3eZr8iQJtp9ax9k80MKRKkPKgqcc0SB1bn+_KVq-Xg@mail.gmail.com>
	<52AF837C.3040201@vmware.com>
	<52B2F5B0.1030809@vmware.com>
	<52B49C34.8060300@vmware.com>
	<CA+HiwqGO91OEE0ZBa0q2RqFEKm5dy5VzXrf7uoYObw9BCKSEVA@mail.gmail.com>
	<CAPpHfduBDxQ7Tw5bZCCgxmJzNHDRG6eELAuseuCEvKt=dw0Yfw@mail.gmail.com>
	<52D0A947.6050706@fuzzy.cz>
	<CAPpHfdskbPbjWYJhkd-FkZCKEUZd03PRAw05NyT0Hd-jxWOyfg@mail.gmail.com>
	<52D478F2.6070005@fuzzy.cz>
	<52D5E1BC.3000208@fuzzy.cz>
	<CAPpHfdv9Pheu5atEEUk75f_S1nf6vCoRqge2yUQ1v3xgZ4UP3g@mail.gmail.com>
	<CAPpHfds0A03zwmYdhKUg6s01Vq1p30n01=3c-tct264QTpV--Q@mail.gmail.com>
	<52D9789E.3030202@vmware.com>
	<CAPpHfdsYTRWuR5YkadvB_G=JiBRLsYRAL7CJkpPiz3AitvhD6g@mail.gmail .com>
	<52DD6B4B.3050806@vmwar e.com>
	<52DDD51E.2080408@fuzzy.cz>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>

On 01/21/2014 04:02 AM, Tomas Vondra wrote:
> On 20.1.2014 19:30, Heikki Linnakangas wrote:
>>
>> Attached is a yet another version, with more bugs fixed and more
>> comments added and updated. I would appreciate some heavy-testing of
>> this patch now. If you could re-run the tests you've been using,
>> that could be great. I've tested the WAL replay by replicating GIN
>> operations over streaming replication. That doesn't guarantee it's
>> correct, but it's a good smoke test.
>
> I gave it a try - the OOM error seems to be gone, but now get this
>
>     PANIC:  cannot insert duplicate items to GIN index page
>
> This only happens when building the index incrementally (i.e. using a
> sequence of INSERT statements into a table with GIN index). When I
> create a new index on a table (already containing the same dataset) it
> works just fine.
>
> Also, I tried to reproduce the issue by running a simple plpgsql loop
> (instead of a complex python script):
>
> DO LANGUAGE plpgsql $$
> DECLARE
>      r tsvector;
> BEGIN
>      FOR r IN SELECT body_tsvector FROM data_table LOOP
>          INSERT INTO idx_table (body_tsvector) VALUES (r);
>      END LOOP;
> END$$;
>
> where data_table is the table with imported data (the same data I
> mentioned in the post about OOM errors), and index_table is an empty
> table with a GIN index. And indeed it fails, but only if I run the block
> in multiple sessions in parallel.

Oh, I see what's going on. I had assumed that there cannot be duplicate 
insertions into the posting tree, but that's dead wrong. The fast 
insertion mechanism depends on a duplicate insertion to do nothing.

Will fix, thanks for the testing!

- Heikki


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers



view thread (119+ messages)  latest in thread

Message-ID: <52DE3F55.80103@vmware.com>
Permalink:  ../52DE3F55.80103@vmware.com/
Also on:    postgresql.org/message-id/52DE3F55.80103@vmware.com

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: hlinnakangas@vmware.com, tv@fuzzy.cz, aekorotkov@gmail.com
  Subject: Re: GIN improvements part 1: additional information
  In-Reply-To: <52DE3F55.80103@vmware.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox