pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Robert Haas <robertmhaas@gmail.com>
To: Bruce Momjian <bruce@momjian.us>
Cc: Matthew Wakeling <matthew@flymine.org>
Cc: Oleg Bartunov <oleg@sai.msu.su>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Teodor Sigaev <teodor@sigaev.ru>
Cc: Kevin Grittner <Kevin.Grittner@wicourts.gov>
Cc: pgsql-performance@postgresql.org
Subject: Re: GiST index performance
Date: Tue, 2 Mar 2010 11:23:22 -0500
Message-ID: <603c8f071003020823k128f1b85y9240d954293d0602@mail.gmail.com> (raw)
In-Reply-To: <201002252344.o1PNi2007379@momjian.us>
References: <alpine.DEB.2.00.0905071250290.2341@aragorn.flymine.org>
	<201002252344.o1PNi2007379@momjian.us>

On Thu, Feb 25, 2010 at 6:44 PM, Bruce Momjian <bruce@momjian.us> wrote:
> Was this corrected?  I don't see any commits to seg.c.

I don't think this was ever reviewed.

It seems like a good patch but I'd be skeptical of committing it now
unless someone has the time to review it carefully.  If not, let's add
it to the next CF so we don't lose it again.

...Robert

>
> ---------------------------------------------------------------------------
>
> Matthew Wakeling wrote:
>> On Thu, 7 May 2009, Oleg Bartunov wrote:
>> > Did you try Guttman quadratic split algorithm ? We also found linear
>> > split algorithm for Rtree.
>>
>> The existing (bugfixed) seg split algorithm is the Guttman quadratic split
>> algorithm. Guttman did all his work on two-dimensional and above data,
>> dismissing one-dimensional data as being handled adequately by B-trees,
>> which is not true for segment overlaps. It turns out that the algorithm
>> has a weakness with certain types of data, and one-dimensional data is
>> almost certain to exercise that weakness. The greater the number of
>> dimensions, the less the weakness is exercised.
>>
>> The problem is that the algorithm does not calculate a split pivot.
>> Instead it finds two suitable entries, and adds the remaining entries to
>> those two in turn. This can lead to the majority of the entries being
>> added to just one side. In fact, I saw lots of cases where 367 entries
>> were being split into two pages of 366 and one entry.
>>
>> Guttman's linear split algorithm has the same weakness.
>>
>> >> One thing I am seeing is a really big difference in performance between
>> >> Postgres/GiST and a Java implementation I have written, using the same
>> >> algorithms. Postgres takes three minutes to perform a set of index lookups
>> >> while java takes six seconds. The old version of bioseg took an hour. I
>> >> can't see anything in the GiST support code that could account for this.
>> >
>> > is the number of index lookups different, or just index lookup time is very
>> > big ?
>>
>> Same number of index lookups. Same algorithms. I have a set of 681879
>> segments, and I load them all into the index. I then query the index for
>> overlaps for each one in turn. For some reason, GiST lookups seem to be
>> slow, even if they are using a good algorithm. I have seen that problem
>> with btree_gist on integers too. I can't see any reason for this is the
>> GiST code - it all seems pretty tight to me. We probably need to do some
>> profiling.
>>
>> Matthew
>>
>> --
>>  I suppose some of you have done a Continuous Maths course. Yes? Continuous
>>  Maths? <menacing stares from audience> Whoah, it was like that, was it!
>>                                         -- Computer Science Lecturer
>>
>> --
>> Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
>> To make changes to your subscription:
>> http://www.postgresql.org/mailpref/pgsql-performance
>
> --
>  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
>  EnterpriseDB                             http://enterprisedb.com
>  PG East:  http://www.enterprisedb.com/community/nav-pg-east-2010.do
>  + If your life is a hard drive, Christ can be your backup. +
>
> --
> Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance
>



view thread (17+ messages)  latest in thread

Message-ID: <603c8f071003020823k128f1b85y9240d954293d0602@mail.gmail.com>
Permalink:  ../603c8f071003020823k128f1b85y9240d954293d0602@mail.gmail.com/
Also on:    postgresql.org/message-id/603c8f071003020823k128f1b85y9240d954293d0602@mail.gmail.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-performance@postgresql.org
  Cc: robertmhaas@gmail.com, bruce@momjian.us, matthew@flymine.org, oleg@sai.msu.su, tgl@sss.pgh.pa.us, teodor@sigaev.ru, Kevin.Grittner@wicourts.gov
  Subject: Re: GiST index performance
  In-Reply-To: <603c8f071003020823k128f1b85y9240d954293d0602@mail.gmail.com>
  References: <alpine.DEB.2.00.0905071250290.2341@aragorn.flymine.org> <201002252344.o1PNi2007379@momjian.us> <603c8f071003020823k128f1b85y9240d954293d0602@mail.gmail.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox