Received: from maia.hub.org (maia-2.hub.org [200.46.204.251]) by mail.postgresql.org (Postfix) with ESMTP id 501A8B5DBCE for ; Mon, 27 Jun 2011 11:34:52 -0300 (ADT) Received: from mail.postgresql.org ([200.46.204.86]) by maia.hub.org (mx1.hub.org [200.46.204.251]) (amavisd-maia, port 10024) with ESMTP id 55795-08 for ; Mon, 27 Jun 2011 14:34:45 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from exprod7og124.obsmtp.com (exprod7og124.obsmtp.com [64.18.2.26]) by mail.postgresql.org (Postfix) with ESMTP id 6F448B5D803 for ; Mon, 27 Jun 2011 11:34:45 -0300 (ADT) Received: from mail-ey0-f174.google.com ([209.85.215.174]) (using TLSv1) by exprod7ob124.postini.com ([64.18.6.12]) with SMTP ID DSNKTgiVBbV4nLFA1mgKpRfNXNJ7wc4dYzvS@postini.com; Mon, 27 Jun 2011 07:34:46 PDT Received: by eyx24 with SMTP id 24so1582741eyx.19 for ; Mon, 27 Jun 2011 07:34:43 -0700 (PDT) Received: by 10.14.95.206 with SMTP id p54mr1936989eef.111.1309185283584; Mon, 27 Jun 2011 07:34:43 -0700 (PDT) Received: from [90.190.247.247] (247.247.190.90.dyn.estpak.ee [90.190.247.247]) by mx.google.com with ESMTPS id r39sm4215708eeb.25.2011.06.27.07.34.40 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 27 Jun 2011 07:34:41 -0700 (PDT) Message-ID: <4E0894FF.8080202@enterprisedb.com> Date: Mon, 27 Jun 2011 17:34:39 +0300 From: Heikki Linnakangas User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; fi-FI; rv:1.9.2.17) Gecko/20110606 Icedove/3.1.10 MIME-Version: 1.0 To: Alexander Korotkov CC: pgsql-hackers Subject: Re: WIP: Fast GiST index build References: <4DEC84D7.7040602@enterprisedb.com> <4DECB141.2040707@enterprisedb.com> <4DF8676F.9040801@enterprisedb.com> <4DFA4AE5.1010806@enterprisedb.com> <4E044D68.5070605@enterprisedb.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=-1.9 tagged_above=-5 required=5 tests=BAYES_00=-1.9 X-Spam-Level: X-Archive-Number: 201106/2090 X-Sequence-Number: 190803 On 27.06.2011 13:45, Alexander Korotkov wrote: > I've added information about testing on some real-life dataset to wiki page. > This dataset have a speciality: data is ordered inside it. In this case > tradeoff was inverse in comparison with expectations about "fast build" > algrorithm. Index built is longer but index quality is significantly better. > I think high speed of regular index built is because sequential inserts are > into near tree parts. That's why number of actual page reads and writes is > low. The difference in tree quality I can't *convincingly explain now.* > I've also maked tests with shuffled data of this dataset. In this case > results was similar to random generated data. Once again, interesting results. The penalty function is called whenever a tuple is routed to the next level down, and the final tree has the same depth with and without the patch, so I would expect the number of penalty calls to be roughly the same. But clearly there's something wrong with that logic; can you explain in layman's terms why the patch adds so many gist penalty calls? And how many calls does it actually add, can you gather some numbers on that? Any ides on how to mitigate that, or do we just have to live with it? Or maybe use some heuristic to use the existing insertion method when the patch is not expected to be helpful? -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com