Received: from maia.hub.org (maia-3.hub.org [200.46.204.243]) by mail.postgresql.org (Postfix) with ESMTP id 63B8AB5DBBF for ; Wed, 13 Jul 2011 10:59:54 -0300 (ADT) Received: from mail.postgresql.org ([200.46.204.86]) by maia.hub.org (mx1.hub.org [200.46.204.243]) (amavisd-maia, port 10024) with ESMTP id 09129-02 for ; Wed, 13 Jul 2011 13:59:48 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from exprod7og118.obsmtp.com (exprod7og118.obsmtp.com [64.18.2.8]) by mail.postgresql.org (Postfix) with ESMTP id 5834FB5DBBE for ; Wed, 13 Jul 2011 10:59:46 -0300 (ADT) Received: from mail-fx0-f48.google.com ([209.85.161.48]) (using TLSv1) by exprod7ob118.postini.com ([64.18.6.12]) with SMTP ID DSNKTh2k0lLZXpO7+amtAvKrCFqE/GeWW3qZ@postini.com; Wed, 13 Jul 2011 06:59:48 PDT Received: by fxg7 with SMTP id 7so4310491fxg.35 for ; Wed, 13 Jul 2011 06:59:44 -0700 (PDT) Received: by 10.223.1.201 with SMTP id 9mr1751916fag.91.1310565582520; Wed, 13 Jul 2011 06:59:42 -0700 (PDT) Received: from [192.168.1.183] (dsl-hkibrasgw2-ff7ac300-240.dhcp.inet.fi [88.195.122.240]) by mx.google.com with ESMTPS id l22sm2092186fam.9.2011.07.13.06.59.40 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 13 Jul 2011 06:59:41 -0700 (PDT) Message-ID: <4E1DA4CB.9020209@enterprisedb.com> Date: Wed, 13 Jul 2011 16:59:39 +0300 From: Heikki Linnakangas User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; fi-FI; rv:1.9.2.18) Gecko/20110626 Icedove/3.1.11 MIME-Version: 1.0 To: Alexander Korotkov CC: pgsql-hackers Subject: Re: WIP: Fast GiST index build References: <4DF8676F.9040801@enterprisedb.com> <4DFA4AE5.1010806@enterprisedb.com> <4E044D68.5070605@enterprisedb.com> <4E0894FF.8080202@enterprisedb.com> <26338.1310134685@sss.pgh.pa.us> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=-1.9 tagged_above=-5 required=5 tests=BAYES_00=-1.9 X-Spam-Level: X-Archive-Number: 201107/660 X-Sequence-Number: 191759 On 12.07.2011 11:34, Alexander Korotkov wrote: > New version of patch with a little more refactoring and comments. Great! The README helps tremendously to understand this, thanks for that. One thing that caught my eye is that when you empty a buffer, you load the entire subtree below that buffer, down to the next buffered or leaf level, into memory. Every page in that subtree is kept pinned. That is a problem; in the general case, the buffer manager can only hold a modest number of pages pinned at a time. Consider that the minimum value for shared_buffers is just 16. That's unrealistically low for any real system, but the default is only 32MB, which equals to just 4096 buffers. A subtree could easily be larger than that. I don't think you're benefiting at all from the buffering that BufFile does for you, since you're reading/writing a full block at a time anyway. You might as well use the file API in fd.c directly, ie. OpenTemporaryFile/FileRead/FileWrite. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com