Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.80) (envelope-from ) id 1WLvVI-0001pU-Ig for pgsql-hackers@arkaria.postgresql.org; Fri, 07 Mar 2014 14:12:00 +0000 Received: from localhost ([127.0.0.1] helo=postgresql.org) by malur.postgresql.org with smtp (Exim 4.80) (envelope-from ) id 1WLvVI-0006Ou-31 for pgsql-hackers@arkaria.postgresql.org; Fri, 07 Mar 2014 14:12:00 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtp (Exim 4.80) (envelope-from ) id 1WLvVH-0006Om-5O for pgsql-hackers@postgresql.org; Fri, 07 Mar 2014 14:11:59 +0000 Received: from smtp-outbound-2.vmware.com ([208.91.2.13]) by magus.postgresql.org with esmtp (Exim 4.80) (envelope-from ) id 1WLvVB-0002iW-HS for pgsql-hackers@postgresql.org; Fri, 07 Mar 2014 14:11:57 +0000 Received: from sc9-mailhost3.vmware.com (sc9-mailhost3.vmware.com [10.113.161.73]) by smtp-outbound-2.vmware.com (Postfix) with ESMTP id C0B0428403; Fri, 7 Mar 2014 06:11:50 -0800 (PST) Received: from zcs-prod-ext-mta-1.vmware.com (zcs-prod-ext-mta-1.vmware.com [10.113.62.223]) by sc9-mailhost3.vmware.com (Postfix) with ESMTP id BBDF24039F; Fri, 7 Mar 2014 06:11:50 -0800 (PST) Received: from zcs-prod-ext-mta-1.vmware.com (localhost.localdomain [127.0.0.1]) by zcs-prod-ext-mta-1.vmware.com (Postfix) with ESMTP id 1754A120034; Fri, 7 Mar 2014 06:11:48 -0800 (PST) Received: from [192.168.1.90] (zimbra-prod-ext-proxy-vip.vmware.com [10.113.63.87]) by zcs-prod-ext-mta-1.vmware.com (Postfix) with ESMTPSA; Fri, 7 Mar 2014 06:11:46 -0800 (PST) Message-ID: <5319D3A3.9070209@vmware.com> Date: Fri, 07 Mar 2014 16:11:47 +0200 From: Heikki Linnakangas Organization: VMware User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20131103 Icedove/17.0.10 MIME-Version: 1.0 To: Robert Haas CC: Greg Stark , Tan Tran , PostgreSQL-development Subject: Re: GSoC on WAL-logging hash indexes References: <54977A8C-4AF5-4BCC-B021-78E9CA978077@gmail.com> <531873E8.6050802@vmware.com> <531992AF.2080306@vmware.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Pg-Spam-Score: -6.9 (------) List-Archive: List-Help: List-ID: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-Mailing-List: pgsql-hackers Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org On 03/07/2014 03:48 PM, Robert Haas wrote: > On Fri, Mar 7, 2014 at 4:34 AM, Heikki Linnakangas > wrote: >> >Hmm. You suggested ensuring that a scan always has at least a pin, and split >> >takes a vacuum-lock. That ought to work. There's no need for the more >> >complicated maneuvers you described, ISTM that you can just replace the >> >heavy-weight share lock with holding a pin on the primary page of the >> >bucket, and an exclusive lock with a vacuum-lock. Note that >> >_hash_expandtable already takes the exclusive lock conditionally, ie. if it >> >doesn't get the lock immediately it just gives up. We could do the same with >> >the cleanup lock. > We could try that. I assume you mean do*just* what you describe > here, without the split-in-progress or moved-by-split flags I > suggested. Yep. > The only issue I see with that is that instead of everyone > piling up on the heavyweight lock, a wait which is interruptible, > they'd all pile up on the buffer content lwlock, a wait which isn't. > And splitting a bucket can involve an arbitrary number of I/O > operations, so that's kind of unappealing. Even checkpoints would be > blocked until the bucket split completed, which seems unfortunate. Hmm. I doubt that's a big deal in practice, although I agree it's a bit ugly. Once we solve the crash-safety of splits, we actually have the option of doing the split in many small steps, even when there's no crash involved. You could for example grab the vacuum-lock, move all the tuples in the first 5 pages, and then release the lock to give other backends that are queued up a chance to do their scans/insertions. Then re-acquire the lock, and continue where you left. Or just bail out and let the next vacuum or insertion to finish it later. - Heikki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers