X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id B2F3B32A010 for ; Sat, 9 Oct 2004 21:37:28 +0100 (BST) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 49483-06 for ; Sat, 9 Oct 2004 20:37:16 +0000 (GMT) Received: from rwcrmhc13.comcast.net (rwcrmhc13.comcast.net [204.127.198.39]) by svr1.postgresql.org (Postfix) with ESMTP id B975932A00B for ; Sat, 9 Oct 2004 21:37:16 +0100 (BST) Received: from sysexperts.com (c-24-6-183-218.client.comcast.net[24.6.183.218]) by comcast.net (rwcrmhc13) with ESMTP id <20041009203713015003c198e>; Sat, 9 Oct 2004 20:37:13 +0000 Received: from localhost (localhost [127.0.0.1]) (uid 1000) by filer with local; Sat, 09 Oct 2004 13:37:12 -0700 id 000064D1.41684BF8.00005448 Date: Sat, 9 Oct 2004 13:37:12 -0700 From: Kevin Brown To: pgsql-performance@postgresql.org Subject: Re: First set of OSDL Shared Mem scalability results, some wierdness ... Message-ID: <20041009203710.GB665@filer> Mail-Followup-To: Kevin Brown , pgsql-performance@postgresql.org References: <200410081443.16109.josh@agliodbs.com> <20041009112048.GA665@filer> <27696.1097334448@sss.pgh.pa.us> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline In-Reply-To: <27696.1097334448@sss.pgh.pa.us> Organization: Frobozzco International User-Agent: Mutt/1.5.6+20040818i X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.0 tagged_above=0.0 required=5.0 tests= X-Spam-Level: X-Archive-Number: 200410/122 X-Sequence-Number: 8598 Tom Lane wrote: > Kevin Brown writes: > > This is why I sometimes wonder whether or not it would be a win to use > > mmap() to access the data and index files -- > > mmap() is Right Out because it does not afford us sufficient control > over when changes to the in-memory data will propagate to disk. The > address-space-management problems you describe are also a nasty > headache, but that one is the showstopper. Huh? Surely fsync() or fdatasync() of the file descriptor associated with the mmap()ed region at the appropriate times would accomplish much of this? I'm particularly confused since PG's entire approach to disk I/O is predicated on the notion that the OS, and not PG, is the best arbiter of when data hits the disk. Otherwise it would be using raw partitions for the highest-speed data store, yes? Also, there isn't any particular requirement to use mmap() for everything -- you can use traditional open/write/close calls for the WAL and mmap() for the data/index files (but it wouldn't surprise me if this would require some extensive code changes). That said, if it's typical for many changes to made to a page internally before PG needs to commit that page to disk, then your argument makes sense, and that's especially true if we simply cannot have the page written to disk in a partially-modified state (something I can easily see being an issue for the WAL -- would the same hold true of the index/data files?). -- Kevin Brown kevin@sysexperts.com