X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 58D25329E4A for ; Thu, 23 Sep 2004 04:58:57 +0100 (BST) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 39394-03 for ; Thu, 23 Sep 2004 03:58:51 +0000 (GMT) Received: from sue.samurai.com (sue.samurai.com [205.207.28.74]) by svr1.postgresql.org (Postfix) with ESMTP id 68830329D28 for ; Thu, 23 Sep 2004 04:58:51 +0100 (BST) Received: from localhost (localhost [127.0.0.1]) by sue.samurai.com (Postfix) with ESMTP id DD4F73F41; Wed, 22 Sep 2004 23:58:50 -0400 (EDT) Received: from sue.samurai.com ([127.0.0.1]) by localhost (sue.samurai.com [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 54452-01-2; Wed, 22 Sep 2004 23:58:50 -0400 (EDT) Received: from localhost (unknown [61.88.101.19]) by sue.samurai.com (Postfix) with ESMTP id 3A2363F3F; Wed, 22 Sep 2004 23:58:48 -0400 (EDT) Subject: Re: O_DIRECT setting From: Neil Conway To: Guy Thornley Cc: pgsql-performance@postgresql.org In-Reply-To: <20040920075734.GK21733@conker.esphion.com> References: <20040920075734.GK21733@conker.esphion.com> Content-Type: text/plain Message-Id: <1095911914.22485.414.camel@localhost.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.6 Date: Thu, 23 Sep 2004 13:58:34 +1000 Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at mailbox.samurai.com X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.0 tagged_above=0.0 required=5.0 tests= X-Spam-Level: X-Archive-Number: 200409/231 X-Sequence-Number: 8359 On Mon, 2004-09-20 at 17:57, Guy Thornley wrote: > According to the manpage, O_DIRECT implies O_SYNC: > > File I/O is done directly to/from user space buffers. The I/O is > synchronous, i.e., at the completion of the read(2) or write(2) > system call, data is guaranteed to have been transferred. This seems like it would be a rather large net loss. PostgreSQL already structures writes so that the writes we need to hit disk immediately (WAL records) are fsync()'ed -- the kernel is given more freedom to schedule how other writes are flushed from the cache. Also, my recollection is that O_DIRECT also disables readahead -- if that's correct, that's not what we want either. BTW, using O_DIRECT has been discussed a few times in the past. Have you checked the list archives? (for both -performance and -hackers) > Would people be interested in a performance benchmark? Sure -- I'd definitely be curious, although as I said I'm skeptical it's a win. > I need some benchmark tips :) Some people have noted that it can be difficult to use contrib/pgbench to get reproducible results -- you might want to look at Jan's TPC-W implementation or the OSDL database benchmarks: http://pgfoundry.org/projects/tpc-w-php/ http://www.osdl.org/lab_activities/kernel_testing/osdl_database_test_suite/ > Incidentally, postgres heap files suffer really, really bad fragmentation, > which affects sequential scan operations (VACUUM, ANALYZE, REINDEX ...) > quite drastically. We have in-house patches that somewhat alleiviate this, > but they are not release quality. Can you elaborate on these "in-house patches"? -Neil