X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 31AFD329D14 for ; Thu, 23 Sep 2004 14:36:39 +0100 (BST) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 17380-10 for ; Thu, 23 Sep 2004 13:36:28 +0000 (GMT) Received: from candle.pha.pa.us (candle.pha.pa.us [207.106.42.251]) by svr1.postgresql.org (Postfix) with ESMTP id E3B32329CA4 for ; Thu, 23 Sep 2004 14:36:30 +0100 (BST) Received: (from pgman@localhost) by candle.pha.pa.us (8.11.6/8.11.6) id i8NDZjR21993; Thu, 23 Sep 2004 09:35:45 -0400 (EDT) From: Bruce Momjian Message-Id: <200409231335.i8NDZjR21993@candle.pha.pa.us> Subject: Re: O_DIRECT setting In-Reply-To: <1095911914.22485.414.camel@localhost.localdomain> To: Neil Conway Date: Thu, 23 Sep 2004 09:35:45 -0400 (EDT) Cc: Guy Thornley , pgsql-performance@postgresql.org X-Mailer: ELM [version 2.4ME+ PL108 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.0 tagged_above=0.0 required=5.0 tests= X-Spam-Level: X-Archive-Number: 200409/234 X-Sequence-Number: 8362 TODO has: * Consider use of open/fcntl(O_DIRECT) to minimize OS caching Should the item be removed? --------------------------------------------------------------------------- Neil Conway wrote: > On Mon, 2004-09-20 at 17:57, Guy Thornley wrote: > > According to the manpage, O_DIRECT implies O_SYNC: > > > > File I/O is done directly to/from user space buffers. The I/O is > > synchronous, i.e., at the completion of the read(2) or write(2) > > system call, data is guaranteed to have been transferred. > > This seems like it would be a rather large net loss. PostgreSQL already > structures writes so that the writes we need to hit disk immediately > (WAL records) are fsync()'ed -- the kernel is given more freedom to > schedule how other writes are flushed from the cache. Also, my > recollection is that O_DIRECT also disables readahead -- if that's > correct, that's not what we want either. > > BTW, using O_DIRECT has been discussed a few times in the past. Have you > checked the list archives? (for both -performance and -hackers) > > > Would people be interested in a performance benchmark? > > Sure -- I'd definitely be curious, although as I said I'm skeptical it's > a win. > > > I need some benchmark tips :) > > Some people have noted that it can be difficult to use contrib/pgbench > to get reproducible results -- you might want to look at Jan's TPC-W > implementation or the OSDL database benchmarks: > > http://pgfoundry.org/projects/tpc-w-php/ > http://www.osdl.org/lab_activities/kernel_testing/osdl_database_test_suite/ > > > Incidentally, postgres heap files suffer really, really bad fragmentation, > > which affects sequential scan operations (VACUUM, ANALYZE, REINDEX ...) > > quite drastically. We have in-house patches that somewhat alleiviate this, > > but they are not release quality. > > Can you elaborate on these "in-house patches"? > > -Neil > > > > ---------------------------(end of broadcast)--------------------------- > TIP 4: Don't 'kill -9' the postmaster > -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073