X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.2]) by svr1.postgresql.org (Postfix) with ESMTP id CA908D1EB9D; Thu, 18 Mar 2004 21:01:20 +0000 (GMT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (neptune.hub.org [200.46.204.2]) (amavisd-new, port 10024) with ESMTP id 18050-06; Thu, 18 Mar 2004 17:01:09 -0400 (AST) Received: from sss.pgh.pa.us (unknown [192.204.191.242]) by svr1.postgresql.org (Postfix) with ESMTP id 0676ED1E150; Thu, 18 Mar 2004 17:00:58 -0400 (AST) Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1]) by sss.pgh.pa.us (8.12.11/8.12.11) with ESMTP id i2IL0s0b012262; Thu, 18 Mar 2004 16:00:54 -0500 (EST) To: josh@agliodbs.com Cc: Kurt Roeckx , Bruce Momjian , Mark Kirkwood , pgsql-performance@postgresql.org, PostgreSQL-development Subject: Re: [HACKERS] fsync method checking In-reply-to: <200403181239.58226.josh@agliodbs.com> References: <20040318191840.GA7088@ping.be> <20040318200359.GA8330@ping.be> <11816.1079641223@sss.pgh.pa.us> <200403181239.58226.josh@agliodbs.com> Comments: In-reply-to Josh Berkus message dated "Thu, 18 Mar 2004 12:39:58 -0800" Date: Thu, 18 Mar 2004 16:00:54 -0500 Message-ID: <12261.1079643654@sss.pgh.pa.us> From: Tom Lane X-Virus-Scanned: by amavisd-new at postgresql.org X-Archive-Number: 200403/271 X-Sequence-Number: 6127 Josh Berkus writes: > 1) This is an OSS project. Why not just recruit a bunch of people on > PERFORMANCE and GENERAL to test the 4 different synch methods using real > databases? No test like reality, I say .... I agree --- that is likely to yield *far* more useful results than any standalone test program, for the purpose of finding out what wal_sync_method to use in real databases. However, there's a second issue here: we would like to move sync/checkpoint responsibility into the bgwriter, and that requires knowing whether it's valid to let one process fsync on behalf of writes that were done by other processes. That's got nothing to do with WAL sync performance. I think that it would be sensible to make a test program that focuses on this one specific question. (There has been some handwaving to the effect that everybody knows this is safe on Unixen, but I question whether the handwavers have seen the internals of HPUX or AIX for instance; and besides we need to worry about Windows now.) A third reason for having a simple test program is to confirm whether your drives are syncing at all (cf. hdparm discussion). > 2) Won't Jan's work on 7.5 memory and I/O management mean that we have to > re-evaluate synching anyway? So far nothing's been done that touches WAL writing. However, I am thinking about making the bgwriter process take some of the load of writing WAL buffers (right now it only writes data-file buffers). And you're right, after that happens we will need to re-measure. The open flags will probably become considerably more attractive than they are now, if the bgwriter handles most non-commit writes of WAL. (We might also think of letting the bgwriter use a different sync method than the backends do.) regards, tom lane