X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 6733A32A15D; Sat, 16 Oct 2004 17:54:24 +0100 (BST) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 10850-03; Sat, 16 Oct 2004 16:54:18 +0000 (GMT) Received: from sss.pgh.pa.us (sss.pgh.pa.us [66.207.139.130]) by svr1.postgresql.org (Postfix) with ESMTP id E1F7F32A2C6; Sat, 16 Oct 2004 17:54:18 +0100 (BST) Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1]) by sss.pgh.pa.us (8.13.1/8.13.1) with ESMTP id i9GGsHSA004665; Sat, 16 Oct 2004 12:54:17 -0400 (EDT) To: pgsql-hackers@postgreSQL.org, pgsql-performance@postgreSQL.org Subject: Getting rid of AtEOXact_Buffers (was Re: [Testperf-general] Re: [PERFORM] First set of OSDL Shared Memscalability results, some wierdness ...) In-reply-to: <26068.1097872442@sss.pgh.pa.us> References: <10202.1097858894@sss.pgh.pa.us> <200410151313.13624.josh@agliodbs.com> <26068.1097872442@sss.pgh.pa.us> Comments: In-reply-to Tom Lane message dated "Fri, 15 Oct 2004 16:34:02 -0400" Date: Sat, 16 Oct 2004 12:54:17 -0400 Message-ID: <4664.1097945657@sss.pgh.pa.us> From: Tom Lane X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.0 tagged_above=0.0 required=5.0 tests= X-Spam-Level: X-Archive-Number: 200410/525 X-Sequence-Number: 60028 I wrote: > Josh Berkus writes: >> First off, two test runs with OProfile are available at: >> http://khack.osdl.org/stp/298124/ >> http://khack.osdl.org/stp/298121/ > Hmm. The stuff above 1% in the first of these is > Counted CPU_CLK_UNHALTED events (clocks processor is not halted) with a unit mask of 0x00 (No unit mask) count 100000 > samples % app name symbol name > ... > 920369 2.1332 postgres AtEOXact_Buffers > ... > In the second test AtEOXact_Buffers is much lower (down around 0.57 > percent) but the other suspects are similar. Since the only difference > in parameters is shared_buffers (36000 vs 9000), it does look like we > are approaching the point where AtEOXact_Buffers is a problem, but so > far it's only a 2% drag. It occurs to me that given the 8.0 resource manager mechanism, we could in fact dispense with AtEOXact_Buffers, or perhaps better turn it into a no-op unless #ifdef USE_ASSERT_CHECKING. We'd just get rid of the special case for transaction termination in resowner.c and let the resource owner be responsible for releasing locked buffers always. The OSDL results suggest that this won't matter much at the level of 10000 or so shared buffers, but for 100000 or more buffers the linear scan in AtEOXact_Buffers is going to become a problem. We could also get rid of the linear search in UnlockBuffers(). The only thing it's for anymore is to release a BM_PIN_COUNT_WAITER flag, and since a backend could not be doing more than one of those at a time, we don't really need an array of flags for that, only a single variable. This does not show in the OSDL results, which I presume means that their test case is not exercising transaction aborts; but I think we need to zap both routines to make the world safe for large shared_buffers values. (See also http://archives.postgresql.org/pgsql-performance/2004-10/msg00218.php) Any objection to doing this for 8.0? regards, tom lane