X-Original-To: pgsql-hackers-win32-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id C79543A591E; Thu, 2 Dec 2004 19:29:04 +0000 (GMT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 71127-07; Thu, 2 Dec 2004 19:28:56 +0000 (GMT) Received: from candle.pha.pa.us (candle.pha.pa.us [207.106.42.251]) by svr1.postgresql.org (Postfix) with ESMTP id C318B3A58F8; Thu, 2 Dec 2004 19:28:26 +0000 (GMT) Received: (from pgman@localhost) by candle.pha.pa.us (8.11.6/8.11.6) id iB2JSHo09487; Thu, 2 Dec 2004 14:28:17 -0500 (EST) From: Bruce Momjian Message-Id: <200412021928.iB2JSHo09487@candle.pha.pa.us> Subject: Re: [PERFORM] scalability issues on win32 In-Reply-To: <6EE64EF3AB31D5448D0007DD34EEB3412A754F@Herge.rcsinc.local> To: Merlin Moncure Date: Thu, 2 Dec 2004 14:28:17 -0500 (EST) Cc: Dave Page , pgsql-performance@postgresql.org, Tom Lane , Magnus Hagander , Win 32 hackers PGSQL X-Mailer: ELM [version 2.4ME+ PL108 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.002 tagged_above=0 required=5 tests=AWL X-Spam-Level: X-Archive-Number: 200412/4 X-Sequence-Number: 2515 Merlin Moncure wrote: > > > This was an intersting Win32/linux comparison. I expected > > > Linux to scale better, but I was surprised how poorly XP > > > scaled. It reinforces our perception that Win32 is for low > > > traffic servers. > > > > That's a bit harsh given the lack of any further investigation so far > > isn't it? Win32 can run perfectly well with other DBMSs with hundreds > of > > users. > > > > Any chance you can profile your test runs Merlin? > > > > Ok, I am starting to strongly suspect the statistics collector of > various kinds of malfeasance. Right now I am running with the stats > collector off and I am getting much better scalability and much more > deterministic query times...that is, even when under moderate to heavy > load query running times are proportional to the number of users on the > system...the system is purring like a kitten right now with over a 100 > users logged in. > > This coupled with the fact that I was getting random restarts with the > collector process makes me think that there is some kind of issue with > the ipc between the collector and the backends that is blocking and/or > is being improperly handled after failure. > > I was running with statement level stats on under scenarios with > extremely high levels of query activity where the server might be > processing 500-1500 queries/second or more spread out over multiple > backends. > > I'll look into this issue more over next several days. I'll dip back > into the code and see if I can come up with a better > answer...unfortunately I can't run the stats collector on until I can > schedule another load test. OK, the big problem is that we are nearing RC1. We would like some feedback on this as soon as possible. A major Win32 cleanup for this could delay the 8.0 release. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073