X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 9407D5E46BF for ; Sun, 8 Aug 2004 15:40:40 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 20789-08 for ; Sun, 8 Aug 2004 18:40:35 +0000 (GMT) Received: from news.hub.org (news.hub.org [200.46.204.72]) by svr1.postgresql.org (Postfix) with ESMTP id 59EB35E40AC for ; Sun, 8 Aug 2004 15:40:35 -0300 (ADT) Received: from news.hub.org (news.hub.org [200.46.204.72]) by news.hub.org (8.12.9/8.12.9) with ESMTP id i78IeUiI024976 for ; Sun, 8 Aug 2004 18:40:32 GMT (envelope-from news@news.hub.org) Received: (from news@localhost) by news.hub.org (8.12.9/8.12.9/Submit) id i78IIFeX019694 for pgsql-performance@postgresql.org; Sun, 8 Aug 2004 18:18:15 GMT From: Martin Foster User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.2) Gecko/20040803 X-Accept-Language: en-ca, en-gb, en-us, en MIME-Version: 1.0 X-Newsgroups: comp.databases.postgresql.performance Subject: Re: Performance Bottleneck References: <007f01c47d54$233b4bf0$8300a8c0@solent> <41164E98.8030504@vulcanus.its.tudelft.nl> In-Reply-To: <41164E98.8030504@vulcanus.its.tudelft.nl> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Lines: 74 Message-ID: Date: Sun, 08 Aug 2004 18:18:15 GMT To: pgsql-performance@postgresql.org X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.0 tagged_above=0.0 required=5.0 tests= X-Spam-Level: X-Archive-Number: 200408/106 X-Sequence-Number: 7742 Arjen van der Meijden wrote: > On 8-8-2004 16:29, Matt Clark wrote: > >> There are two well-worn and very mature techniques for dealing with the >> issue of web apps using one DB connection per apache process, both of >> which >> work extremely well and attack the issue at its source. >> >> 1) Use a front-end caching proxy like Squid as an accelerator. Static >> content will be served by the accelerator 99% of the time. Additionally, >> large pages can be served immediately to the accelerator by Apache, which >> can then go on to serve another request without waiting for the end >> user's >> dial-up connection to pull the data down. Massive speedup, fewer apache >> processes needed. > > > Another version of this 1) is to run with a "content accelerator"; our > "favourite" is to run Tux in front of Apache. It takes over the > connection-handling stuff, has a very low memoryprofile (compared to > Apache) and very little overhead. What it does, is to serve up all > "simple" content (although you can have cgi/php/perl and other languages > being processed by it, entirely disabling the need for apache in some > cases) and forwards/proxies everything it doesn't understand to an > Apache/other webserver running at the same machine (which runs on > another port). > > I think there are a few advantages over Squid; since it is partially > done in kernel-space it can be slightly faster in serving up content, > apart from its simplicity which will probably matter even more. You'll > have no caching issues for pages that should not be cached or static > files that change periodically (like every few seconds). Afaik Tux can > handle more than 10 times as much ab-generated requests per second than > a default-compiled Apache on the same machine. > And besides the speed-up, you can do any request you where able to do > before, since Tux will simply forward it to Apache if it didn't > understand it. > > Anyway, apart from all that. Reducing the amount of apache-connections > is nice, but not really the same as reducing the amount of > pooled-connections using a db-pool... You may even be able to run with > 1000 http-connections, 40 apache-processes and 10 db-connections. In > case of the non-pooled setup, you'd still have 40 db-connections. > > In a simple test I did, I did feel pgpool had quite some overhead > though. So it should be well tested, to find out where the > turnover-point is where it will be a gain instead of a loss... > > Best regards, > > Arjen van der Meijden > Other then images, there are very few static pages being loaded up by the user. Since they make up a very small portion of the traffic, it tends to be an optimization we can forgo for now. I attempted to make use of pgpool. At the default 32 connections pre-forked the webserver almost immediately tapped out the pgpool base and content stopped being served because no new processes were being forked to make up for it. So I raised it to a higher value (256) and it immediately segfaulted and dropped the core. So not sure exactly how to proceed, since I rather need the thing to fork additional servers as load hits and not the other way around. Unless I had it configured oddly, but it seems work differently then an Apache server would to handle content. Martin Foster Creator/Designer Ethereal Realms martin@ethereal-realms.org