Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.72) (envelope-from ) id 1TctP0-0006KA-9I for pgsql-performance@arkaria.postgresql.org; Mon, 26 Nov 2012 07:46:50 +0000 Received: from localhost ([127.0.0.1] helo=postgresql.org) by malur.postgresql.org with smtp (Exim 4.72) (envelope-from ) id 1TctOz-00088B-57 for pgsql-performance@arkaria.postgresql.org; Mon, 26 Nov 2012 07:46:49 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtp (Exim 4.72) (envelope-from ) id 1TctOy-000885-3A for pgsql-performance@postgresql.org; Mon, 26 Nov 2012 07:46:48 +0000 Received: from smtp-outbound-2.vmware.com ([208.91.2.13]) by magus.postgresql.org with esmtp (Exim 4.72) (envelope-from ) id 1TctOt-0007It-Nq for pgsql-performance@postgresql.org; Mon, 26 Nov 2012 07:46:46 +0000 Received: from sc9-mailhost2.vmware.com (sc9-mailhost2.vmware.com [10.113.161.72]) by smtp-outbound-2.vmware.com (Postfix) with ESMTP id 222C428637; Sun, 25 Nov 2012 23:46:40 -0800 (PST) Received: from zcs-prod-ext-mta-2.vmware.com (zcs-prod-ext-mta-2.vmware.com [10.113.62.224]) by sc9-mailhost2.vmware.com (Postfix) with ESMTP id 1E807B04B9; Sun, 25 Nov 2012 23:46:40 -0800 (PST) Received: from localhost (localhost.localdomain [127.0.0.1]) by zcs-prod-ext-mta-2.vmware.com (Postfix) with ESMTP id EFD7BC19F7; Sun, 25 Nov 2012 23:46:39 -0800 (PST) X-Virus-Scanned: amavisd-new at zcs-prod-ext-mta-2.vmware.com Received: from zcs-prod-ext-mta-2.vmware.com ([127.0.0.1]) by localhost (zcs-prod-ext-mta-2.vmware.com [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id yj2DI9AOoywI; Sun, 25 Nov 2012 23:46:39 -0800 (PST) Received: from [192.168.1.90] (unknown [10.113.62.14]) by zcs-prod-ext-mta-2.vmware.com (Postfix) with ESMTPSA id A5568C16B4; Sun, 25 Nov 2012 23:46:38 -0800 (PST) Message-ID: <50B31E59.9050004@vmware.com> Date: Mon, 26 Nov 2012 09:46:33 +0200 From: Heikki Linnakangas Organization: VMware User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0.10) Gecko/20121027 Icedove/10.0.10 MIME-Version: 1.0 To: Catalin Iacob CC: Jeff Janes , Marko Kreen , Merlin Moncure , pgsql-performance@postgresql.org Subject: Re: How to keep queries low latency as concurrency increases References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Pg-Spam-Score: -7.3 (-------) List-Archive: List-Help: List-ID: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-Mailing-List: pgsql-performance Precedence: bulk Sender: pgsql-performance-owner@postgresql.org On 25.11.2012 18:30, Catalin Iacob wrote: > So it seems we're just doing too many connections and too many > queries. Each page view from a user translates to multiple requests to > the application server and each of those translates to a connection > and at least a few queries (which are done in middleware and therefore > happen for each and every query). One pgbouncer can handle lots of > concurrent idle connections and lots of queries/second but our 9000 > queries/second to seem push it too much. The longer term solution for > us would probably be to do less connections (by doing less Django > requests for a page) and less queries, before our deadline we were > just searching for a short term solution to handle an expected traffic > spike. The typical solution to that is caching, see https://docs.djangoproject.com/en/1.4/topics/cache/. - Heikki -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance