Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.80) (envelope-from ) id 1VFmPI-0007H1-Lo for pgsql-performance@arkaria.postgresql.org; Sat, 31 Aug 2013 14:44:08 +0000 Received: from localhost ([127.0.0.1] helo=postgresql.org) by malur.postgresql.org with smtp (Exim 4.80) (envelope-from ) id 1VFmPH-0006c1-Nt for pgsql-performance@arkaria.postgresql.org; Sat, 31 Aug 2013 14:44:07 +0000 Received: from makus.postgresql.org ([2001:4800:7903:4::125]) by malur.postgresql.org with esmtp (Exim 4.80) (envelope-from ) id 1VFmPG-0006bu-FI for pgsql-performance@postgresql.org; Sat, 31 Aug 2013 14:44:06 +0000 Received: from nm1-vm1.bullet.mail.bf1.yahoo.com ([98.139.213.163]) by makus.postgresql.org with esmtp (Exim 4.80) (envelope-from ) id 1VFmPC-0002dj-JE for pgsql-performance@postgresql.org; Sat, 31 Aug 2013 14:44:05 +0000 Received: from [98.139.215.142] by nm1.bullet.mail.bf1.yahoo.com with NNFMP; 31 Aug 2013 14:44:01 -0000 Received: from [98.139.212.199] by tm13.bullet.mail.bf1.yahoo.com with NNFMP; 31 Aug 2013 14:44:01 -0000 Received: from [127.0.0.1] by omp1008.mail.bf1.yahoo.com with NNFMP; 31 Aug 2013 14:44:01 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 661659.96076.bm@omp1008.mail.bf1.yahoo.com Received: (qmail 19631 invoked by uid 60001); 31 Aug 2013 14:44:01 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ymail.com; s=s1024; t=1377960241; bh=ASe1Vh2ir7l/shKOkqsRMMmPlNWuMVbw5SBeuQlEyT4=; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=PaF6X29lmvNAGJ783hV75hZO6ln3YJJ9WQKnjaLtFSkO/gVf5oIa9gviFkLi/U4dr1hqb5EqhBwOcFsqxIrT6bt1jSVwYZdA1dNPhUZNhMx2HiTEJKzNNYW0SsrZP99PWy0s6Kos0v3/lgpP+ZxiKNp8fCSggJxSsDnR9MswQFc= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=ymail.com; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=2Hp1z5GVhtfKOMgp0lbYeIiQRlLarNjQdNh23BB9St1FRc70BYSyetbk4GVE/MGPcwH5fppNDNONg15k8r+iKW2oABYLXzljvLoEl2wieCe6uPZXhmGUlbEFjxNYEKySv/uqxvIDsNt7a2Y6eHFXe88D905Sw2LoOhNr49RT+PI=; X-YMail-OSG: ZtNTjosVM1no7MPGTi0bbanGCcMJwjVeo0G52_41zBD0AvK .SDaaZzbWoASpLDZYOvVJaKtlOQeIcp82JYSpcYkZpYq7cnYIo_5Udxjahd2 O2gDuHP3j_FoRLcEk7.hKSqz7PCCjzFta3Xb4gwj3ukoYAeK6C7H8.6jy4Ze xyPC1Nu9uEMVQmJ.m_V5r2odR..aLMoI4FJNSSK2Ng8fuLSaq5kGhXBokcE0 bnoZ58mwh.YecYJPDLgpJGHJFgvxAOXHYC0F7GZOBL_RoS6q58hEPThIT5ZE Hkyt1ADAs_6M8SlHKxHkvjeZ8rkPlbzPThDJg1u6.lDrJe7sy5GKR9qbpoCx FmSUjkWI4qRgWp2FNk0wipXYJ66QLcADxpFI20aibnJa4ouAizqFoWZYFYMk u8W3kbxWeFZCBdQb7woIs855FwNG1OpQRvIDa1s9RpyZH2_41n_w2pLIZgGZ vAmEHrEJv_QAwfMFDZwkUg_y5rUUmSfCoKZEQTpeVM.UVo6Aq4qEbIkmIPEN AnqQ7Fwi5aUkC0s6DKp3xG.PAFi22sMvYkuGppaq.FeGUZg2IMnbkK192ZqZ AwFbnyvk4bHnMA6DGGMw.Q_K90nKKQohBQWT0GULge_jibMDvu53nLJaKN3H sNEaYFASBpEmc.kf1Hz8w2uTyIzpR7zjSiHApx1kWezxLIeaZXoMdeQ-- Received: from [76.255.18.237] by web162901.mail.bf1.yahoo.com via HTTP; Sat, 31 Aug 2013 07:44:01 PDT X-Rocket-MIMEInfo: 002.001, YnNyZWVqaXRoaW4gPGJzcmVlaml0aGluQGdtYWlsLmNvbT4gd3JvdGU6Cgo.IFdoYXQgSSBwb3N0ZWQgaXMgYWJvdXQgYSBuZXcgc2V0dXAgdGhhdCdzIGdvaW5nIHRvIGNvbWUKPiB1cC4uRGlzY3Vzc2lvbnMgYXJlIG9uIHdoZXRoZXIgdG8gc2V0dXAgREIgY2x1c3RlciB0byBoYW5kbGUgMTAwMAo.IGNvbmN1cnJlbnQgdXNlcnMuCgpJIHByZXZpb3VzbHkgd29ya2VkIGZvciBXaXNjb25zaW4gQ291cnRzLCB3aGVyZSB3ZSBoYWQgYSBzaW5nbGUKc2VydmVyIHdoaWNoIGhhbmRsZWQgYWJvdXQgMzAwMCB3ZWIBMAEBAQE- X-Mailer: YahooMailWebService/0.8.156.576 References: <1377778443804-5768917.post@n5.nabble.com> <521F61B9.1000709@archonet.com> <521F63DD.3050301@commandprompt.com> <1377792804612-5768951.post@n5.nabble.com> Message-ID: <1377960241.19370.YahooMailNeo@web162901.mail.bf1.yahoo.com> Date: Sat, 31 Aug 2013 07:44:01 -0700 (PDT) From: Kevin Grittner Reply-To: Kevin Grittner Subject: Re: How clustering for scale out works in PostgreSQL To: bsreejithin , "pgsql-performance@postgresql.org" In-Reply-To: <1377792804612-5768951.post@n5.nabble.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Pg-Spam-Score: 1.5 (+) List-Archive: List-Help: List-ID: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-Mailing-List: pgsql-performance Precedence: bulk Sender: pgsql-performance-owner@postgresql.org bsreejithin wrote: > What I posted is about a new setup that's going to come > up..Discussions are on whether to setup DB cluster to handle 1000 > concurrent users. I previously worked for Wisconsin Courts, where we had a single server which handled about 3000 web users collectively generating hundreds of web hits per second generating thousands of queries per second, while at the same time functioning as a replication target from 80 sources sending about 20 transactions per second which modified data (many having a large number of DML statements per transaction) against a 3 TB database.=A0 The same machine also hosted a transaction repository for all modifications to the database, indexed for audit reports and ad hoc queries; that was another 3 TB.=A0 Each of these was running on a 40-drive RAID. Shortly before I left we upgraded from a machine with 16 cores and 256 GB RAM to one with 32 cores and 512 GB RAM, because there is constant growth in both database size and load.=A0 Performance was still good on the smaller machine, but monitoring showed we were approaching saturation.=A0 We had started to see some performance degradation on the old machine, but were able to buy time by reducing the size of the web connection pool (in the Java application code) from 65 to 35.=A0 Testing different connection pool sizes showed that pool size to be optimal for our workload on that machine; your ideal pool size can only be determined through testing. You can poke around in this application here, if you like: http://wcca.wicourts.gov/ -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company --=20 Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance