Received: from maia.hub.org (maia-5.hub.org [200.46.204.29]) by mail.postgresql.org (Postfix) with ESMTP id 731751337B5C for ; Thu, 7 Oct 2010 15:57:56 -0300 (ADT) Received: from mail.postgresql.org ([200.46.204.86]) by maia.hub.org (mx1.hub.org [200.46.204.29]) (amavisd-maia, port 10024) with ESMTP id 97343-03 for ; Thu, 7 Oct 2010 18:57:49 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from tamriel.snowman.net (tamriel.snowman.net [72.66.115.51]) by mail.postgresql.org (Postfix) with ESMTP id 8CCDC1337B4A for ; Thu, 7 Oct 2010 15:57:48 -0300 (ADT) Received: by tamriel.snowman.net (Postfix, from userid 1000) id 2882322941; Thu, 7 Oct 2010 14:57:48 -0400 (EDT) Date: Thu, 7 Oct 2010 14:57:48 -0400 From: Stephen Frost To: Dan Harris Cc: pgsql-performance Subject: Re: large dataset with write vs read clients Message-ID: <20101007185748.GZ26232@tamriel.snowman.net> References: <4CAE1192.5080807@drivefaster.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="q//942cwY0lD3Qrv" Content-Disposition: inline In-Reply-To: <4CAE1192.5080807@drivefaster.net> X-Editor: Vim http://www.vim.org/ X-Info: http://www.snowman.net X-Operating-System: Linux/2.6.26-1-amd64 (x86_64) X-Uptime: 14:57:16 up 406 days, 6:22, 14 users, load average: 0.22, 0.13, 0.10 User-Agent: Mutt/1.5.18 (2008-05-17) X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=-1.91 tagged_above=-10 required=5 tests=BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01 X-Spam-Level: X-Archive-Number: 201010/63 X-Sequence-Number: 40482 --q//942cwY0lD3Qrv Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable * Dan Harris (fbsd@drivefaster.net) wrote: > On 10/7/10 11:47 AM, Aaron Turner wrote: >> Basically, each connection is taking about 100MB resident. As we need >> to increase the number of threads to be able to query all the devices >> in the 5 minute window, we're running out of memory. > I think the first thing to do is look into using a connection pooler =20 > like pgpool to reduce your connection memory overhead. Yeah.. Having the number of database connections be close to the number of processors is usually recommended. Stephen --q//942cwY0lD3Qrv Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAkyuGCwACgkQrzgMPqB3kigQUQCfYkWKsDz0+IZvvm+MMetS7AWK oysAn2Hy46gCldKTUwXRIy2DZnAS2sTr =u67x -----END PGP SIGNATURE----- --q//942cwY0lD3Qrv--