X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (mx1.hub.org [200.46.208.251]) by postgresql.org (Postfix) with ESMTP id 95D459FA619 for ; Fri, 5 May 2006 11:31:49 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.208.251]) (amavisd-new, port 10024) with ESMTP id 49077-01 for ; Fri, 5 May 2006 11:31:38 -0300 (ADT) X-Greylist: from auto-whitelisted by SQLgrey- Received: from globalrelay.com (mail1.globalrelay.com [216.18.71.77]) by postgresql.org (Postfix) with ESMTP id 33E8E9FA372 for ; Fri, 5 May 2006 11:31:37 -0300 (ADT) X-Virus-Scanned: Scanned by GRC-AntiVirus Gateway X-GR-Acctd: YES Received: from [67.90.96.2] (HELO DaveEMachine) by globalrelay.com (CommuniGate Pro SMTP 4.2.3) with ESMTP id 89497475; Fri, 05 May 2006 07:31:33 -0700 From: "Dave Dutcher" To: "'mcelroy, tim'" Cc: Subject: Re: Memory and/or cache issues? Date: Fri, 5 May 2006 09:31:33 -0500 Message-ID: <007d01c67050$9bf11340$8300a8c0@tridecap.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_007E_01C67026.B31B0B40" X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.2627 Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2869 In-reply-to: <0C4841B42F87D51195BD00B0D020F5CB044B260D@MORPHEUS> X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200605/111 X-Sequence-Number: 18898 This is a multi-part message in MIME format. ------=_NextPart_000_007E_01C67026.B31B0B40 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit For a standard config most of the memory used by Postgres is the shared buffers. The shared buffers are a cache to store blocks read from the disk, so if you do a query, Postgres will allocate and fill the shared buffers up to the max amount you set in your postgresql.conf file. Postgres doesn't release that memory between queries because the point is to be able to pull data from ram instead of the disk on the next query. Are you sure your settings in postgresql.conf are standard? What are your settings for shared_buffers and work_mem? -----Original Message----- From: pgsql-performance-owner@postgresql.org [mailto:pgsql-performance-owner@postgresql.org] On Behalf Of mcelroy, tim Sent: Friday, May 05, 2006 8:58 AM To: 'Tom Lane' Cc: pgsql-performance@postgresql.org Subject: Re: [PERFORM] Memory and/or cache issues? Are you saying the kernel's disc cache may be getting whacked? No, I understand that PG should use as much memory as it can and the system as well. The main problem here is that with almost all the 8GB of RAM 'in use' when I try to do a pg_dump or vacuumdb I run out of memory and the system crashes.... I well understand that unused memory is not a good thing, just that when you have none and can't do the maint work....bad stuff happens. For example, I just created a benchdb on my DEV box with 1,000,000 tuples. As this ran the mem in use jumped up 1G and it hasn't gone down? Once the PG process has finished its task shouldn't it release the memory it used? Thanks, Tim -----Original Message----- From: Tom Lane [mailto:tgl@sss.pgh.pa.us] Sent: Friday, May 05, 2006 9:44 AM To: mcelroy, tim Cc: pgsql-performance@postgresql.org Subject: Re: [PERFORM] Memory and/or cache issues? "mcelroy, tim" writes: > I see this on all the postgres installations, no matter what I > set the postgresql.conf settings to regarding memory allocation, once > postgres starts up 95% of the memory on the box is used. Is there a way > within Linux to 'see' what or who is actually using this memory? Probably kernel disk cache. Are you under the misimpression that unused memory is a good thing? If a Unix-ish system *isn't* showing near zero free memory under load, the kernel is wasting valuable resources. regards, tom lane ------=_NextPart_000_007E_01C67026.B31B0B40 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Message
For a=20 standard config most of the memory used by Postgres is the shared = buffers. =20 The shared buffers are a cache to store blocks read from the disk, so if = you do=20 a query, Postgres will allocate and fill the shared buffers up to the = max amount=20 you set in your postgresql.conf file.  Postgres doesn't release = that=20 memory between queries because the point is to be able to pull data = from=20 ram instead of the disk on the next query.
 
Are you sure your settings in postgresql.conf are=20 standard?  What are your settings for shared_buffers and=20 work_mem?
 
 
-----Original Message-----
From:=20 pgsql-performance-owner@postgresql.org=20 [mailto:pgsql-performance-owner@postgresql.org] On Behalf Of = mcelroy,=20 tim
Sent: Friday, May 05, 2006 8:58 AM
To: 'Tom=20 Lane'
Cc: = pgsql-performance@postgresql.org
Subject: Re:=20 [PERFORM] Memory and/or cache issues?

Are you saying the kernel's disc cache may be = getting=20 whacked?  No, I understand that PG should use as much memory as = it can=20 and the system as well.  The main problem here is that with = almost all=20 the 8GB of RAM 'in use' when I try to do a pg_dump or vacuumdb I run = out of=20 memory and the system crashes....

I well understand that unused memory is not a good = thing, just=20 that when you have none and can't do the maint work....bad stuff=20 happens.  For example, I just created a benchdb on my DEV box = with=20 1,000,000 tuples.  As this ran the mem in use jumped up 1G and it = hasn't=20 gone down?  Once the PG process has finished its task shouldn't = it=20 release the memory it used?

Thanks,
Tim =


 -----Original Message-----
From:=20   Tom Lane [mailto:tgl@sss.pgh.pa.us] =
Sent:   Friday, May 05, 2006 9:44 AM =
To:     mcelroy, tim
Cc:     = pgsql-performance@postgresql.org=20
Subject:        = Re:=20 [PERFORM] Memory and/or cache issues?

"mcelroy, tim" <tim.mcelroy@bostonstock.com>=20 writes:
> I see this on all the postgres=20 installations, no matter what I
> set the = postgresql.conf settings to regarding memory allocation, once =
> postgres starts up 95% of the memory on the box is = used.  Is=20 there a way
> within Linux to 'see' what = or who is=20 actually using this memory?

Probably kernel disk cache.  Are you under the=20 misimpression that unused
memory is a good=20 thing?  If a Unix-ish system *isn't* showing near zero =
free memory under load, the kernel is wasting valuable=20 resources.

       =20        =20         regards, tom = lane

------=_NextPart_000_007E_01C67026.B31B0B40--