DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com;
	h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references;
	b=DEbmxml9cNTB3g9AGBMlSNvpMlrhJ5KBkn48BlbWtseHnJQD8+m+wRGE2sRwNOsz9z69/mgaTtRzlPpQ1Dq3cJ4s0kv7SG6uT8LSkZxtMsHGgMh/Sa+OGExjZ8oofdiCmXa5C3c7dFV0Z4L26TYKAKaGjNuQvJMjAnP331jHv2s=
Message-ID: <b918cf3d0512080600q5c841fc8lecd96c3bad66cfdc@mail.gmail.com>
Date: Thu, 8 Dec 2005 14:00:06 +0000
From: Mike Rylander <mrylander@gmail.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Subject: Re: Memory Leakage Problem
Cc: pgsql general <pgsql-general@postgresql.org>
In-Reply-To: <6353.1134016733@sss.pgh.pa.us>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline
References: <c10e7feb0512060122u33a9c8eakb178525f3834b00@mail.gmail.com>
	<1133891282.11803.7.camel@state.g2switchworks.com>
	<c10e7feb0512061816o21305ac7n576694f2624dc6ed@mail.gmail.com>
	<1133991204.11803.43.camel@state.g2switchworks.com>
	<c10e7feb0512071825h30c93c00kdf5ea2a11e613f0b@mail.gmail.com>
	<b918cf3d0512071946m3cbfc630y679e0350a7f6b864@mail.gmail.com>
	<6353.1134016733@sss.pgh.pa.us>

On 12/8/05, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Mike Rylander <mrylander@gmail.com> writes:
> > To cut to the chase, here are
> > some numbers for everyone to digest:
> >            total gnu ps resident size
> > # ps ax -o rss|perl -e '$x +=3D $_ for (<>);print "$x\n";'
> > 5810492
> >            total gnu ps virual size
> > # ps ax -o vsz|perl -e '$x +=3D $_ for (<>);print "$x\n";'
> > 10585400
> >            total gnu ps "if all pages were dirtied and swapped" size
> > # ps ax -o size|perl -e '$x +=3D $_ for (<>);print "$x\n";'
> > 1970952
>
> I wouldn't put any faith in those numbers at all, because you'll be
> counting the PG shared memory multiple times.
>
> On the Linux versions I've used lately, ps and top report a process'
> memory size as including all its private memory, plus all the pages
> of shared memory that it has touched since it started.  So if you run
> say a seqscan over a large table in a freshly-started backend, the
> reported memory usage will ramp up from a couple meg to the size of
> your shared_buffer arena plus a couple meg --- but in reality the
> space used by the process is staying constant at a couple meg.

Right, I can definitely see that happening.  Some backends are upwards
of 200M, some are just a few since they haven't been touched yet.

>
> Now, multiply that effect by N backends doing this at once, and you'll
> have a very skewed view of what's happening in your system.

Absolutely ...
>
> I'd trust the totals reported by free and dstat a lot more than summing
> per-process numbers from ps or top.
>

And there's the part that's confusing me:  the numbers for used memory
produced by free and dstat, after subtracting the buffers/cache
amounts, are /larger/ than those that ps and top report. (top says the
same thing as ps, on the whole.)


> > Now, I'm not blaming Pg for the apparent discrepancy in calculated vs.
> > reported-by-free memory usage, but I only noticed this after upgrading
> > to 8.1.
>
> I don't know of any reason to think that 8.1 would act differently from
> older PG versions in this respect.
>

Neither can I, which is why I don't blame it. ;)  I'm just reporting
when/where I noticed the issue.

>                         regards, tom lane
>


--
Mike Rylander
mrylander@gmail.com
GPLS -- PINES Development
Database Developer
http://open-ils.org