pg.ddx.io pgsql-performance@postgresql.org mailing list archive help / color / mirror / Atom feed From: John A Meinel <john@arbash-meinel.com>
To: Jeremiah Jahn <jeremiah@cs.earlham.edu>
Cc: Ron <rjpeace@earthlink.net>
Cc: postgres performance <pgsql-performance@postgresql.org>
Subject: Re: extremly low memory usage
Date: Sat, 20 Aug 2005 21:26:04 -0500
Message-ID: <4307E63C.7050408@arbash-meinel.com> (raw )
In-Reply-To: <1124561769.27881.200.camel@bluejay.goodinassociates.com >
References: <1124331108.27881.96.camel@bluejay.goodinassociates.com >
<EA4219B9-E2B0-4756-ABF2-D84F245FDFB5@torgo.978.org >
<1124386761.27881.119.camel@bluejay.goodinassociates.com >
<4304CB77.4070403@arbash-meinel.com >
<1124470109.27881.152.camel@bluejay.goodinassociates.com >
<43061472.2080201@arbash-meinel.com >
<6.2.3.4.0.20050819140714.05c181e8@pop.earthlink.net >
<43063196.9000001@arbash-meinel.com >
<1124485299.27881.192.camel@bluejay.goodinassociates.com >
<43064A06.8020102@arbash-meinel.com >
<1124561769.27881.200.camel@bluejay.goodinassociates.com >
Jeremiah Jahn wrote:
> I'm just watching gnome-system-monoitor. Which after careful
> consideration.....and looking at dstat means I'm on CRACK....GSM isn't
> showing cached memory usage....I asume that the cache memory usage is
> where data off of the disks would be cached...?
>
Well a simple "free" also tells you how much has been cached.
I believe by reading the _cach line, it looks like you have 4.6G cached.
So you are indeed using memory.
I'm still concerned why it seems to be taking 3-4ms per index lookup,
when things should already be cached in RAM.
Now, I may be wrong about whether the indexes are cached, but I sure
would expect them to be.
What is the time for a cached query on your system (with normal nested
loops)? (give the EXPLAIN ANALYZE for the *second* run, or maybe the
fourth).
I'm glad that we aren't seeing something weird with your kernel, at least.
John
=:->
>
>
> memory output from dstat is this for a few seconds:
>
> ---procs--- ------memory-usage----- ---paging-- --disk/sda----disk/sdb- ----swap--- ----total-cpu-usage----
> run blk new|_used _buff _cach _free|__in_ _out_|_read write:_read write|_used _free|usr sys idl wai hiq siq
> 0 0 0|1336M 10M 4603M 17M| 490B 833B|3823B 3503k:1607k 4285k| 160k 2048M| 4 1 89 7 0 0
> 1 0 0|1337M 10M 4600M 18M| 0 0 | 0 0 : 0 464k| 160k 2048M| 25 0 75 0 0 0
> 1 0 0|1337M 10M 4600M 18M| 0 0 | 0 0 : 0 0 | 160k 2048M| 25 0 75 0 0 0
> 1 0 0|1337M 10M 4600M 18M| 0 0 | 0 48k: 0 0 | 160k 2048M| 25 0 75 0 0 0
> 1 0 0|1337M 10M 4600M 18M| 0 0 | 0 0 : 0 0 | 160k 2048M| 25 0 75 0 0 0
> 1 0 0|1337M 10M 4600M 18M| 0 0 | 0 132k: 0 0 | 160k 2048M| 25 0 75 0 0 0
> 1 0 0|1337M 10M 4600M 18M| 0 0 | 0 36k: 0 0 | 160k 2048M| 25 0 75 0 0 0
> 1 0 0|1337M 10M 4600M 18M| 0 0 | 0 0 : 0 0 | 160k 2048M| 25 0 75 0 0 0
> 1 0 0|1337M 10M 4600M 18M| 0 0 | 0 12k: 0 0 | 160k 2048M| 25 0 75 0 0 0
> 1 0 0|1337M 10M 4600M 18M| 0 0 | 0 0 : 0 0 | 160k 2048M| 25 0 75 0 0 0
> 2 0 0|1353M 10M 4585M 18M| 0 0 | 0 0 : 0 0 | 160k 2048M| 25 1 75 0 0 0
> 1 0 0|1321M 10M 4616M 19M| 0 0 | 0 0 : 0 0 | 160k 2048M| 18 8 74 0 0 0
> 1 0 0|1326M 10M 4614M 17M| 0 0 | 0 0 :4096B 0 | 160k 2048M| 16 10 74 1 0 0
> 1 0 0|1330M 10M 4609M 17M| 0 0 | 0 12k:4096B 0 | 160k 2048M| 17 9 74 0 0 0
> 0 1 0|1343M 10M 4596M 17M| 0 0 | 0 0 : 0 316M| 160k 2048M| 5 10 74 11 0 1
> 0 1 0|1339M 10M 4596M 21M| 0 0 | 0 0 : 0 0 | 160k 2048M| 0 0 74 25 0 1
> 0 2 0|1334M 10M 4596M 25M| 0 0 | 0 4096B: 0 0 | 160k 2048M| 0 0 54 44 0 1
> 1 0 0|1326M 10M 4596M 34M| 0 0 | 0 0 : 0 364k| 160k 2048M| 4 1 60 34 0 1
> 1 0 0|1290M 10M 4596M 70M| 0 0 | 0 12k: 0 0 | 160k 2048M| 24 1 75 0 0 0
> 1 0 0|1301M 10M 4596M 59M| 0 0 | 0 20k: 0 0 | 160k 2048M| 21 4 75 0 0 0
> 1 0 0|1312M 10M 4596M 48M| 0 0 | 0 0 : 0 0 | 160k 2048M| 22 4 75 0 0 0
> 1 0 0|1323M 10M 4596M 37M| 0 0 | 0 0 : 0 24k| 160k 2048M| 21 4 75 0 0 0
> 1 0 0|1334M 10M 4596M 25M| 0 0 | 0 0 : 0 56k| 160k 2048M| 21 4 75 0 0 0
>
>
>
> On Fri, 2005-08-19 at 16:07 -0500, John A Meinel wrote:
>
>>Jeremiah Jahn wrote:
>>
>>>Rebuild in progress with just ext3 on the raid array...will see if this
>>>helps the access times. If it doesn't I'll mess with the stripe size. I
>>>have REINDEXED, CLUSTERED, tablespaced and cached with 'cat table/index
>>>
>>>
>>>>/dev/null' none of this seems to have helped, or even increased my
>>>
>>>memory usage. argh! The only thing about this new system that I'm
>>>unfamiliar with is the array setup and LVM, which is why I think that's
>>>where the issue is. clustering and indexing as well as vacuum etc are
>>>things that I do and have been aware of for sometime. Perhaps slony is a
>>>factor, but I really don't see it causing problems on index read speed
>>>esp. when it's not running.
>>>
>>>thanx for your help, I really appreciate it.
>>>-jj-
>>>
>>
>>By the way, how are you measuring memory usage? Can you give the output
>>of that command, just to make sure you are reading it correctly.
>>
>>John
>>=:->
>>
Attachments:
signature.asc (application/pgp-signature, 252B) download
view thread (39+ messages) latest in thread
Message-ID: <4307E63C.7050408@arbash-meinel.com>
Permalink: ../4307E63C.7050408@arbash-meinel.com/
Also on: postgresql.org/message-id/4307E63C.7050408@arbash-meinel.com
copy link · copy postgr.es
reply Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: john@arbash-meinel.com, jeremiah@cs.earlham.edu, rjpeace@earthlink.net
Subject: Re: extremly low memory usage
In-Reply-To: <4307E63C.7050408@arbash-meinel.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox