pg.ddx.io pgsql-admin@postgresql.org mailing list archive
help / color / mirror / Atom feedcache Memory of server
5+ messages / 3 participants
[nested] [flat]
* cache Memory of server
@ 2015-06-08 13:53 AL-Temimi, Muthana <muthana.al-temimi@tu-harburg.hamburg.de>
2015-06-08 14:15 ` Re: cache Memory of server lst_hoe02@kwsoft.de
2015-06-08 14:22 ` Re: cache Memory of server Jan Lentfer <Jan.Lentfer@web.de>
0 siblings, 2 replies; 5+ messages in thread
From: AL-Temimi, Muthana @ 2015-06-08 13:53 UTC (permalink / raw)
To: pgsql-admin
See the free command:
am 08.06.2015 um 15:13 Uhr: --active connection: 305
srvpgsql1:/opt/pgsql_data # free
total used free shared buffers cached
Mem: 12199684 8758400 3441284 1269784 231324 7139400
-/+ buffers/cache: 1387676 10812008
Swap: 6289404 0 6289404
----------------------------------------------------------------------
am 08.06.2015 um 15:53 Uhr: --active connection: 278
srvpgsql1:/opt/pgsql_data # free
total used free shared buffers cached
Mem: 12199684 8686228 3513456 1269784 232164 7164288
-/+ buffers/cache: 1289776 10909908
Swap: 6289404 0 6289404
Von: AL-Temimi, Muthana
Gesendet: Montag, 8. Juni 2015 15:24
An: pgsql-admin@postgresql.org
Betreff: cache Memory
Hello,
we have postgresql 9.1 installed on Suse Enterprice Linux Server. The RAM is 12GB, When the concurrent connections 275 reached the cached memory on Server 5GB and when the concurrent increased into 305 increase also the cache memory 8.5GB. but when the concurrent connections decreased into for example 165 the cache memory not decreased. The question why? Did I mess something the postgresql.conf
any help will be grateful..
Best Regards
Muthana AL-Temimi
M.Sc. Informations- und Kommunikations-Systeme
Technische Universitaet Hamburg Harburg
-Rechenzentrum-
Am Schwarzenberg-Campus 3
D-21073 Hamburg
Tel.: +49.40.42878.2338
Fax.: +49.40.42793.5160
E-Mail: m.al@tu-harburg.de<mailto:m.al@tu-harburg.de>
http://www.tu-harburg.de/rzt
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: cache Memory of server
2015-06-08 13:53 cache Memory of server AL-Temimi, Muthana <muthana.al-temimi@tu-harburg.hamburg.de>
@ 2015-06-08 14:15 ` lst_hoe02@kwsoft.de
1 sibling, 0 replies; 5+ messages in thread
From: lst_hoe02@kwsoft.de @ 2015-06-08 14:15 UTC (permalink / raw)
To: pgsql-admin
Zitat von "AL-Temimi, Muthana" <muthana.al-temimi@tu-harburg.hamburg.de>:
> See the free command:
>
> am 08.06.2015 um 15:13 Uhr: --active connection: 305
> srvpgsql1:/opt/pgsql_data # free
> total used free shared buffers cached
> Mem: 12199684 8758400 3441284 1269784 231324 7139400
> -/+ buffers/cache: 1387676 10812008
> Swap: 6289404 0 6289404
>
> ----------------------------------------------------------------------
>
> am 08.06.2015 um 15:53 Uhr: --active connection: 278
>
> srvpgsql1:/opt/pgsql_data # free
> total used free shared buffers cached
> Mem: 12199684 8686228 3513456 1269784 232164 7164288
> -/+ buffers/cache: 1289776 10909908
> Swap: 6289404 0 6289404
>
This is the memory usage of the system (OS) and at least on Linux it
is using all available RAM for caching as needed and release it only
it it's needed otherwise. So what you are seeing here is expected
behaviour of Linux...
Regards
Andreas
Attachments:
smime.p7s (application/pkcs7-signature, 5.8K) download
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: cache Memory of server
2015-06-08 13:53 cache Memory of server AL-Temimi, Muthana <muthana.al-temimi@tu-harburg.hamburg.de>
@ 2015-06-08 14:22 ` Jan Lentfer <Jan.Lentfer@web.de>
2015-06-09 08:15 ` Re: cache Memory of server Jan Lentfer <Jan.Lentfer@web.de>
1 sibling, 1 reply; 5+ messages in thread
From: Jan Lentfer @ 2015-06-08 14:22 UTC (permalink / raw)
To: pgsql-admin
Am 2015-06-08 15:53, schrieb AL-Temimi, Muthana:
> See the free command:
>
> am 08.06.2015 um 15:13 Uhr: --active connection: 305
>
> srvpgsql1:/opt/pgsql_data # free
>
> total used free shared buffers cached
>
> Mem: 12199684 8758400 3441284 1269784 231324 7139400
>
> -/+ buffers/cache: 1387676 10812008
>
> Swap: 6289404 0 6289404
>
>
> ----------------------------------------------------------------------
>
>
> am 08.06.2015 um 15:53 Uhr: --active connection: 278
>
> srvpgsql1:/opt/pgsql_data # free
>
> total used free shared buffers cached
>
> Mem: 12199684 8686228 3513456 1269784 232164 7164288
>
> -/+ buffers/cache: 1289776 10909908
>
> Swap: 6289404 0 6289404
>
That is basically what Scott said: you are watching the Kernel FS
cache. It may only be a coincidence that it increased together with the
postgres sessions. A high number here is usually somehting good, because
a lot of your filesystems reads will be served from RAM. Looking at your
numbers, I would say you are "all good" (except as Scott said, mabye try
to reduce number of parallel sessions) - big fs cache and still free
RAM.
What are your settings for shared buffers btw?
Jan
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: cache Memory of server
2015-06-08 13:53 cache Memory of server AL-Temimi, Muthana <muthana.al-temimi@tu-harburg.hamburg.de>
2015-06-08 14:22 ` Re: cache Memory of server Jan Lentfer <Jan.Lentfer@web.de>
@ 2015-06-09 08:15 ` Jan Lentfer <Jan.Lentfer@web.de>
2015-06-09 08:47 ` Re: cache Memory of server Jan Lentfer <Jan.Lentfer@web.de>
0 siblings, 1 reply; 5+ messages in thread
From: Jan Lentfer @ 2015-06-09 08:15 UTC (permalink / raw)
To: AL-Temimi, Muthana <muthana.al-temimi@tu-harburg.hamburg.de>; +Cc: pgsql-admin
Shared buffers seem too low to me, usually you go for 20 - 25% of RAM for a dedicated DB server. work_mem also seems low, but that depends on your actual queries. You should at least log the usage of temp files and maybe look into pgbadger to analyze your logs.
Max_connections is way too high - espcially when you use a pooler. I get along using ca. 100 connections serving several hundreds of users using jdbc pooling.
You might want to take a look at pgtune.
Von meinem iPad gesendet
> Am 09.06.2015 um 10:07 schrieb AL-Temimi, Muthana <muthana.al-temimi@tu-harburg.hamburg.de>:
>
> Hello Jan,
>
> The shared_buffers ist 1024MB of the postgresql database and the kernel.shmmax=2147483648 (2GB) of linux OS.
>
> And here is the some postgresql configurations:
>
> work_mem=4MB
> max_connections=2800
> shared_buffers=1024MB
>
> and the configuration of pgpool
> init_childern=350
> max_pool=4
>
> Regards
> Muthana
>
> -----Ursprüngliche Nachricht-----
> Von: pgsql-admin-owner@postgresql.org [mailto:pgsql-admin-owner@postgresql.org] Im Auftrag von Jan Lentfer
> Gesendet: Montag, 8. Juni 2015 16:23
> An: pgsql-admin@postgresql.org
> Betreff: Re: [ADMIN] cache Memory of server
>
> Am 2015-06-08 15:53, schrieb AL-Temimi, Muthana:
>> See the free command:
>>
>> am 08.06.2015 um 15:13 Uhr: --active connection: 305
>>
>> srvpgsql1:/opt/pgsql_data # free
>>
>> total used free shared buffers cached
>>
>> Mem: 12199684 8758400 3441284 1269784 231324 7139400
>>
>> -/+ buffers/cache: 1387676 10812008
>>
>> Swap: 6289404 0 6289404
>>
>>
>> ----------------------------------------------------------------------
>>
>>
>> am 08.06.2015 um 15:53 Uhr: --active connection: 278
>>
>> srvpgsql1:/opt/pgsql_data # free
>>
>> total used free shared buffers cached
>>
>> Mem: 12199684 8686228 3513456 1269784 232164 7164288
>>
>> -/+ buffers/cache: 1289776 10909908
>>
>> Swap: 6289404 0 6289404
>>
>
>
> That is basically what Scott said: you are watching the Kernel FS cache. It may only be a coincidence that it increased together with the postgres sessions. A high number here is usually somehting good, because a lot of your filesystems reads will be served from RAM. Looking at your numbers, I would say you are "all good" (except as Scott said, mabye try to reduce number of parallel sessions) - big fs cache and still free RAM.
> What are your settings for shared buffers btw?
>
> Jan
>
>
> --
> Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org) To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-admin
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: cache Memory of server
2015-06-08 13:53 cache Memory of server AL-Temimi, Muthana <muthana.al-temimi@tu-harburg.hamburg.de>
2015-06-08 14:22 ` Re: cache Memory of server Jan Lentfer <Jan.Lentfer@web.de>
2015-06-09 08:15 ` Re: cache Memory of server Jan Lentfer <Jan.Lentfer@web.de>
@ 2015-06-09 08:47 ` Jan Lentfer <Jan.Lentfer@web.de>
0 siblings, 0 replies; 5+ messages in thread
From: Jan Lentfer @ 2015-06-09 08:47 UTC (permalink / raw)
To: AL-Temimi, Muthana <muthana.al-temimi@tu-harburg.hamburg.de>; +Cc: pgsql-admin
Please keep the list in copy.
Look at the log_ configuration lines in postgresql.conf, especially log_temp_files. 2GB shared buffers seems more reasonable to me. The max_connection setting seems like a waste of resources to me, but I need to reread the pgpool manual. But maybe your num_init_children is too high in the first place? You start at 350 initial sessions?
Von meinem iPad gesendet
> Am 09.06.2015 um 10:29 schrieb AL-Temimi, Muthana <muthana.al-temimi@tu-harburg.hamburg.de>:
>
> the max_connection=2800 because requiered configurations from pgpools see below:
>
> max_pool*num_init_children <= (max_connections - superuser_reserved_connections) (no query canceling needed)
> max_pool*num_init_children*2 <= (max_connections - superuser_reserved_connections) (query canceling needed)
>
> and here is the link for it: http://www.pgpool.net/docs/latest/pgpool-en.html
>
> should I increase the shared_buffer of the postgresql to 2GB instead of 1GB?
> Work_mem is just for sort the result. What should be the expacted value of work_mem?
> I don't know how to log the usage of temp files?
>
> Regards
> Muthana
>
> -----Ursprüngliche Nachricht-----
> Von: Jan Lentfer [mailto:Jan.Lentfer@web.de]
> Gesendet: Dienstag, 9. Juni 2015 10:16
> An: AL-Temimi, Muthana
> Cc: pgsql-admin
> Betreff: Re: AW: [ADMIN] cache Memory of server
>
> Shared buffers seem too low to me, usually you go for 20 - 25% of RAM for a dedicated DB server. work_mem also seems low, but that depends on your actual queries. You should at least log the usage of temp files and maybe look into pgbadger to analyze your logs.
> Max_connections is way too high - espcially when you use a pooler. I get along using ca. 100 connections serving several hundreds of users using jdbc pooling.
> You might want to take a look at pgtune.
>
> Von meinem iPad gesendet
>
>> Am 09.06.2015 um 10:07 schrieb AL-Temimi, Muthana <muthana.al-temimi@tu-harburg.hamburg.de>:
>>
>> Hello Jan,
>>
>> The shared_buffers ist 1024MB of the postgresql database and the kernel.shmmax=2147483648 (2GB) of linux OS.
>>
>> And here is the some postgresql configurations:
>>
>> work_mem=4MB
>> max_connections=2800
>> shared_buffers=1024MB
>>
>> and the configuration of pgpool
>> init_childern=350
>> max_pool=4
>>
>> Regards
>> Muthana
>>
>> -----Ursprüngliche Nachricht-----
>> Von: pgsql-admin-owner@postgresql.org [mailto:pgsql-admin-owner@postgresql.org] Im Auftrag von Jan Lentfer
>> Gesendet: Montag, 8. Juni 2015 16:23
>> An: pgsql-admin@postgresql.org
>> Betreff: Re: [ADMIN] cache Memory of server
>>
>> Am 2015-06-08 15:53, schrieb AL-Temimi, Muthana:
>>> See the free command:
>>>
>>> am 08.06.2015 um 15:13 Uhr: --active connection: 305
>>>
>>> srvpgsql1:/opt/pgsql_data # free
>>>
>>> total used free shared buffers cached
>>>
>>> Mem: 12199684 8758400 3441284 1269784 231324 7139400
>>>
>>> -/+ buffers/cache: 1387676 10812008
>>>
>>> Swap: 6289404 0 6289404
>>>
>>>
>>> ----------------------------------------------------------------------
>>>
>>>
>>> am 08.06.2015 um 15:53 Uhr: --active connection: 278
>>>
>>> srvpgsql1:/opt/pgsql_data # free
>>>
>>> total used free shared buffers cached
>>>
>>> Mem: 12199684 8686228 3513456 1269784 232164 7164288
>>>
>>> -/+ buffers/cache: 1289776 10909908
>>>
>>> Swap: 6289404 0 6289404
>>
>>
>> That is basically what Scott said: you are watching the Kernel FS cache. It may only be a coincidence that it increased together with the postgres sessions. A high number here is usually somehting good, because a lot of your filesystems reads will be served from RAM. Looking at your numbers, I would say you are "all good" (except as Scott said, mabye try to reduce number of parallel sessions) - big fs cache and still free RAM.
>> What are your settings for shared buffers btw?
>>
>> Jan
>>
>>
>> --
>> Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org) To make changes to your subscription:
>> http://www.postgresql.org/mailpref/pgsql-admin
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 5+ messages in thread
end of thread, other threads:[~2015-06-09 08:47 UTC | newest]
Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2015-06-08 13:53 cache Memory of server AL-Temimi, Muthana <muthana.al-temimi@tu-harburg.hamburg.de>
2015-06-08 14:15 ` lst_hoe02@kwsoft.de
2015-06-08 14:22 ` Jan Lentfer <Jan.Lentfer@web.de>
2015-06-09 08:15 ` Jan Lentfer <Jan.Lentfer@web.de>
2015-06-09 08:47 ` Jan Lentfer <Jan.Lentfer@web.de>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox