pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedSHMMAX / SHMALL Was (Re: postgresql-8.0.1 performance tuning)
5+ messages / 3 participants
[nested] [flat]
* SHMMAX / SHMALL Was (Re: postgresql-8.0.1 performance tuning)
@ 2005-06-03 03:45 Paul McGarry <paul.mcgarry@gmail.com>
2 siblings, 1 reply; 5+ messages in thread
From: Paul McGarry @ 2005-06-03 03:45 UTC (permalink / raw)
To: Mark Kirkwood <markir@paradise.net.nz>; +Cc: Cosimo Streppone <cosimo@streppone.it>; Martin Fandel <martin.fandel@alphyra-evs.de>; pgsql-performance
On 6/1/05, Mark Kirkwood <markir@paradise.net.nz> wrote:
> Cosimo Streppone wrote:
> > ######### Config ############
> >> /etc/sysctl.conf:
> >> kernel.shmall = 786432000
> >> kernel.shmmax = 786432000
> >
> > I think you have a problem here.
> > kernel.shmmax should *not* be set to an amount of RAM, but
> > to maximum number of shared memory pages, which on a typical linux system
> > is 4kb. Google around:
> >
> This is somewhat confusing :
>
> kernel.shmmax is in bytes (max single segment size)
> kernel.shmall is in (4k) pages (max system wide allocated segment pages)
Can someone resummarize the situation with these linux parameters for
the dummies? I thought I had my calculations all sorted out but now
I've confused myself again.
The documentation at
http://www.postgresql.org/docs/8.0/interactive/kernel-resources.html
puts the same figure into both values but the posts here seem to
suggest that is wrong?
Or is it different on a 2.4 kernel and the documentation needs updating?
In my specific case I have about 800meg of memory on a linux 2.4 kernel box.
Based on the powerpostgresql.com Performance Checklist [1] and
Annotated Postgresql.conf [2] I understand that:
-I should have less than 1/3 of my total memory as shared_buffers
-For my server 15000 is a fairly reasonable starting point for
shared_buffers which is ~120MB
-I have 100 max_connections.
So I was going to set SHMMAX to 134217728 (ie 128 Meg)
What should SHMALL be?
The current system values are
postgres@localhost:~/data$ cat /proc/sys/kernel/shmmax
33554432
postgres@localhost:~/data$ cat /proc/sys/kernel/shmall
2097152
ie SHMALL seems to be 1/16 of SHMMAX
Paul
[1] http://www.powerpostgresql.com/PerfList/
[2] http://www.powerpostgresql.com/Downloads/annotated_conf_80.html
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: SHMMAX / SHMALL Was (Re: postgresql-8.0.1 performance
@ 2005-06-03 06:11 Mark Kirkwood <markir@paradise.net.nz>
parent: Paul McGarry <paul.mcgarry@gmail.com>
0 siblings, 0 replies; 5+ messages in thread
From: Mark Kirkwood @ 2005-06-03 06:11 UTC (permalink / raw)
To: paul@paulmcgarry.com; +Cc: Cosimo Streppone <cosimo@streppone.it>; Martin Fandel <martin.fandel@alphyra-evs.de>; pgsql-performance
Paul McGarry wrote:
> Based on the powerpostgresql.com Performance Checklist [1] and
> Annotated Postgresql.conf [2] I understand that:
> -I should have less than 1/3 of my total memory as shared_buffers
> -For my server 15000 is a fairly reasonable starting point for
> shared_buffers which is ~120MB
> -I have 100 max_connections.
>
> So I was going to set SHMMAX to 134217728 (ie 128 Meg)
>
> What should SHMALL be?
>
> The current system values are
> postgres@localhost:~/data$ cat /proc/sys/kernel/shmmax
> 33554432
> postgres@localhost:~/data$ cat /proc/sys/kernel/shmall
> 2097152
>
> ie SHMALL seems to be 1/16 of SHMMAX
>
No - shmall is in 4k pages _ so this amounts to 8G! This is fine -
unless you wish to decrease it in order to prevent too many shared
memory applications running.
BTW - the docs have been amended for 8.1 to suggest shmmax=134217728 and
shmall=2097152 (was going to point you at them - but I cannot find them
on the Postgresql site anymore...).
There seems to be some longstanding confusion in the Linux community
about the units for shmall (some incorrect documentation from Oracle on
the issue does not help I am sure....) - to the point where I downloaded
kernel source to check (reproducing here):
linux-2.6.11.1/include/linux/shm.h:13->
#define SHMMAX 0x2000000 /* max shared seg size (bytes) */
#define SHMMIN 1 /* min shared seg size (bytes) */
#define SHMMNI 4096 /* max num of segs system wide */
#define SHMALL (SHMMAX/PAGE_SIZE*(SHMMNI/16)) /* max shm system wide
(pages) */
#define SHMSEG SHMMNI
Hope that helps
Best wishes
Mark
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: SHMMAX / SHMALL Was (Re: postgresql-8.0.1 performance tuning)
@ 2005-06-03 08:49 Martin Fandel <martin.fandel@alphyra-evs.de>
2 siblings, 1 reply; 5+ messages in thread
From: Martin Fandel @ 2005-06-03 08:49 UTC (permalink / raw)
To: Mark Kirkwood <markir@paradise.net.nz>; +Cc: pgsql-performance
Aah ok :)
I've set my values now as follow (2GB RAM):
SHMMAX=`cat /proc/meminfo | grep MemTotal | cut -d: -f 2 | awk '{print
$1*1024/3}'`
echo kernel.shmmax=${SHMMAX} >> /etc/sysctl.conf
SHMALL=`expr ${SHMALL} / 4096 \* \( 4096 / 16 \)`
echo kernel.shmall=${SHMALL} >> /etc/sysctl.conf
sysctl.conf:
kernel.shmmax=708329472
kernel.shmall=44270592
postgresql.conf:
max_connections=500
shared_buffers=40000 # ~312MB, min. 1000, max ~ 83000
best regards,
Martin
Am Freitag, den 03.06.2005, 18:11 +1200 schrieb Mark Kirkwood:
> Paul McGarry wrote:
>
> > Based on the powerpostgresql.com Performance Checklist [1] and
> > Annotated Postgresql.conf [2] I understand that:
> > -I should have less than 1/3 of my total memory as shared_buffers
> > -For my server 15000 is a fairly reasonable starting point for
> > shared_buffers which is ~120MB
> > -I have 100 max_connections.
> >
> > So I was going to set SHMMAX to 134217728 (ie 128 Meg)
> >
> > What should SHMALL be?
> >
> > The current system values are
> > postgres@localhost:~/data$ cat /proc/sys/kernel/shmmax
> > 33554432
> > postgres@localhost:~/data$ cat /proc/sys/kernel/shmall
> > 2097152
> >
> > ie SHMALL seems to be 1/16 of SHMMAX
> >
>
> No - shmall is in 4k pages _ so this amounts to 8G! This is fine -
> unless you wish to decrease it in order to prevent too many shared
> memory applications running.
>
> BTW - the docs have been amended for 8.1 to suggest shmmax=134217728 and
> shmall=2097152 (was going to point you at them - but I cannot find them
> on the Postgresql site anymore...).
>
> There seems to be some longstanding confusion in the Linux community
> about the units for shmall (some incorrect documentation from Oracle on
> the issue does not help I am sure....) - to the point where I downloaded
> kernel source to check (reproducing here):
>
>
> linux-2.6.11.1/include/linux/shm.h:13->
>
> #define SHMMAX 0x2000000 /* max shared seg size (bytes) */
> #define SHMMIN 1 /* min shared seg size (bytes) */
> #define SHMMNI 4096 /* max num of segs system wide */
> #define SHMALL (SHMMAX/PAGE_SIZE*(SHMMNI/16)) /* max shm system wide
> (pages) */
> #define SHMSEG SHMMNI
>
>
> Hope that helps
>
> Best wishes
>
> Mark
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: SHMMAX / SHMALL Was (Re: postgresql-8.0.1 performance
@ 2005-06-03 09:10 Mark Kirkwood <markir@paradise.net.nz>
parent: Martin Fandel <martin.fandel@alphyra-evs.de>
0 siblings, 0 replies; 5+ messages in thread
From: Mark Kirkwood @ 2005-06-03 09:10 UTC (permalink / raw)
To: Martin Fandel <martin.fandel@alphyra-evs.de>; +Cc: pgsql-performance
Martin Fandel wrote:
> Aah ok :)
>
> I've set my values now as follow (2GB RAM):
>
> SHMMAX=`cat /proc/meminfo | grep MemTotal | cut -d: -f 2 | awk '{print
> $1*1024/3}'`
> echo kernel.shmmax=${SHMMAX} >> /etc/sysctl.conf
> SHMALL=`expr ${SHMALL} / 4096 \* \( 4096 / 16 \)`
> echo kernel.shmall=${SHMALL} >> /etc/sysctl.conf
>
> sysctl.conf:
> kernel.shmmax=708329472
> kernel.shmall=44270592
>
> postgresql.conf:
> max_connections=500
> shared_buffers=40000 # ~312MB, min. 1000, max ~ 83000
>
Hmmm - shmall set to 168G... err why? Apologies for nit picking a little
- but shmall seems unreasonably high. I can't see much reason for
setting it bigger than (physical RAM in bytes)/4096 myself. So in your
case this is 2*(1024*1024*1024)/4096 = 524288
Cheers
Mark
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: SHMMAX / SHMALL Was (Re: postgresql-8.0.1 performance tuning)
@ 2005-06-03 09:19 Martin Fandel <martin.fandel@alphyra-evs.de>
2 siblings, 0 replies; 5+ messages in thread
From: Martin Fandel @ 2005-06-03 09:19 UTC (permalink / raw)
To: Mark Kirkwood <markir@paradise.net.nz>; +Cc: pgsql-performance
ok i set it to 524288. ;)
Am Freitag, den 03.06.2005, 21:10 +1200 schrieb Mark Kirkwood:
> Martin Fandel wrote:
> > Aah ok :)
> >
> > I've set my values now as follow (2GB RAM):
> >
> > SHMMAX=`cat /proc/meminfo | grep MemTotal | cut -d: -f 2 | awk '{print
> > $1*1024/3}'`
> > echo kernel.shmmax=${SHMMAX} >> /etc/sysctl.conf
> > SHMALL=`expr ${SHMALL} / 4096 \* \( 4096 / 16 \)`
> > echo kernel.shmall=${SHMALL} >> /etc/sysctl.conf
> >
> > sysctl.conf:
> > kernel.shmmax=708329472
> > kernel.shmall=44270592
> >
> > postgresql.conf:
> > max_connections=500
> > shared_buffers=40000 # ~312MB, min. 1000, max ~ 83000
> >
>
> Hmmm - shmall set to 168G... err why? Apologies for nit picking a little
> - but shmall seems unreasonably high. I can't see much reason for
> setting it bigger than (physical RAM in bytes)/4096 myself. So in your
> case this is 2*(1024*1024*1024)/4096 = 524288
>
> Cheers
>
> Mark
>
^ permalink raw reply [nested|flat] 5+ messages in thread
end of thread, other threads:[~2005-06-03 09:19 UTC | newest]
Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2005-06-03 03:45 ` Paul McGarry <paul.mcgarry@gmail.com>
2005-06-03 06:11 ` Re: SHMMAX / SHMALL Was (Re: postgresql-8.0.1 performance Mark Kirkwood <markir@paradise.net.nz>
2005-06-03 08:49 ` Martin Fandel <martin.fandel@alphyra-evs.de>
2005-06-03 09:10 ` Re: SHMMAX / SHMALL Was (Re: postgresql-8.0.1 performance Mark Kirkwood <markir@paradise.net.nz>
2005-06-03 09:19 ` Martin Fandel <martin.fandel@alphyra-evs.de>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox