pg.ddx.io  pgsql-admin@postgresql.org mailing list archive  
help / color / mirror / Atom feed
vacuum killed because of out of memory
3+ messages / 3 participants
[nested] [flat]

* vacuum killed because of out of memory
@ 2007-08-27 14:13 Geoffrey <lists@serioustechnology.com>
  2007-08-27 14:21 ` Re: vacuum killed because of out of memory Alvaro Herrera <alvherre@commandprompt.com>
  2007-08-29 17:43 ` Re: vacuum killed because of out of memory Scott Marlowe <scott.marlowe@gmail.com>
  0 siblings, 2 replies; 3+ messages in thread

From: Geoffrey @ 2007-08-27 14:13 UTC (permalink / raw)
  To: pgsql-admin

I've recently reviewed the various recent threads on out of memory 
problems.  We just had a similar issue last night.  We have 11 
postmasters running on two machines in a cluster environment.  Five on 
one, six on the other.  They've been running in this manner for a little 
over a year now.

Configuration:
Quad dual-core Opertons
8 gig memory
Red Hat Advance Server 4

relevant postgresql.conf settings:

tcpip_socket = true
max_connections = 35
shared_buffers = 16000
checkpoint_segments = 10
log_min_error_statement = warning
log_connections = true
log_pid = true
log_timestamp = true

We run a 'vacuum full analyze' once a week (and I've seen a thread that 
says this should not be necessary).

Just the same, last night, while running a nightly 'vacuum full' process 
for our largest database (7.5G base), the vacuum process was killed by 
the OS because of out of memory issues.

Aug 27 00:59:07 gan-lxc-01 kernel: Out of Memory: Killed process 26169 
(postmaster).

The process 26169 does appear to correspond to the vacuum process and 
not the database postmaster process.  The postmaster process did not 
die.  We did see the following in the database log:

2007-08-27 00:59:07 [13586] LOG:  server process (PID 26169) was 
terminated by signal 9
2007-08-27 00:59:07 [13586] LOG:  terminating any other active server 
processes
2007-08-27 00:59:07 [7790] WARNING:  terminating connection because of 
crash of another server process
DETAIL:  The postmaster has commanded this server process to roll back 
the current transaction and exit, because another server process exited 
abnormally and possibly corrupted shared memory.
HINT:  In a moment you should be able to reconnect to the database and 
repeat your command.
2007-08-27 00:59:07 [13586] LOG:  all server processes terminated; 
reinitializing
2007-08-27 00:59:07 [2031] LOG:  database system was interrupted at 
2007-08-27 00:58:59 EDT
2007-08-27 00:59:07 [2031] LOG:  checkpoint record is at 18/B3DF3B94
2007-08-27 00:59:07 [2031] LOG:  redo record is at 18/B3DF3B94; undo 
record is at 0/0; shutdown FALSE
2007-08-27 00:59:07 [2031] LOG:  next transaction ID: 63340557; next 
OID: 6459085
2007-08-27 00:59:07 [2031] LOG:  database system was not properly shut 
down; automatic recovery in progress
2007-08-27 00:59:07 [2031] LOG:  redo starts at 18/B3DF3BD4
2007-08-27 00:59:08 [2033] LOG:  connection received: 
host=198.212.166.38 port=33787
2007-08-27 00:59:08 [2033] FATAL:  the database system is starting up
2007-08-27 00:59:11 [2035] LOG:  connection received: 
host=XXX.XXX.XXX.XXX port=33788

So, my question is, based on the configuration of this box and the 
configuration of postgresql, can anyone point to anything that might 
cause this to happen?

-- 
Until later, Geoffrey

Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety.
  - Benjamin Franklin



^ permalink  raw  reply  [nested|flat] 3+ messages in thread

* Re: vacuum killed because of out of memory
  2007-08-27 14:13 vacuum killed because of out of memory Geoffrey <lists@serioustechnology.com>
@ 2007-08-27 14:21 ` Alvaro Herrera <alvherre@commandprompt.com>
  1 sibling, 0 replies; 3+ messages in thread

From: Alvaro Herrera @ 2007-08-27 14:21 UTC (permalink / raw)
  To: Geoffrey <lists@serioustechnology.com>; +Cc: pgsql-admin

Geoffrey wrote:

> Aug 27 00:59:07 gan-lxc-01 kernel: Out of Memory: Killed process 26169 
> (postmaster).

> So, my question is, based on the configuration of this box and the 
> configuration of postgresql, can anyone point to anything that might cause 
> this to happen?

An operating system configured to overcommit memory will do this.  The
usual suggestion is to configure it not to.

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support



^ permalink  raw  reply  [nested|flat] 3+ messages in thread

* Re: vacuum killed because of out of memory
  2007-08-27 14:13 vacuum killed because of out of memory Geoffrey <lists@serioustechnology.com>
@ 2007-08-29 17:43 ` Scott Marlowe <scott.marlowe@gmail.com>
  1 sibling, 0 replies; 3+ messages in thread

From: Scott Marlowe @ 2007-08-29 17:43 UTC (permalink / raw)
  To: Geoffrey <lists@serioustechnology.com>; +Cc: pgsql-admin

On 8/27/07, Geoffrey <lists@serioustechnology.com> wrote:
> I've recently reviewed the various recent threads on out of memory
> problems.  We just had a similar issue last night.  We have 11
> postmasters running on two machines in a cluster environment.  Five on
> one, six on the other.  They've been running in this manner for a little
> over a year now.

> So, my question is, based on the configuration of this box and the
> configuration of postgresql, can anyone point to anything that might
> cause this to happen?


Too high a setting for maint_work_mem



^ permalink  raw  reply  [nested|flat] 3+ messages in thread


end of thread, other threads:[~2007-08-29 17:43 UTC | newest]

Thread overview: 3+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2007-08-27 14:13 vacuum killed because of out of memory Geoffrey <lists@serioustechnology.com>
2007-08-27 14:21 ` Alvaro Herrera <alvherre@commandprompt.com>
2007-08-29 17:43 ` Scott Marlowe <scott.marlowe@gmail.com>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox