X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id ACF243A576F for ; Tue, 25 Jan 2005 15:13:21 +0000 (GMT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 72150-04 for ; Tue, 25 Jan 2005 15:12:46 +0000 (GMT) Received: from wproxy.gmail.com (wproxy.gmail.com [64.233.184.207]) by svr1.postgresql.org (Postfix) with ESMTP id 6E1E83A540F for ; Tue, 25 Jan 2005 15:10:51 +0000 (GMT) Received: by wproxy.gmail.com with SMTP id 67so314642wri for ; Tue, 25 Jan 2005 07:10:52 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:mime-version:content-type:content-transfer-encoding; b=eskBMGfjslkt4FKwL0/Y1IpvBXN04zrBhNkpmHHEo5xTz0thUhpBgiLMs+dlVT+DcInpZkvMGpikT38c+QNJr+KZTJb2CYya+//4Oax6Erv9eBAtjZu5WAGWNfKDFe1TfTSpN4O2d4L/IQ9nRf/OPRCSdIg3jVOzY0JIsqfxY2w= Received: by 10.54.53.6 with SMTP id b6mr48109wra; Tue, 25 Jan 2005 07:10:51 -0800 (PST) Received: by 10.54.4.71 with HTTP; Tue, 25 Jan 2005 07:10:50 -0800 (PST) Message-ID: <6c21003b05012507104989efed@mail.gmail.com> Date: Tue, 25 Jan 2005 09:10:50 -0600 From: Don Drake Reply-To: Don Drake To: pgsql-performance@postgresql.org Subject: Postgres stopped running (shmget failed) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.024 tagged_above=0 required=5 tests=RCVD_BY_IP X-Spam-Level: X-Archive-Number: 200501/464 X-Sequence-Number: 10154 My db server is running under high load recently and the number of connections during the morning hours is actually very high. This morning I found the postgres not running and the following in my log file: DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory. HINT: In a moment you should be able to reconnect to the database and repeat your command. 2005-01-25 01:38:00 WARNING: terminating connection because of crash of another server process DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory. HINT: In a moment you should be able to reconnect to the database and repeat your command. 2005-01-25 01:38:05 WARNING: terminating connection because of crash of another server process DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory. HINT: In a moment you should be able to reconnect to the database and repeat your command. 2005-01-25 01:38:16 LOG: all server processes terminated; reinitializing 2005-01-25 01:38:22 FATAL: could not create shared memory segment: Cannot allocate memory DETAIL: Failed system call was shmget(key=5432001, size=273383424, 03600). HINT: This error usually means that PostgreSQL's request for a shared memory segment exceeded available memory or swap space. To reduce the request size (currently 273383424 bytes), reduce PostgreSQL's shared_buffers parameter (currently 32768) and/or its max_connections parameter (currently 40). The PostgreSQL documentation contains more information about shared memory configuration. 2005-01-25 08:00:07 LOG: database system was interrupted at 2005-01-25 00:30:15 CST I'm confused to as to what is the problem. My shared memory kernel setting are as follows: [root@katie data]# tail /etc/sysctl.conf # Controls whether core dumps will append the PID to the core filename. # Useful for debugging multi-threaded applications. kernel.core_uses_pid = 1 # For POSTGRESQL -Drake 8/1/04 kernel.shmall = 2097152 kernel.shmmax = 1073741824 kernel.shmmni = 4096 kernel.sem = 250 32000 100 128 [root@katie data]# cat /proc/sys/kernel/shmall 2097152 [root@katie data]# cat /proc/sys/kernel/shmmax 1073741824 Here's my ipcs output after restarting the server: [root@katie data]# ipcs ------ Shared Memory Segments -------- key shmid owner perms bytes nattch status 0x0052e2c1 196608 postgres 600 273383424 11 ------ Semaphore Arrays -------- key semid owner perms nsems 0x0052e2c1 589824 postgres 600 17 0x0052e2c2 622593 postgres 600 17 0x0052e2c3 655362 postgres 600 17 ------ Message Queues -------- key msqid owner perms used-bytes messages I have 2GB of RAM, is this telling me I need more RAM? There are some other processes running on this server besides postgres. Thanks. -Don -- Donald Drake President Drake Consulting http://www.drakeconsult.com/ 312-560-1574