Received: from maia.hub.org (maia-3.hub.org [200.46.204.243]) by mail.postgresql.org (Postfix) with ESMTP id AE75B1337B4A for ; Tue, 8 Mar 2011 00:24:09 -0400 (AST) Received: from mail.postgresql.org ([200.46.204.86]) by maia.hub.org (mx1.hub.org [200.46.204.243]) (amavisd-maia, port 10024) with ESMTP id 50367-08 for ; Tue, 8 Mar 2011 04:24:02 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from web39706.mail.mud.yahoo.com (web39706.mail.mud.yahoo.com [209.191.106.52]) by mail.postgresql.org (Postfix) with SMTP id 123E7133798E for ; Tue, 8 Mar 2011 00:24:01 -0400 (AST) Received: (qmail 44743 invoked by uid 60001); 8 Mar 2011 04:24:00 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1299558240; bh=n4CQxx3axFRETV/Zi7uzwf1Fwhei8XdVHeJgIss7zjo=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type; b=jg24ROiJZnJmpinuLxxH0YHezPXilFJYWluP78ZH0bEXrMGDPs8z6Oa/QI0t/F/BJi2nmxbi1F7UaVkM0kv1dvDN5dOqBViUqUu22oytDu90Eiz4HVpqbWrTYbUPOgz7CUZlCAkJq4T4F2+sORHMG+WngoPZ4S3DyGrYJRzL5f4= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type; b=FjBJTAgxjUPt7n0LLTwjqSbMvt0/TMx0FyTAa45KcmuukHF+EvefKE3e4TV5oJ7VikKhJHAWANOxCcHBG0i13AXPKZm1fMrpoYlKMoWLlU1+Z+BGiwIuaAIe4YYiObkN8cHVGkVUKpKrKwdmxL5b08xagC7VyN6jDUzLi+P0ZDU=; Message-ID: <624088.44544.qm@web39706.mail.mud.yahoo.com> X-YMail-OSG: dq0gHkIVM1kH5LxIP5I6xCnf3llqQCUSw3JLLMTusYyze7Q a770l.68UipzRPmIxVUIFgly0tWuclK7gxqoo14fTxclAs3I2uZ4aDul569y c1Ngybw5_.sxUIyFZnYwlWYveMUkTOQK74SBFo9WWp9n4Av4pBKQ9CfsN.NB y38LTwzP9AZjgR_4nHSjfwibpzsXQx8A_WUdcp3xQSKCpeUqLL4myuq2Fwha aFTf6ThAQ2j8hIaaxhdreJIzU86E3jb6IMcliLWMF9oA9BFWHFZiGMiEwbbm G.yy7z6pjvTjpgtAzXDuPcj8OHO3uOPaYJd_K Received: from [69.76.130.83] by web39706.mail.mud.yahoo.com via HTTP; Mon, 07 Mar 2011 20:24:00 PST X-Mailer: YahooMailClassic/11.4.20 YahooMailWebService/0.8.109.292656 Date: Mon, 7 Mar 2011 20:24:00 -0800 (PST) From: Bob Lunney Subject: PG Server Crash To: pgsql-admin@postgresql.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=-1.898 tagged_above=-10 required=5 tests=BAYES_00=-1.9, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RFC_ABUSE_POST=0.001, T_RP_MATCHES_RCVD=-0.01, T_TO_NO_BRKTS_FREEMAIL=0.01 X-Spam-Level: X-Archive-Number: 201103/84 X-Sequence-Number: 36956 OK, I got a nasty surprise today. The server threw everybody out of the pool and logged this: 10.224.12.4(52763) idle: pthread_mutex_lock.c:80: __pthread_mutex_lock: Assertion `mutex->__data.__owner == 0' failed. :4d45a006.78ec:2011-03-07 11:57:45.316 EST:LOG: server process (PID 21298) was terminated by signal 6: Aborted :4d45a006.78ec:2011-03-07 11:57:45.316 EST:LOG: terminating any other active server processes It then stopped all the backends, restarted, recovered and worked fine for the rest of the day. The particulars: SLES 10 PG 8.4.1 8-way Intel Xeon E5345 @ 2.33GHz 32 GB RAM max_connections = 1500 shared_buffers = 1536MB work_mem = 32MB maintenance_work_mem = 256MB checkpoint_segments = 2048 effective_cache_size = 20GB effective_io_concurrency = 6 random_page_cost = 2.0 This server has been rock solid running 8.4.1 for over a year. It has Slony replicating to another mirror image server that has not had any problems whatsoever. Any ideas? Thanks in advance for all your help, and especially for the amazing database software! Regards, Bob Lunney