Received: from maia.hub.org (maia-3.hub.org [200.46.204.243]) by mail.postgresql.org (Postfix) with ESMTP id 668771337B93 for ; Tue, 8 Mar 2011 01:25:18 -0400 (AST) Received: from mail.postgresql.org ([200.46.204.86]) by maia.hub.org (mx1.hub.org [200.46.204.243]) (amavisd-maia, port 10024) with ESMTP id 61755-08 for ; Tue, 8 Mar 2011 05:25:11 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from web39705.mail.mud.yahoo.com (web39705.mail.mud.yahoo.com [209.191.106.51]) by mail.postgresql.org (Postfix) with SMTP id B2F5F1337B5D for ; Tue, 8 Mar 2011 01:25:10 -0400 (AST) Received: (qmail 4800 invoked by uid 60001); 8 Mar 2011 05:25:09 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1299561909; bh=XI8LDmACitM8OXcfP8EgiZU/9K15SiHvl0MRVUvPDk8=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=fQEES3zv8RIErIQkz+RM2YzDre9L4gc9ElIwkidTXpBkve1nGGs66w26cnWmR3mBJ7bryuVtbqxJwxt6Me3WGR0UsOeCNYw5ZPRc89iTFlT/d+NuXk/gXqAbsrmziJdXW5t9nsZX1Xun/LGM9ETBfyByS85Vs044hbHTXYC7vG8= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=hDfKBRMR4aFqtYS0LLZP+KPgRVGwn1DTA0SXBm3WnLlVtVyc5bKfcNLCVUlE/Y1STIxTm9iCoQDR9krpqi42AjPvV5cTEkKW1wfmhdl086Vwr/WJP1AhIlFjrCrfZ/Uv+j+ZGKoKGUArQfOIKzIEYDZSy2EGTnxSfChokTwrOIw=; Message-ID: <583856.4119.qm@web39705.mail.mud.yahoo.com> X-YMail-OSG: xe1.jsAVM1lvbPe0BHQWgJdrl_DjlXO2XnlOdMT14l1vHVY 4QeymXN.HsgrB.I9g.oGUcMT482cbN02C8mTDHOJwTHBsJHjsU2SO5cXWoZ_ 26HmC.jLge.SNYrWQo4VsRiUL.EIzydRkNSvs_MojSxBv6JpFzc5V7AEtxzH Zyzu8P7qj1anzz4UJRRA9aqvySDVs5CFKSz3t5cGYWx3Gucs_BonxHUqeDTu b9SRQE9v4ZomxcUFAEqSn6e3ucB3E_jrYXeZUCQgBvSgjCqKzs71EeFFa3U2 d_C2_4aA9fwg.m0Mar_hW_vXU5SRJy1.S2RUg2OiJGNmyQ.j7NG_7uQFzX9Q DfU.sqaqtQJTo4J5D Received: from [69.76.130.83] by web39705.mail.mud.yahoo.com via HTTP; Mon, 07 Mar 2011 21:25:09 PST X-Mailer: YahooMailClassic/11.4.20 YahooMailWebService/0.8.109.292656 Date: Mon, 7 Mar 2011 21:25:09 -0800 (PST) From: Bob Lunney Subject: Re: PG Server Crash To: pgsql-admin@postgresql.org In-Reply-To: <624088.44544.qm@web39706.mail.mud.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=-1.898 tagged_above=-10 required=5 tests=BAYES_00=-1.9, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RFC_ABUSE_POST=0.001, T_RP_MATCHES_RCVD=-0.01, T_TO_NO_BRKTS_FREEMAIL=0.01 X-Spam-Level: X-Archive-Number: 201103/85 X-Sequence-Number: 36957 Sorry, I should have included the fact that PG was compiled from source on = the same machine where it runs using gcc 4.1.0, and config.log has:=0A=0A./= configure --with-python --with-gssapi --enable-thread-safety=0A=0ABob Lunne= y=0A=0A--- On Mon, 3/7/11, Bob Lunney wrote:=0A=0A> = From: Bob Lunney =0A> Subject: [ADMIN] PG Server Cras= h=0A> To: pgsql-admin@postgresql.org=0A> Date: Monday, March 7, 2011, 11:24= PM=0A> OK, I got a nasty surprise=0A> today.=A0 The server threw everybody= out of the pool and=0A> logged this:=0A> =0A> 10.224.12.4(52763) idle: pth= read_mutex_lock.c:80:=0A> __pthread_mutex_lock: Assertion `mutex->__data.__= owner =3D=3D=0A> 0' failed.=0A> :4d45a006.78ec:2011-03-07 11:57:45.316 EST:= LOG:=A0=0A> server process (PID 21298) was terminated by signal 6:=0A> Abor= ted=0A> :4d45a006.78ec:2011-03-07 11:57:45.316 EST:LOG:=A0=0A> terminating = any other active server processes=0A> =0A> It then stopped all the backends= , restarted, recovered and=0A> worked fine for the rest of the day.=0A> =0A= > The particulars:=0A> =0A> SLES 10=0A> PG 8.4.1=0A> 8-way Intel Xeon E5345= @ 2.33GHz=0A> 32 GB RAM=0A> =0A> max_connections =3D 1500=0A> shared_buffe= rs =3D 1536MB=0A> work_mem =3D 32MB=A0 =A0 =A0 =A0 =A0 =A0=0A> =A0 =A0 =A0 = =A0 =A0 =A0=A0=A0=0A> maintenance_work_mem =3D 256MB=A0 =A0 =A0 =A0=0A> =A0= =A0 =0A> checkpoint_segments =3D 2048=0A> effective_cache_size =3D 20GB=0A= > effective_io_concurrency =3D 6=0A> random_page_cost =3D 2.0=0A> =0A> =0A>= This server has been rock solid running 8.4.1 for over a=0A> year.=A0 It h= as Slony replicating to another mirror image=0A> server that has not had an= y problems whatsoever.=0A> =0A> Any ideas?=0A> =0A> Thanks in advance for a= ll your help, and especially for the=0A> amazing database software!=0A> =0A= > Regards,=0A> =0A> Bob Lunney=0A> =0A> =0A> =A0 =A0 =A0 =0A> =0A> -- =0A> = Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)=0A> To make = changes to your subscription:=0A> http://www.postgresql.org/mailpref/pgsql-= admin=0A> =0A=0A=0A