Received: from magus.postgresql.org (magus.postgresql.org [87.238.57.229]) by mail.postgresql.org (Postfix) with ESMTP id B989223D696 for ; Mon, 23 Jul 2012 12:49:49 -0300 (ADT) Received: from mx5.framestore.com ([193.203.83.15]) by magus.postgresql.org with esmtp (Exim 4.72) (envelope-from ) id 1StKtG-0006X1-7t for pgsql-performance@postgresql.org; Mon, 23 Jul 2012 15:49:48 +0000 Received: from vm-zimbra.ldn.framestore.com ([172.16.73.68] helo=zimbra.framestore.com) by mx5.framestore.com with esmtp (Exim 4.72) (envelope-from ) id 1StKt1-000714-A6; Mon, 23 Jul 2012 16:49:33 +0100 Received: from [172.16.84.176] (sys367.ldn.framestore.com [172.16.84.176]) by zimbra.framestore.com (Postfix) with ESMTPA id ABCA71E86207; Mon, 23 Jul 2012 16:49:18 +0100 (BST) Subject: Re: Odd blocking (or massively latent) issue - even with EXPLAIN From: Jim Vanns To: Tom Lane Cc: Jim Vanns , pgsql-performance@postgresql.org In-Reply-To: <29948.1343056156@sss.pgh.pa.us> References: <1343032905.12579.3.camel@sys367.ldn.framestore.com> <29948.1343056156@sss.pgh.pa.us> Content-Type: text/plain; charset="UTF-8" Organization: Framestore Date: Mon, 23 Jul 2012 16:49:25 +0100 Message-ID: <1343058565.12579.126.camel@sys367.ldn.framestore.com> Mime-Version: 1.0 X-Mailer: Evolution 2.28.3 (2.28.3-1.fc12) Content-Transfer-Encoding: 7bit X-Pg-Spam-Score: -1.9 (-) X-Archive-Number: 201207/272 X-Sequence-Number: 47453 On Mon, 2012-07-23 at 11:09 -0400, Tom Lane wrote: > Jim Vanns writes: > > We're seeing SELECT statements and even EXPLAIN (no ANAYLZE) > > statements hang indefinitely until *something* (we don't know what) > > releases some kind of resource or no longer becomes a massive bottle > > neck. These are the symptoms. > > Does anything show up as blocked in the pg_locks view? Nope. > Could you attach to the stuck process with gdb and get a stack trace? Haven't been quite brave enough to do that yet - this is a production server. I did manage to strace a process though - it (the server side process of a psql EXPLAIN) appeared to spin on an awful lot of semop() calls with the occasional read(). Of course, in the context of a shared memory system such as postgres I'd expect to see quite a lot of semop() calls but I've no idea how much is normal and how much is excessive. Jim > regards, tom lane > -- Jim Vanns Systems Programmer Framestore