Received: from maia.hub.org (maia-2.hub.org [200.46.204.251]) by mail.postgresql.org (Postfix) with ESMTP id 5E0B11337BD3 for ; Wed, 2 Mar 2011 19:54:10 -0400 (AST) Received: from mail.postgresql.org ([200.46.204.86]) by maia.hub.org (mx1.hub.org [200.46.204.251]) (amavisd-maia, port 10024) with ESMTP id 61257-02; Wed, 2 Mar 2011 23:54:03 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from b.mail.sonic.net (b.mail.sonic.net [64.142.19.5]) by mail.postgresql.org (Postfix) with ESMTP id CBC2D133616B; Wed, 2 Mar 2011 19:54:02 -0400 (AST) Received: from newshell.sonic.net (bolt.sonic.net [208.201.242.19]) by b.mail.sonic.net (8.13.8.Beta0-Sonic/8.13.7) with ESMTP id p22Nrucd008616 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 2 Mar 2011 15:53:56 -0800 Received: from bolt.sonic.net (IDENT:+ktIRZBeh2gAVhixm+js5GTsyIz8Ftxq@localhost.localdomain [127.0.0.1]) by newshell.sonic.net (8.13.6/8.12.1) with ESMTP id p22Nrs0m014927; Wed, 2 Mar 2011 15:53:54 -0800 Received: (from daveg@localhost) by bolt.sonic.net (8.13.6/8.12.1/Submit) id p22NrseA014926; Wed, 2 Mar 2011 15:53:54 -0800 Date: Wed, 2 Mar 2011 15:53:54 -0800 From: daveg To: Alvaro Herrera Cc: Heikki Linnakangas , Maxim Boguk , Fujii Masao , pgsql-admin , PostgreSQL-development Subject: Re: [HACKERS] Re: PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum Message-ID: <20110302235354.GP26397@sonic.net> References: <4D2D6A56.9060808@enterprisedb.com> <20110228212832.GH26397@sonic.net> <4D6C1B16.6030708@enterprisedb.com> <20110301212043.GN26397@sonic.net> <20110302213034.GO26397@sonic.net> <1299101996-sup-4206@alvh.no-ip.org> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <1299101996-sup-4206@alvh.no-ip.org> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=-1.91 tagged_above=-10 required=5 tests=BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, T_RP_MATCHES_RCVD=-0.01 X-Spam-Level: X-Archive-Number: 201103/19 X-Sequence-Number: 36891 On Wed, Mar 02, 2011 at 06:45:13PM -0300, Alvaro Herrera wrote: > Excerpts from daveg's message of mié mar 02 18:30:34 -0300 2011: > > > After a restart and vacuum of all dbs with no other activity things were > > quiet for a couple hours and then we started seeing these PD_ALL_VISIBLE > > messages again. > > > > Going back through the logs we have been getting these since at least before > > mid January. Oddly, this only happens on four systems which are all new Dell > > 32 core Nehalem 512GB machines using iscsi partitions served off a Netapp. > > Our older 8 core 64GB hosts have never logged any of these errors. I'm not > > saying it is related to the hw, as these hosts are doing a lot more work than > > the old hosts so it may be a concurrency problem that just never came up at > > lower levels before. > > > > Postgresql version is 8.4.4. > > I don't see how this could be related, but since you're running on NFS, > maybe it is, somehow: > http://archives.postgresql.org/message-id/4D40DDB7.1010000@credativ.com > (for example what if the visibility map fork's last page is overwritten?) Running on ISCSI, not nfs. But it is still a Netapp, so who knows. I'll look. Also, we are not seeing any of the "unexpected data beyond EOF" errors, just thousands per day of the PD_ALL_VISIBLE error. -dg -- David Gould daveg@sonic.net 510 536 1443 510 282 0869 If simplicity worked, the world would be overrun with insects.