Received: from localhost (unknown [200.46.204.184]) by postgresql.org (Postfix) with ESMTP id 4ADFC9F98A4 for ; Sat, 15 Sep 2007 20:49:07 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.204.184]) (amavisd-maia, port 10024) with ESMTP id 30007-08 for ; Sat, 15 Sep 2007 20:48:57 -0300 (ADT) X-Greylist: from auto-whitelisted by SQLgrey-1.7.5 Received: from sss.pgh.pa.us (sss.pgh.pa.us [66.207.139.130]) by postgresql.org (Postfix) with ESMTP id 92E7F9F97D1 for ; Sat, 15 Sep 2007 20:49:04 -0300 (ADT) Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1]) by sss.pgh.pa.us (8.14.1/8.14.1) with ESMTP id l8FNn3nJ028917 for ; Sat, 15 Sep 2007 19:49:03 -0400 (EDT) To: pgsql-hackers@postgreSQL.org Subject: Hmmm ... isn't count_nondeletable_pages all wet? Date: Sat, 15 Sep 2007 19:49:03 -0400 Message-ID: <28916.1189900143@sss.pgh.pa.us> From: Tom Lane X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200709/692 X-Sequence-Number: 107844 While looking at the HOT patch I suddenly started to question the sanity of vacuumlazy.c's count_nondeletable_pages(). It sits there and does a HeapTupleSatisfiesVacuum on any tuples it finds, and is willing to truncate away a page that contains only DEAD tuples. The problem with this theory is that any index entries linking to those tuples won't have been cleaned up, and will therefore emerge as index corruption once the table grows again (because they will link to tuples that in all probability don't match the index entries). Now since this test is made using the same OldestXmin threshold that we used in the vacuuming pass, tuples that were RECENTLY_DEAD will still be that way. However, it seems to me that there's a race condition anyway: 1. VACUUM scans and cleans a page. 2. Some other transaction inserts a tuple into that page. 3. The inserting transaction aborts. 4. VACUUM returns to the page and sees the tuple as HEAPTUPLE_DEAD. In this scenario we could truncate the page away and not have cleaned up the index entries linking to it. I'm thinking that count_nondeletable_pages should not bother itself with visibility tests, but just forget truncation if it finds any items whatsoever on the page. Is this analysis accurate, or am I missing something? If it is accurate, do we need to postpone the upcoming releases to fix it? I am thinking that some previously unexplained reports of index corruption might now be explained ... regards, tom lane