Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.80) (envelope-from ) id 1ZHHci-0005WE-II for pgsql-hackers@arkaria.postgresql.org; Mon, 20 Jul 2015 20:25:16 +0000 Received: from localhost ([127.0.0.1] helo=postgresql.org) by malur.postgresql.org with smtp (Exim 4.84) (envelope-from ) id 1ZHHch-0007XW-SA for pgsql-hackers@arkaria.postgresql.org; Mon, 20 Jul 2015 20:25:15 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.84) (envelope-from ) id 1ZHHbP-00063K-5g for pgsql-hackers@postgresql.org; Mon, 20 Jul 2015 20:23:55 +0000 Received: from mail-wi0-x22a.google.com ([2a00:1450:400c:c05::22a]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.84) (envelope-from ) id 1ZHHbL-0005t0-Op for pgsql-hackers@postgresql.org; Mon, 20 Jul 2015 20:23:54 +0000 Received: by wibud3 with SMTP id ud3so96834499wib.1 for ; Mon, 20 Jul 2015 13:23:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:reply-to:user-agent:mime-version:to:cc :subject:references:in-reply-to:content-type; bh=gZgLfg9I+8n1uOQQsarRccjPyNBNvw5K6e/1QMbuGxw=; b=LHu912VIWj6q4bRb1dj71y8S9tm5C31mDtls+BFHGBYjBwuDm/bS4PKjKfJFCzSyuS fet5cIwaechi5ie0ELT1OLUEk72haWQggdUOQQvIM7wreUeRJYZH1AJpSCosFxymVgo9 YzoZ7QFUV7EGa7jUthbe6ZBbs2gWiJeOTrP0QohhkGm6mxdXyNoZetAs31BQmtJc/60O FaOX+gBjxtn3K0ccUaMToLZ4C1PsK85lC7ml1rM1jWvvlZDO27ISWihg/tA34DPP2w3R jINz9OJeJrHwdGchhDhg34N41soVoLyNPb3x29IWQtcyv+t34XH1iTph+2ekziuMDQrc FB2w== X-Received: by 10.181.25.234 with SMTP id it10mr24431397wid.41.1437423831179; Mon, 20 Jul 2015 13:23:51 -0700 (PDT) Received: from [192.168.1.105] (dsl-hkibrasgw1-58c38f-82.dhcp.inet.fi. [88.195.143.82]) by smtp.gmail.com with ESMTPSA id w8sm33620404wja.15.2015.07.20.13.23.49 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 20 Jul 2015 13:23:50 -0700 (PDT) Message-ID: <55AD58CD.2000607@iki.fi> Date: Mon, 20 Jul 2015 23:23:41 +0300 From: Heikki Linnakangas Reply-To: hlinnaka@iki.fi User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Icedove/31.7.0 MIME-Version: 1.0 To: Alvaro Herrera CC: pgsql-hackers Subject: Re: All-zero page in GIN index causes assertion failure References: <55ACADCD.6020206@iki.fi> <55AD0266.4050300@iki.fi> <20150720161124.GR2301@postgresql.org> In-Reply-To: <20150720161124.GR2301@postgresql.org> Content-Type: multipart/mixed; boundary="------------040707010900070608030606" X-Pg-Spam-Score: -2.4 (--) List-Archive: List-Help: List-ID: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-Mailing-List: pgsql-hackers Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org This is a multi-part message in MIME format. --------------040707010900070608030606 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit On 07/20/2015 07:11 PM, Alvaro Herrera wrote: > Heikki Linnakangas wrote: > >> Looking closer, heap vacuum does a similar thing, but there are two >> mitigating factors that make it safe in practice, I think: >> >> 1. An empty heap page is all-zeroes except for the small page header in the >> beginning of the page. For a torn page to matter, the page would need to be >> torn in the header, but we rely elsewhere (control file) anyway that a >> 512-byte sector update is atomic, so that shouldn't happen. Note that this >> hinges on the fact that there is no special area on heap pages, so you >> cannot rely on this for index pages. >> >> 2. The redo of the first insert/update on a heap page will always >> re-initialize the page, even when full-page-writes are disabled. This is the >> XLOG_HEAP_INIT_PAGE optimization. So it's not just an optimization, it's >> required for correctness. > > Hmm, I guess this is a case of an optimization hiding a bug :-( I mean, > if we didn't have that, we would have noticed this a lot sooner, I > imagine. Yeah, probably. I came up with the attached, for GIN and SP-GiST. Instead of WAL-logging the page initialization in SP-GiST vacuum, I changed it so that it simply leaves the page as all-zeroes, and adds it to the FSM. The code that grabs a target page from the FSM handles that, and initializes the page anyway, so that was simpler. This made the SP-GiST is-deleted flag obsolete, it's no longer set, although the code still checks for it for backwards-compatibility. (even that may actually be unnecessary, as a page that's marked as deleted must also be empty, and wherever we check for the deleted-flag, we also check for PageIsEmpty())) - Heikki --------------040707010900070608030606 Content-Type: application/x-patch; name="fix-gin-spgist-vacuum-1.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="fix-gin-spgist-vacuum-1.patch" ZGlmZiAtLWdpdCBhL3NyYy9iYWNrZW5kL2FjY2Vzcy9naW4vZ2ludmFjdXVt LmMgYi9zcmMvYmFja2VuZC9hY2Nlc3MvZ2luL2dpbnZhY3V1bS5jCmluZGV4 IGViYTU3MmIuLjEzMTU3NjIgMTAwNjQ0Ci0tLSBhL3NyYy9iYWNrZW5kL2Fj Y2Vzcy9naW4vZ2ludmFjdXVtLmMKKysrIGIvc3JjL2JhY2tlbmQvYWNjZXNz L2dpbi9naW52YWN1dW0uYwpAQCAtNzEwLDcgKzcxMCw3IEBAIGdpbnZhY3V1 bWNsZWFudXAoUEdfRlVOQ1RJT05fQVJHUykKIAkJTG9ja0J1ZmZlcihidWZm ZXIsIEdJTl9TSEFSRSk7CiAJCXBhZ2UgPSAoUGFnZSkgQnVmZmVyR2V0UGFn ZShidWZmZXIpOwogCi0JCWlmIChHaW5QYWdlSXNEZWxldGVkKHBhZ2UpKQor CQlpZiAoUGFnZUlzTmV3KHBhZ2UpIHx8IEdpblBhZ2VJc0RlbGV0ZWQocGFn ZSkpCiAJCXsKIAkJCUFzc2VydChibGtubyAhPSBHSU5fUk9PVF9CTEtOTyk7 CiAJCQlSZWNvcmRGcmVlSW5kZXhQYWdlKGluZGV4LCBibGtubyk7CmRpZmYg LS1naXQgYS9zcmMvYmFja2VuZC9hY2Nlc3Mvc3BnaXN0L3NwZ3ZhY3V1bS5j IGIvc3JjL2JhY2tlbmQvYWNjZXNzL3NwZ2lzdC9zcGd2YWN1dW0uYwppbmRl eCBkYzY5ZDFlLi4zNmU5M2FkIDEwMDY0NAotLS0gYS9zcmMvYmFja2VuZC9h Y2Nlc3Mvc3BnaXN0L3NwZ3ZhY3V1bS5jCisrKyBiL3NyYy9iYWNrZW5kL2Fj Y2Vzcy9zcGdpc3Qvc3BndmFjdXVtLmMKQEAgLTYyMSwxNCArNjIxLDEwIEBA IHNwZ3ZhY3V1bXBhZ2Uoc3BnQnVsa0RlbGV0ZVN0YXRlICpiZHMsIEJsb2Nr TnVtYmVyIGJsa25vKQogCXsKIAkJLyoKIAkJICogV2UgZm91bmQgYW4gYWxs LXplcm8gcGFnZSwgd2hpY2ggY291bGQgaGFwcGVuIGlmIHRoZSBkYXRhYmFz ZQotCQkgKiBjcmFzaGVkIGp1c3QgYWZ0ZXIgZXh0ZW5kaW5nIHRoZSBmaWxl LiAgSW5pdGlhbGl6ZSBhbmQgcmVjeWNsZSBpdC4KKwkJICogY3Jhc2hlZCBq dXN0IGFmdGVyIGV4dGVuZGluZyB0aGUgZmlsZS4gIFJlY3ljbGUgaXQuCiAJ CSAqLwotCQlTcEdpc3RJbml0QnVmZmVyKGJ1ZmZlciwgMCk7Ci0JCVNwR2lz dFBhZ2VTZXREZWxldGVkKHBhZ2UpOwotCQkvKiBXZSBkb24ndCBib3RoZXIg dG8gV0FMLWxvZyB0aGlzIGFjdGlvbjsgZWFzeSB0byByZWRvICovCi0JCU1h cmtCdWZmZXJEaXJ0eShidWZmZXIpOwogCX0KLQllbHNlIGlmIChTcEdpc3RQ YWdlSXNEZWxldGVkKHBhZ2UpKQorCWVsc2UgaWYgKFBhZ2VJc0VtcHR5KHBh Z2UpKQogCXsKIAkJLyogbm90aGluZyB0byBkbyAqLwogCX0KQEAgLTY1OSwy NSArNjU1LDE4IEBAIHNwZ3ZhY3V1bXBhZ2Uoc3BnQnVsa0RlbGV0ZVN0YXRl ICpiZHMsIEJsb2NrTnVtYmVyIGJsa25vKQogCSAqLwogCWlmICghU3BHaXN0 QmxvY2tJc1Jvb3QoYmxrbm8pKQogCXsKLQkJLyogSWYgcGFnZSBpcyBub3cg ZW1wdHksIG1hcmsgaXQgZGVsZXRlZCAqLwotCQlpZiAoUGFnZUlzRW1wdHko cGFnZSkgJiYgIVNwR2lzdFBhZ2VJc0RlbGV0ZWQocGFnZSkpCi0JCXsKLQkJ CVNwR2lzdFBhZ2VTZXREZWxldGVkKHBhZ2UpOwotCQkJLyogV2UgZG9uJ3Qg Ym90aGVyIHRvIFdBTC1sb2cgdGhpcyBhY3Rpb247IGVhc3kgdG8gcmVkbyAq LwotCQkJTWFya0J1ZmZlckRpcnR5KGJ1ZmZlcik7Ci0JCX0KLQotCQlpZiAo U3BHaXN0UGFnZUlzRGVsZXRlZChwYWdlKSkKKwkJaWYgKFBhZ2VJc0VtcHR5 KHBhZ2UpKQogCQl7CiAJCQlSZWNvcmRGcmVlSW5kZXhQYWdlKGluZGV4LCBi bGtubyk7CiAJCQliZHMtPnN0YXRzLT5wYWdlc19kZWxldGVkKys7CiAJCX0K IAkJZWxzZQorCQl7CisJCQlTcEdpc3RTZXRMYXN0VXNlZFBhZ2UoaW5kZXgs IGJ1ZmZlcik7CiAJCQliZHMtPmxhc3RGaWxsZWRCbG9jayA9IGJsa25vOwor CQl9CiAJfQogCi0JU3BHaXN0U2V0TGFzdFVzZWRQYWdlKGluZGV4LCBidWZm ZXIpOwotCiAJVW5sb2NrUmVsZWFzZUJ1ZmZlcihidWZmZXIpOwogfQogCmRp ZmYgLS1naXQgYS9zcmMvaW5jbHVkZS9hY2Nlc3Mvc3BnaXN0X3ByaXZhdGUu aCBiL3NyYy9pbmNsdWRlL2FjY2Vzcy9zcGdpc3RfcHJpdmF0ZS5oCmluZGV4 IDQxM2Y3MWUuLjQ4ZGFkZDUgMTAwNjQ0Ci0tLSBhL3NyYy9pbmNsdWRlL2Fj Y2Vzcy9zcGdpc3RfcHJpdmF0ZS5oCisrKyBiL3NyYy9pbmNsdWRlL2FjY2Vz cy9zcGdpc3RfcHJpdmF0ZS5oCkBAIC00OCwxNCArNDgsMTQgQEAgdHlwZWRl ZiBTcEdpc3RQYWdlT3BhcXVlRGF0YSAqU3BHaXN0UGFnZU9wYXF1ZTsKIAog LyogRmxhZyBiaXRzIGluIHBhZ2Ugc3BlY2lhbCBzcGFjZSAqLwogI2RlZmlu ZSBTUEdJU1RfTUVUQQkJCSgxPDwwKQotI2RlZmluZSBTUEdJU1RfREVMRVRF RAkJKDE8PDEpCisjZGVmaW5lIFNQR0lTVF9ERUxFVEVECQkoMTw8MSkJCS8q IG5ldmVyIHNldCwgYnV0IGtlZXAgZm9yIGJhY2t3YXJkcworCQkJCQkJCQkJ CSAqIGNvbXBhdGliaWxpdHkgKi8KICNkZWZpbmUgU1BHSVNUX0xFQUYJCQko MTw8MikKICNkZWZpbmUgU1BHSVNUX05VTExTCQkoMTw8MykKIAogI2RlZmlu ZSBTcEdpc3RQYWdlR2V0T3BhcXVlKHBhZ2UpICgoU3BHaXN0UGFnZU9wYXF1 ZSkgUGFnZUdldFNwZWNpYWxQb2ludGVyKHBhZ2UpKQogI2RlZmluZSBTcEdp c3RQYWdlSXNNZXRhKHBhZ2UpIChTcEdpc3RQYWdlR2V0T3BhcXVlKHBhZ2Up LT5mbGFncyAmIFNQR0lTVF9NRVRBKQogI2RlZmluZSBTcEdpc3RQYWdlSXNE ZWxldGVkKHBhZ2UpIChTcEdpc3RQYWdlR2V0T3BhcXVlKHBhZ2UpLT5mbGFn cyAmIFNQR0lTVF9ERUxFVEVEKQotI2RlZmluZSBTcEdpc3RQYWdlU2V0RGVs ZXRlZChwYWdlKSAoU3BHaXN0UGFnZUdldE9wYXF1ZShwYWdlKS0+ZmxhZ3Mg fD0gU1BHSVNUX0RFTEVURUQpCiAjZGVmaW5lIFNwR2lzdFBhZ2VJc0xlYWYo cGFnZSkgKFNwR2lzdFBhZ2VHZXRPcGFxdWUocGFnZSktPmZsYWdzICYgU1BH SVNUX0xFQUYpCiAjZGVmaW5lIFNwR2lzdFBhZ2VTdG9yZXNOdWxscyhwYWdl KSAoU3BHaXN0UGFnZUdldE9wYXF1ZShwYWdlKS0+ZmxhZ3MgJiBTUEdJU1Rf TlVMTFMpCiAK --------------040707010900070608030606 Content-Type: text/plain Content-Disposition: inline Content-Transfer-Encoding: 8bit MIME-Version: 1.0 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers --------------040707010900070608030606--