Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cx2nS-0000aD-IA for pgsql-hackers@arkaria.postgresql.org; Sun, 09 Apr 2017 02:41:46 +0000 Received: from localhost ([127.0.0.1] helo=postgresql.org) by malur.postgresql.org with smtp (Exim 4.84_2) (envelope-from ) id 1cx2nS-0002ft-2Q for pgsql-hackers@arkaria.postgresql.org; Sun, 09 Apr 2017 02:41:46 +0000 Received: from makus.postgresql.org ([2001:4800:1501:1::229]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1cx2nQ-0002fY-GN for pgsql-hackers@postgresql.org; Sun, 09 Apr 2017 02:41:44 +0000 Received: from mail-it0-x22b.google.com ([2607:f8b0:4001:c0b::22b]) by makus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.84_2) (envelope-from ) id 1cx2nM-0002KB-Bi for pgsql-hackers@postgresql.org; Sun, 09 Apr 2017 02:41:42 +0000 Received: by mail-it0-x22b.google.com with SMTP id 70so2376903ita.0 for ; Sat, 08 Apr 2017 19:41:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc:message-id:references :to; bh=g4qPJZGIQNemQ9slYP14jbCAVbGZ/1ogP8Txo3oIF6k=; b=PkJ3uU4U7XzIczGCDeQa4QbhqtwNPTVb0O+7gQgERABUOuJ0rJr7wcm/Vi2SJmjQdU Sa8arRHw5GyK1D21jBJ/5kVZXQcMnChw5EIVcM+4bDqi4R4FCdqIsmF11vRa+kSnDMxD Wy+HX46nRVA6yFakGDYBNGGve/3/Dw1OQkQ7ndR9Esre47r20e3Q6JfchcFjVc0LINH2 HetI34uP6O4Vq8NB6n5M+hRtGtuiTRkCwwJL1TQtrWdyV9TOUTxz941EE7lNHQN1iB31 Fv5GjPeQdfaPJ2OS7d7COtXpHqXEUo5lxSRHgpWNDY+QN6r5nhzGh6b/Phh4WqdqiiLJ CYDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to; bh=g4qPJZGIQNemQ9slYP14jbCAVbGZ/1ogP8Txo3oIF6k=; b=bOPuBAOinfruLXWss9GjsI0YwW1qyWFU8pETAwO7xVCqFOH8srv+vQE+0jU3JnvlDU oTWvwv0hyyB5d8rxKyW2E4LZf4347wsfZEjRanYMffluYWHXjTkkOoJLcC048gTTVHmq BtUCw5NyAF7LCDjoQrW+L7x5OgKfJZycCD2vNJin15V+0c/pCH//I05t/U4UjKJy0zY5 uzlEQgUMtumj0TzghL5H9enx1ALIYoIdneDP3UoC881aFjEfHuvIV/p8qy4gckmSVz1P slnyoi0/zqxn7rWxmQrExhMhl71ST2t5hKNh5klqKAwK/BYAtMtMfVDZsnqlGJXrk/pq cxLw== X-Gm-Message-State: AN3rC/7UCJtnyrv2CJUrFu9z6s4N6HHI5qqcdIr4Tk5W52zlbJ9NIBBJ GHW3DEq7DAdXSg== X-Received: by 10.36.115.146 with SMTP id y140mr5936223itb.21.1491705699119; Sat, 08 Apr 2017 19:41:39 -0700 (PDT) Received: from [192.168.2.121] (24-113-198-113.wavecable.com. [24.113.198.113]) by smtp.gmail.com with ESMTPSA id w133sm1755441itf.2.2017.04.08.19.41.37 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sat, 08 Apr 2017 19:41:38 -0700 (PDT) Content-Type: multipart/mixed; boundary="Apple-Mail=_F96CDFA1-57A8-46F2-B678-F64ADE9B8049" Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: [sqlsmith] Planner crash on foreign table join From: Mark Dilger In-Reply-To: <8929.1491705350@sss.pgh.pa.us> Date: Sat, 8 Apr 2017 19:41:36 -0700 Cc: Robert Haas , Andrew Gierth , Andreas Seltenreich , "pgsql-hackers@postgresql.org" , Simon Riggs Message-Id: References: <87inmf5rdx.fsf@credativ.de> <8737djxrup.fsf@news-spur.riddles.org.uk> <11491.1491668863@sss.pgh.pa.us> <26987.1491681445@sss.pgh.pa.us> <2060.1491686428@sss.pgh.pa.us> <1952.1491696836@sss.pgh.pa.us> <42F09AA5-C096-40DD-9153-215C9FE84EA6@gmail.com> <7BC9E277-E52D-4426-B5C6-E3052A02DA74@gmail.com> <7426E0CD-8790-4C7E-8D3C-8289AFD349A2@gmail.com> <8929.1491705350@sss.pgh.pa.us> To: Tom Lane X-Mailer: Apple Mail (2.3124) X-Pg-Spam-Score: -2.0 (--) List-Archive: List-Help: List-ID: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-Mailing-List: pgsql-hackers Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org --Apple-Mail=_F96CDFA1-57A8-46F2-B678-F64ADE9B8049 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii > On Apr 8, 2017, at 7:35 PM, Tom Lane wrote: >=20 > Mark Dilger writes: >> This is very near where the original crash reported in this thread was c= rashing, probably only >> different due to the extra lines of Assert that were added. Am I missin= g some portion of the >> fix that you are testing? I have only applied the patch that Tom includ= ed in the previous email. >=20 > No, that was the whole patch --- but did you do a full "make clean" and > rebuild? The patch changed NodeTag numbers which would affect the whole > tree. I had trouble applying your patch using mark$ patch -p 1 < patch patching file src/backend/nodes/bitmapset.c Hunk #1 FAILED at 115. Hunk #2 FAILED at 146. Hunk #3 FAILED at 190. Hunk #4 FAILED at 205. Hunk #5 FAILED at 229. Hunk #6 FAILED at 265. Hunk #7 FAILED at 298. Hunk #8 FAILED at 324. Hunk #9 FAILED at 364. Hunk #10 FAILED at 444. Hunk #11 FAILED at 463. Hunk #12 FAILED at 488. Hunk #13 FAILED at 517. Hunk #14 FAILED at 554. Hunk #15 FAILED at 598. Hunk #16 FAILED at 635. Hunk #17 FAILED at 665. Hunk #18 FAILED at 694. Hunk #19 FAILED at 732. Hunk #20 FAILED at 770. Hunk #21 FAILED at 789. Hunk #22 FAILED at 825. Hunk #23 FAILED at 853. Hunk #24 FAILED at 878. Hunk #25 FAILED at 927. Hunk #26 FAILED at 981. Hunk #27 FAILED at 1027. 27 out of 27 hunks FAILED -- saving rejects to file src/backend/nodes/bitma= pset.c.rej patching file src/include/nodes/bitmapset.h Hunk #1 FAILED at 20. Hunk #2 FAILED at 38. 2 out of 2 hunks FAILED -- saving rejects to file src/include/nodes/bitmaps= et.h.rej patching file src/include/nodes/nodes.h Hunk #1 FAILED at 291. 1 out of 1 hunk FAILED -- saving rejects to file src/include/nodes/nodes.h.= rej So I did the patching manually against my copy of the current master, aba696d1af9a267eee85d69845c3cdeccf788525, and then ran: make distclean; ./configure --enable-cassert --enable-tap-tests --enable-de= pend && make -j4 && make check-world I am attaching my patch, which I admit on closer inspection differs from yo= urs, but not in any way I can tell is relevant: --Apple-Mail=_F96CDFA1-57A8-46F2-B678-F64ADE9B8049 Content-Disposition: attachment; filename=patch.mark.1 Content-Type: application/octet-stream; name="patch.mark.1" Content-Transfer-Encoding: 7bit diff --git a/src/backend/nodes/bitmapset.c b/src/backend/nodes/bitmapset.c index bf8545d..943dbb7 100644 --- a/src/backend/nodes/bitmapset.c +++ b/src/backend/nodes/bitmapset.c @@ -115,6 +115,7 @@ bms_copy(const Bitmapset *a) if (a == NULL) return NULL; + Assert(IsA(a, Bitmapset)); size = BITMAPSET_SIZE(a->nwords); result = (Bitmapset *) palloc(size); memcpy(result, a, size); @@ -145,6 +146,8 @@ bms_equal(const Bitmapset *a, const Bitmapset *b) } else if (b == NULL) return bms_is_empty(a); + Assert(IsA(a, Bitmapset)); + Assert(IsA(b, Bitmapset)); /* Identify shorter and longer input */ if (a->nwords <= b->nwords) { @@ -187,6 +190,7 @@ bms_make_singleton(int x) wordnum = WORDNUM(x); bitnum = BITNUM(x); result = (Bitmapset *) palloc0(BITMAPSET_SIZE(wordnum + 1)); + result->type = T_Bitmapset; result->nwords = wordnum + 1; result->words[wordnum] = ((bitmapword) 1 << bitnum); return result; @@ -201,7 +205,10 @@ void bms_free(Bitmapset *a) { if (a) + { + Assert(IsA(a, Bitmapset)); pfree(a); + } } @@ -227,6 +234,8 @@ bms_union(const Bitmapset *a, const Bitmapset *b) return bms_copy(b); if (b == NULL) return bms_copy(a); + Assert(IsA(a, Bitmapset)); + Assert(IsA(b, Bitmapset)); /* Identify shorter and longer input; copy the longer one */ if (a->nwords <= b->nwords) { @@ -259,6 +268,8 @@ bms_intersect(const Bitmapset *a, const Bitmapset *b) /* Handle cases where either input is NULL */ if (a == NULL || b == NULL) return NULL; + Assert(IsA(a, Bitmapset)); + Assert(IsA(b, Bitmapset)); /* Identify shorter and longer input; copy the shorter one */ if (a->nwords <= b->nwords) { @@ -292,6 +303,8 @@ bms_difference(const Bitmapset *a, const Bitmapset *b) return NULL; if (b == NULL) return bms_copy(a); + Assert(IsA(a, Bitmapset)); + Assert(IsA(b, Bitmapset)); /* Copy the left input */ result = bms_copy(a); /* And remove b's bits from result */ @@ -316,6 +329,8 @@ bms_is_subset(const Bitmapset *a, const Bitmapset *b) return true; /* empty set is a subset of anything */ if (b == NULL) return bms_is_empty(a); + Assert(IsA(a, Bitmapset)); + Assert(IsA(b, Bitmapset)); /* Check common words */ shortlen = Min(a->nwords, b->nwords); for (i = 0; i < shortlen; i++) @@ -358,6 +373,8 @@ bms_subset_compare(const Bitmapset *a, const Bitmapset *b) } if (b == NULL) return bms_is_empty(a) ? BMS_EQUAL : BMS_SUBSET2; + Assert(IsA(a, Bitmapset)); + Assert(IsA(b, Bitmapset)); /* Check common words */ result = BMS_EQUAL; /* status so far */ shortlen = Min(a->nwords, b->nwords); @@ -427,6 +444,7 @@ bms_is_member(int x, const Bitmapset *a) elog(ERROR, "negative bitmapset member not allowed"); if (a == NULL) return false; + Assert(IsA(a, Bitmapset)); wordnum = WORDNUM(x); bitnum = BITNUM(x); if (wordnum >= a->nwords) @@ -448,6 +466,8 @@ bms_overlap(const Bitmapset *a, const Bitmapset *b) /* Handle cases where either input is NULL */ if (a == NULL || b == NULL) return false; + Assert(IsA(a, Bitmapset)); + Assert(IsA(b, Bitmapset)); /* Check words in common */ shortlen = Min(a->nwords, b->nwords); for (i = 0; i < shortlen; i++) @@ -471,6 +491,7 @@ bms_overlap_list(const Bitmapset *a, const List *b) if (a == NULL || b == NIL) return false; + Assert(IsA(a, Bitmapset)); foreach(lc, b) { int x = lfirst_int(lc); @@ -501,6 +522,8 @@ bms_nonempty_difference(const Bitmapset *a, const Bitmapset *b) return false; if (b == NULL) return !bms_is_empty(a); + Assert(IsA(a, Bitmapset)); + Assert(IsA(b, Bitmapset)); /* Check words in common */ shortlen = Min(a->nwords, b->nwords); for (i = 0; i < shortlen; i++) @@ -531,6 +554,7 @@ bms_singleton_member(const Bitmapset *a) if (a == NULL) elog(ERROR, "bitmapset is empty"); + Assert(IsA(a, Bitmapset)); nwords = a->nwords; for (wordnum = 0; wordnum < nwords; wordnum++) { @@ -574,6 +598,7 @@ bms_get_singleton_member(const Bitmapset *a, int *member) if (a == NULL) return false; + Assert(IsA(a, Bitmapset)); nwords = a->nwords; for (wordnum = 0; wordnum < nwords; wordnum++) { @@ -610,6 +635,7 @@ bms_num_members(const Bitmapset *a) if (a == NULL) return 0; + Assert(IsA(a, Bitmapset)); nwords = a->nwords; for (wordnum = 0; wordnum < nwords; wordnum++) { @@ -639,6 +665,7 @@ bms_membership(const Bitmapset *a) if (a == NULL) return BMS_EMPTY_SET; + Assert(IsA(a, Bitmapset)); nwords = a->nwords; for (wordnum = 0; wordnum < nwords; wordnum++) { @@ -667,6 +694,7 @@ bms_is_empty(const Bitmapset *a) if (a == NULL) return true; + Assert(IsA(a, Bitmapset)); nwords = a->nwords; for (wordnum = 0; wordnum < nwords; wordnum++) { @@ -704,6 +732,7 @@ bms_add_member(Bitmapset *a, int x) elog(ERROR, "negative bitmapset member not allowed"); if (a == NULL) return bms_make_singleton(x); + Assert(IsA(a, Bitmapset)); wordnum = WORDNUM(x); bitnum = BITNUM(x); @@ -741,6 +770,7 @@ bms_del_member(Bitmapset *a, int x) elog(ERROR, "negative bitmapset member not allowed"); if (a == NULL) return NULL; + Assert(IsA(a, Bitmapset)); wordnum = WORDNUM(x); bitnum = BITNUM(x); if (wordnum < a->nwords) @@ -764,6 +794,8 @@ bms_add_members(Bitmapset *a, const Bitmapset *b) return bms_copy(b); if (b == NULL) return a; + Assert(IsA(a, Bitmapset)); + Assert(IsA(b, Bitmapset)); /* Identify shorter and longer input; copy the longer one if needed */ if (a->nwords < b->nwords) { @@ -801,6 +833,8 @@ bms_int_members(Bitmapset *a, const Bitmapset *b) pfree(a); return NULL; } + Assert(IsA(a, Bitmapset)); + Assert(IsA(b, Bitmapset)); /* Intersect b into a; we need never copy */ shortlen = Min(a->nwords, b->nwords); for (i = 0; i < shortlen; i++) @@ -824,6 +858,8 @@ bms_del_members(Bitmapset *a, const Bitmapset *b) return NULL; if (b == NULL) return a; + Assert(IsA(a, Bitmapset)); + Assert(IsA(b, Bitmapset)); /* Remove b's bits from a; we need never copy */ shortlen = Min(a->nwords, b->nwords); for (i = 0; i < shortlen; i++) @@ -847,6 +883,8 @@ bms_join(Bitmapset *a, Bitmapset *b) return b; if (b == NULL) return a; + Assert(IsA(a, Bitmapset)); + Assert(IsA(b, Bitmapset)); /* Identify shorter and longer input; use longer one as result */ if (a->nwords < b->nwords) { @@ -889,6 +927,7 @@ bms_first_member(Bitmapset *a) if (a == NULL) return -1; + Assert(IsA(a, Bitmapset)); nwords = a->nwords; for (wordnum = 0; wordnum < nwords; wordnum++) { @@ -942,6 +981,7 @@ bms_next_member(const Bitmapset *a, int prevbit) if (a == NULL) return -2; + Assert(IsA(a, Bitmapset)); nwords = a->nwords; prevbit++; mask = (~(bitmapword) 0) << BITNUM(prevbit); @@ -987,6 +1027,7 @@ bms_hash_value(const Bitmapset *a) if (a == NULL) return 0; /* All empty sets hash to 0 */ + Assert(IsA(a, Bitmapset)); for (lastword = a->nwords; --lastword >= 0;) { if (a->words[lastword] != 0) diff --git a/src/include/nodes/bitmapset.h b/src/include/nodes/bitmapset.h index 109f7b0..d672ee5 100644 --- a/src/include/nodes/bitmapset.h +++ b/src/include/nodes/bitmapset.h @@ -20,6 +20,8 @@ #ifndef BITMAPSET_H #define BITMAPSET_H +#include "nodes/nodes.h" + /* * Forward decl to save including pg_list.h */ @@ -36,6 +38,7 @@ typedef int32 signedbitmapword; /* must be the matching signed type */ typedef struct Bitmapset { + NodeTag type; int nwords; /* number of words in array */ bitmapword words[FLEXIBLE_ARRAY_MEMBER]; /* really [nwords] */ } Bitmapset; diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h index f59d719..46408c2 100644 --- a/src/include/nodes/nodes.h +++ b/src/include/nodes/nodes.h @@ -291,6 +291,7 @@ typedef enum NodeTag T_List, T_IntList, T_OidList, + T_Bitmapset, /* * TAGS FOR EXTENSIBLE NODES (extensible.h) --Apple-Mail=_F96CDFA1-57A8-46F2-B678-F64ADE9B8049 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii I'm going to pull completely fresh sources and reapply and retest, though y= ou are welcome to review my patch while I do that. mark= --Apple-Mail=_F96CDFA1-57A8-46F2-B678-F64ADE9B8049 Content-Type: text/plain Content-Disposition: inline Content-Transfer-Encoding: 8bit MIME-Version: 1.0 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers --Apple-Mail=_F96CDFA1-57A8-46F2-B678-F64ADE9B8049--