Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1iaRWk-0000yr-E4 for pgsql-bugs@arkaria.postgresql.org; Thu, 28 Nov 2019 21:40:43 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1iaRWi-0000Qm-S0 for pgsql-bugs@arkaria.postgresql.org; Thu, 28 Nov 2019 21:40:40 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1iaRWi-0000Ns-DV for pgsql-bugs@lists.postgresql.org; Thu, 28 Nov 2019 21:40:40 +0000 Received: from mail-wm1-x343.google.com ([2a00:1450:4864:20::343]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1iaRWe-00049o-6T for pgsql-bugs@lists.postgresql.org; Thu, 28 Nov 2019 21:40:39 +0000 Received: by mail-wm1-x343.google.com with SMTP id b11so12199637wmj.4 for ; Thu, 28 Nov 2019 13:40:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=2ndquadrant-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=IHYzfjyWqpwjurGeTsBOUVG+4SwIAAZjRAu2+k3cLes=; b=A0smQYckwzhduA0PUromCY3H3B47GUhIJgMfz+hdBcSMSEnfpBsm9ng2XoKD4iPJsl UjQKf7lId88rVqNaGOvC25DJtrbsYIT0GQ1/Fyxiv/P09cjfzauJSZG0MFNy9hnuOMB+ 6sKUdccRJN6GTzRYGSdHUPg4gV83D2qrn/RSRz6CD8cXBlZ5jw9GypQO6SAMQbpRyDdr 0uZJCAUEDZ90hwSIJfOkSdT15OSdkAZNVXwgXs4l9HUKH2KDtQIMqo6IIa0gvGbOeGf/ rJR4GdVWITl9kklcZ+ihXssbko5EPd8XGU4MfbaS8WOlB2XWj79ifINl0FnpPp6OZPoe P8vQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=IHYzfjyWqpwjurGeTsBOUVG+4SwIAAZjRAu2+k3cLes=; b=Ok9kpl+do6l1h+gD9i6jN96ou6CvetgInzmukZHbYAG8ORAbC6vFSm6nkAqJRzYbrD 6Hm4iUAiX9JByqcgUkas27sxX76ZkjbJ6TmkpAV/m7qMw2DWi35hnUzhPXo/Ep2Pl47T ZgH6vpKIy//pJnhmySaKwxEQicugAFycc5SWAp3/YeMw4/A6XBRrFVsxOJbCPoQOei9v caO+ZmUnlv/N1R2YJGsZ26FF2nDIKPWS7PIomQU5pdz90pG8navfklvb5qYECYMHOIgi tqIFZThGKwSLdhZe/rX28cDKd3LOrjFSuc+zCISEUJ41S2Mzy2dbJDGDVbTkzc64OFkA 9TDg== X-Gm-Message-State: APjAAAVvykRxTOtUsvgUeOhyjZj83DoSEFat6s7Hf8LP59tl/RAdVg9D hkygxBWttCdRL97lV7xqH5bHPw== X-Google-Smtp-Source: APXvYqwkOrl7RyhPR2f18kti1gk0aQ+npSLSCfBmJtDU7R0lDv/XVAK8NcsN3/cRnGP6K0PvyTrxsA== X-Received: by 2002:a7b:c19a:: with SMTP id y26mr10202793wmi.152.1574977234625; Thu, 28 Nov 2019 13:40:34 -0800 (PST) Received: from localhost (ip-86-49-253-92.net.upcbroadband.cz. [86.49.253.92]) by smtp.gmail.com with ESMTPSA id k20sm10887787wmj.10.2019.11.28.13.40.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 Nov 2019 13:40:33 -0800 (PST) Date: Thu, 28 Nov 2019 22:40:30 +0100 From: Tomas Vondra To: Dean Rasheed Cc: Daniel Gustafsson , Dmitry Dolgov <9erthalion6@gmail.com>, Manuel Rigger , PostgreSQL mailing lists Subject: Re: Failed assertion clauses != NIL Message-ID: <20191128214030.rhvbp4r2molt3z47@development> References: <20191119133839.5qj5zia4pkd3zldt@localhost> <520C6E76-53A5-4EF2-A855-8054582221A5@yesql.se> <20191119150840.nmezhsf2tpip7osh@development> <20191121145308.zzdcs5uozhtlmcos@development> <20191124234011.iq7wnq2gohdcsroj@development> <20191126174131.p3zv5t2oeqkinx4d@development> <20191127003311.ppjl6nvw4b2nojdi@development> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk On Wed, Nov 27, 2019 at 09:26:11AM +0000, Dean Rasheed wrote: >On Wed, 27 Nov 2019 at 00:33, Tomas Vondra wrote: >> >> >On Tue, Nov 26, 2019 at 10:56:33AM +0000, Dean Rasheed wrote: >> > >> >>In fact, on closer inspection, I don't think you need to pass it to >> >>choose_best_statistics() at all, since its callers already check >> >>clauses against estimatedClauses. Therefore, in >> >>choose_best_statistics(), incompatible and already-estimated clauses >> >>both appear the same (as NULL/empty attribute sets), and therefore the >> >>estimatedClauses check will never be tripped. >> > >> >Right, but I'm thinking about the patch that allows applying multiple >> >statistics. With that applied, this changes to a while loop - and we'll >> >either have to rebuild the list_attnums or pass the bitmap. >> >> On second thought, that's not quite relevant in backbranches, so I've >> removed the parameters for now. I'll add it in the patch that adds >> support for multiple stats. >> > >Or alternatively, that patch could just NULL-out the relevant >list_attnums[] array entry once the corresponding clause has been >estimated, which would avoid needing to change >choose_best_statistics() again. > >> Attached is the 0001 part, addressing (hopefully) all the comments. >> > >I just spotted a trivial comment typo in dependencies_clauselist_selectivity(): > >+ /* >+ * We expect the bitmaps ton contain a single attribute number. >+ */ >+ attnum = bms_singleton_member(list_attnums[listidx]); > >s/ton/to/ > >Also, in statext_mcv_clauselist_selectivity(), clauses_attnums is now >unused, so there's no point in computing it (unless you wanted to add >the Assert() you talked about, but I don't think it's really >necessary). > >Otherwise it looks good to me. > I've pushed this, after adding a regression test for OR clauses that are not fully covered by the MCV statistic. The existing regression queries test almost exclusively AND clauses, so I've considered adding a couple more, but then I reliazed the support for OR clauses is somewhat limited so the patch improving support for OR clauses is a better opportunity. And now that I look at Dean's message again, I realize I forgot to remove the unnecessary clauses_attnum variable, so I'll fix that. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services