Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1iZeqJ-0000XK-Je for pgsql-bugs@arkaria.postgresql.org; Tue, 26 Nov 2019 17:41:39 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1iZeqI-0005zV-DK for pgsql-bugs@arkaria.postgresql.org; Tue, 26 Nov 2019 17:41:38 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1iZeqI-0005zM-4y for pgsql-bugs@lists.postgresql.org; Tue, 26 Nov 2019 17:41:38 +0000 Received: from mail-wm1-x341.google.com ([2a00:1450:4864:20::341]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1iZeqF-0000m7-2M for pgsql-bugs@lists.postgresql.org; Tue, 26 Nov 2019 17:41:37 +0000 Received: by mail-wm1-x341.google.com with SMTP id z19so4343402wmk.3 for ; Tue, 26 Nov 2019 09:41:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=2ndquadrant-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=xRPKMGP5oVCXqydAjlD5imHl3ZEvx5P2onrgLDloclg=; b=dFnDc178TYia7xDgRkhxYAu114gBGZtAe8SWmfqDBUx0iF2bxoSIi7k5keCCp2eJpf 14OXJg9KyxlkdMp4YEF+kliKCAkRacPlUBP4a/kP970DVHcOxVKWtEuhPGalKcBQ2d5X iHExnRJhpuodQN0t5MLYZf8FwJRMTwp+zYJ2L+IvULwy9NEZWgehOmRcl48wHXZPjzNW BmFV3mjQP+bBJ2IYH3rGQbIP4yMqlsjLVaGL+qBknFVuMBIuUEUyjRXW5X4WF/s0Lt+q h8H8FpZ5E4QOjDt+ge8GiCgu62283yQiYVWsmrvTwd+7uIrN73jSpnozxJMcqwLqfx7X Kr7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=xRPKMGP5oVCXqydAjlD5imHl3ZEvx5P2onrgLDloclg=; b=ZoCawTq0DpWVSk5FP3yQ9cMEHhzhmzwEb4nw7ciDq7dtbzT2uGJ0zxlmroFl97ZmuU ay0PC8jDUL6B6amjitsGVbUsaNbHz3GhvEe8b5u2LT0uP8LOCIRc96I86ZziF1CkbyTB KHLTt22xGOj1pP/oV2k8XYiBQkAbw435SipCQzevIibLAeQdMa2Wewxd/kaU1xKeYFEt XqyzyRzGiEuNFZMGBp3YRKLuTy/8yYed6vJ8fhzjahILWj5xDB/WSbYUtf96r1x4BVup RfWaq/AnfWQRcU9Pf77BHis+0Ty/7UvDsLGZdqzvtcCJMaPg+TQc6eZrs7txvPzLl4/F gjdw== X-Gm-Message-State: APjAAAV+5bX1BOL6zojkU5i3aEmN2nBALxlaX1fU/jtJKiF4MPKoQh2H j4iz42tT6jD+2r9jgwFjocbs+w== X-Google-Smtp-Source: APXvYqzj3bHIPMdhd87q42VeMJgFS+yPUt7FgEEpbaiFJFHmKR6h+/BEIJZ7fuDN3so7mP8DxZYN9A== X-Received: by 2002:a1c:ed16:: with SMTP id l22mr163829wmh.151.1574790094077; Tue, 26 Nov 2019 09:41:34 -0800 (PST) Received: from localhost (static-84-42-175-93.net.upcbroadband.cz. [84.42.175.93]) by smtp.gmail.com with ESMTPSA id b1sm15787664wrs.74.2019.11.26.09.41.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Nov 2019 09:41:33 -0800 (PST) Date: Tue, 26 Nov 2019 18:41:31 +0100 From: Tomas Vondra To: Dean Rasheed Cc: Daniel Gustafsson , Dmitry Dolgov <9erthalion6@gmail.com>, Manuel Rigger , PostgreSQL mailing lists Subject: Re: Failed assertion clauses != NIL Message-ID: <20191126174131.p3zv5t2oeqkinx4d@development> References: <20191119133839.5qj5zia4pkd3zldt@localhost> <520C6E76-53A5-4EF2-A855-8054582221A5@yesql.se> <20191119150840.nmezhsf2tpip7osh@development> <20191121145308.zzdcs5uozhtlmcos@development> <20191124234011.iq7wnq2gohdcsroj@development> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk On Tue, Nov 26, 2019 at 10:56:33AM +0000, Dean Rasheed wrote: >On Sun, 24 Nov 2019 at 23:40, Tomas Vondra wrote: >> >> Hi, >> >> Attached are two patched, related to this bug report. >> >> >> 0001 - Fix choose_best_statistics to check clauses individually >> --------------------------------------------------------------- >> >> This modifies the choose_best_statistics function to properly check >> which clauses are actually covered by each statistic object, and only >> use attnums from those. >> >> The patch ended up pretty small, because we already have all the >> necessary info (per-clause attnums) precalculated. Which means this >> should not be much more expensive than before. >> >> The main drawback is that this does change signature of a function >> defined in statistics.h - we have to pass more info (per-clause bitmaps >> and info which clauses are already estimated). Which means ABI break. >> >> I'm not sure how likely it is that external code is calling this >> function, but the probability is non-zero. So maybe even if the patch is >> fairly small, in backbranches we should use the simple fix with just >> returning if the list is NIL. >> > >On a quick read-through that algorithm makes a lot more sense. It >seems pretty unlikely that anyone would be using >choose_best_statistics() anywhere else, so I think maybe it's fine to >change that in back-branches. > >A couple of comments: > >1). I think you should pass estimatedClauses to >choose_best_statistics() as a pure input parameter (i.e., remove one >"*"), since it doesn't (and must not) modify that set. Yeah, good point. >In fact, on closer inspection, I don't think you need to pass it to >choose_best_statistics() at all, since its callers already check >clauses against estimatedClauses. Therefore, in >choose_best_statistics(), incompatible and already-estimated clauses >both appear the same (as NULL/empty attribute sets), and therefore the >estimatedClauses check will never be tripped. > Right, but I'm thinking about the patch that allows applying multiple statistics. With that applied, this changes to a while loop - and we'll either have to rebuild the list_attnums or pass the bitmap. >2). The new parameter "clauses" should probably be called something >like "clause_attnums" or some such, to better reflect what it actually >is. And it should be documented that NULL values represent >incompatible/already-estimated clauses. > Yes, agreed. >3). In statext_mcv_clauselist_selectivity(), the check for at least 2 >attributes is no longer needed, because choose_best_statistics() now >does that, so there's also no need to compute clauses_attnums there. > Good point. I'll replace that with an assert. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services