Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1iZlGr-000815-3O for pgsql-bugs@arkaria.postgresql.org; Wed, 27 Nov 2019 00:33:29 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1iZlGo-0000i9-0b for pgsql-bugs@arkaria.postgresql.org; Wed, 27 Nov 2019 00:33:26 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1iZlGn-0000i2-KO for pgsql-bugs@lists.postgresql.org; Wed, 27 Nov 2019 00:33:25 +0000 Received: from mail-wm1-x344.google.com ([2a00:1450:4864:20::344]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1iZlGf-0004UM-4U for pgsql-bugs@lists.postgresql.org; Wed, 27 Nov 2019 00:33:24 +0000 Received: by mail-wm1-x344.google.com with SMTP id u18so5185815wmc.3 for ; Tue, 26 Nov 2019 16:33:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=2ndquadrant-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=kTFtCc2Z1jzzt0lZM6O0/FKtBqrOnb+BugECxAlcTpg=; b=VKKg7Mfxpu2KrR94LeEEF88ksRuLQFG2DNoDf9191wIRwh1knTzWHTY6knUK/DuZkP qJ/d+3dBLm+wmYeWOb4oGDi/BFN3qanhD7DduqCyOz3t+HMZH8hrrBasNWAu55aTEtxi lwe14u2IRWukbUMUbnP6AFouBntgjYKwp5eyWdAup1ZzFjNr/ozykzWfUrLaF6JOYS5V 4NgyZW1USJwFiw4YRfLikwuFslg1HX8wAMjp/I/4jcsXoVRcEaPr7//5KMP2fPlHOeCZ GY97a30n3ESSI5HXm7rtAvFGmyVTneR63nAaUtVugIqcf5UNS09jajePmXjsq3eDmpqS 9IgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=kTFtCc2Z1jzzt0lZM6O0/FKtBqrOnb+BugECxAlcTpg=; b=DhbS49rv5l3ry/nh2oIGe4lknG+ZCLInJVN+WfDYBTkp3JEBcMfPqtOkehS+6zv6vV g255LUNtbz+q2eHkmuqDEDlInoTL2tEt3uVpGsfN0uhcoB5avOUNN/BZjz0bkeVEzQg8 3Av8b5DPdPKNP7wNCPkZX4E6ZgaqDye8XQ6h9/Cf0oQFsq/b2cIjIyeNcEvNvyMtNG/L d1DL0+punFm6SRPHbKunGLopffhSHkDILjGtzomZ73DmQImzlA3P+braWH+ix/eiJwmh pod+3Zjc+axI7XRSdyJ0Lr+LPjg/4pyzuIiKVQ2wcQ/VvASOdMXEWW+FAq8qub8Wbw17 M9/g== X-Gm-Message-State: APjAAAXaY8sGj3E6WqPw/iKvcMGQwpebpw1MrAtpGDUiJJKXJMKpEi20 orz0aBuLpJOdXsnLbBLOrtgtVg== X-Google-Smtp-Source: APXvYqxxruBfjsCwUdv/mLB7/n+2sNvEGlHC3wa49v/bMsO7kIQYEStXo9yoCRr2KC9wGrRKEJV1hw== X-Received: by 2002:a05:600c:1:: with SMTP id g1mr1459110wmc.131.1574814796165; Tue, 26 Nov 2019 16:33:16 -0800 (PST) Received: from localhost (ip-86-49-253-92.net.upcbroadband.cz. [86.49.253.92]) by smtp.gmail.com with ESMTPSA id i127sm5126107wma.35.2019.11.26.16.33.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Nov 2019 16:33:14 -0800 (PST) Date: Wed, 27 Nov 2019 01:33:11 +0100 From: Tomas Vondra To: Dean Rasheed Cc: Daniel Gustafsson , Dmitry Dolgov <9erthalion6@gmail.com>, Manuel Rigger , PostgreSQL mailing lists Subject: Re: Failed assertion clauses != NIL Message-ID: <20191127003311.ppjl6nvw4b2nojdi@development> References: <20191119133839.5qj5zia4pkd3zldt@localhost> <520C6E76-53A5-4EF2-A855-8054582221A5@yesql.se> <20191119150840.nmezhsf2tpip7osh@development> <20191121145308.zzdcs5uozhtlmcos@development> <20191124234011.iq7wnq2gohdcsroj@development> <20191126174131.p3zv5t2oeqkinx4d@development> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="b7xqzbr3x5db4egn" Content-Disposition: inline In-Reply-To: <20191126174131.p3zv5t2oeqkinx4d@development> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk --b7xqzbr3x5db4egn Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline On Tue, Nov 26, 2019 at 06:41:31PM +0100, Tomas Vondra wrote: >On Tue, Nov 26, 2019 at 10:56:33AM +0000, Dean Rasheed wrote: >>On Sun, 24 Nov 2019 at 23:40, Tomas Vondra wrote: >>> >>>Hi, >>> >>>Attached are two patched, related to this bug report. >>> >>> >>>0001 - Fix choose_best_statistics to check clauses individually >>>--------------------------------------------------------------- >>> >>>This modifies the choose_best_statistics function to properly check >>>which clauses are actually covered by each statistic object, and only >>>use attnums from those. >>> >>>The patch ended up pretty small, because we already have all the >>>necessary info (per-clause attnums) precalculated. Which means this >>>should not be much more expensive than before. >>> >>>The main drawback is that this does change signature of a function >>>defined in statistics.h - we have to pass more info (per-clause bitmaps >>>and info which clauses are already estimated). Which means ABI break. >>> >>>I'm not sure how likely it is that external code is calling this >>>function, but the probability is non-zero. So maybe even if the patch is >>>fairly small, in backbranches we should use the simple fix with just >>>returning if the list is NIL. >>> >> >>On a quick read-through that algorithm makes a lot more sense. It >>seems pretty unlikely that anyone would be using >>choose_best_statistics() anywhere else, so I think maybe it's fine to >>change that in back-branches. >> >>A couple of comments: >> >>1). I think you should pass estimatedClauses to >>choose_best_statistics() as a pure input parameter (i.e., remove one >>"*"), since it doesn't (and must not) modify that set. > >Yeah, good point. > >>In fact, on closer inspection, I don't think you need to pass it to >>choose_best_statistics() at all, since its callers already check >>clauses against estimatedClauses. Therefore, in >>choose_best_statistics(), incompatible and already-estimated clauses >>both appear the same (as NULL/empty attribute sets), and therefore the >>estimatedClauses check will never be tripped. >> > >Right, but I'm thinking about the patch that allows applying multiple >statistics. With that applied, this changes to a while loop - and we'll >either have to rebuild the list_attnums or pass the bitmap. > On second thought, that's not quite relevant in backbranches, so I've removed the parameters for now. I'll add it in the patch that adds support for multiple stats. Attached is the 0001 part, addressing (hopefully) all the comments. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services --b7xqzbr3x5db4egn Content-Type: application/gzip Content-Disposition: attachment; filename*0="0001-Fix-choose_best_statistics-to-check-clauses-individu.pat"; filename*1="ch.gz" Content-Transfer-Encoding: base64 H4sICOXD3V0AAzAwMDEtRml4LWNob29zZV9iZXN0X3N0YXRpc3RpY3MtdG8tY2hlY2stY2xh dXNlcy1pbmRpdmlkdS5wYXRjaADNWftv20YS/ln8K6Y4oJEs6v2ynKaI4ziIAdtJLTXJXVEI fCytrUmuyiUtK23+987sLh+S7MSBD7gzEssmd2dn5/HNN+M3iYiAjdjYHbruyO8PpmPPC9jE HbFJd9x1u4eHUycY9Qauw1y4EDHM2Ap6E+h2j9Q/6He7PesNijmCuYgcCR9E7CcO/JTevgyy z583be/zz9ZrJ2VHMMtiG/pDuBS3uK83VVIGR70uNLu9bteaZe4fzEuP4Lf3x/OTt9Dr9H+H N/wOvKUQki1cJtOFTJ2Uy5R7ElKBb5h3A17oZJJJ4LHPb7mfOWG4sayPSxbDins3PL6GdMmA 9gO7S1nsMx8qgoQ6FwKRgAMhPgQR5EJti6fPJMQCt8Yiu17SsaEQN+Ck+C+Ns0iS0MTxUpQa kEXpML1dS0OzONZ6KULWhhMRS+6zRJ3G7pxoFTLcwCX8mbFkA2ueLuHdVX78kWUBwOz0/PRk Dgfw5urdBaTw8e3p1SnU07YDL6DXoPX4i1v9xVO/WJaS51Rv67OAx6gqetMTYRbFEuqO7Tba cMXCDRkL3+zdzCpupi5SqmjDmj3zUZS5Fy2pnJZJx6V7z5doL18waZEp1yK5sckua1zMmLYY Rg5POcpRBgOJqqiDWvlBCQtYwmKPdHRiC3VMuJulTHnHE7f40gd3s62CDSFzfBUDAli0Sje7 LrbwhcsAo4NHDjkxk3nIVC7ixD7+R9UkSzBWHB5mCWtb1px85y2d+JrJb0bqesm9ZRGvTsIA javj1ehvq4NEHG6MGsUtZRlcQYY7ihuLmMk2vBHK9tYqYbdcZHI7uCLmoF13rmTifs3DUJnQ ZYUb0eHoBE9EK1yNDtRxSfsLf+Ddj7N0KZIj64qtRIKWa73aHMGFE2cshCt+fc0S6zWXXiYl uvUIlmm6kkedzkrI9DppM9mJOifHzWzy7njydpR4p/0Xh0F/PFgPL//zesxX7xaf3iX/fnXY G8fZ+Sc3ms3fTle//PIyQuO3r9V31NBqtVoWyMTruI53g8ndKa/Y8dmK0h1jBm3kYSr9Df0x NPOvlv56cHuOFsqZJOBvGHbL7ffL4bEXZj6ryil/bC8BlBowAtpnwQACHrI8hjAAhhNEMooy yoV6s6FA02ch0w9amNU+DwJota45gkvncVd3H7fOQhBldzAdTgdu3x8Npu22xwY9bzqaOlNA mB4Ph2Txx55roXUeffbLl9Cajnp2bwhN84mPqosWOvYogxf4geGLiJ9u6u9DJ45ZchYHAg4S IVLbgtornkbOSrIU4MDk3CIHthdw+ev5+XNcNcvVOb1L9X5SkN5cfHhdORoOqoo8t1q1Y0zN yyxyEfUOlEZG+HOrWTn7YOcd1Hic1mo1esr9O3yAjzoHBiF4QFmWsGeEOBuVrZS3CJ4Rv16q HMV7IACoApJh4h90SGQA9R+WjtSBuhDBAsueX09Y2PqZHtFhNszmx/PZ4vTTfPH69P3p5evT y5Oz01mjgQJqCUuzJIZeu0satWpVpdFa9eplG7BCzBJeXfLPTASVdw04wMvv7q0aY3dv5R1t Rk3yL4XTi5DF1+mybvzXaOT20sFyiEHSw2DRn08JFm1CN5ILLhcRo8vUjYtsOCiKQ64H/Pij 0rU4b0P7SsA0Z9fL2oWewG8cE/xHbRpt97/Q1rUti/1mjv0dTacfUUA9vIZ0jpwbttAVMxVx 3RzwXKm4H/q0w/H9/Jo7C2yobv9C31go2Vf1PItvHbxbGQhfV9nkHok2T5tNOu2L8iqiTN8+ hKb5fIpXv+TZhZzHL1mgzHj6IAfEVcjccmtherVqtBDVvr+872bZrjmV4XREP5SBZK3vOuN+ ObYyujmqanv767lEqBMLwAD3lsQ5NOp4Igt9ApxAZNp4MXG9W8W3loZR+aICQLSvYVw4HNi9 EfkQf+g/MTXRK8zxlvWwMK5JHQrvS+EzvG4O8TrwykCs5Smkom0r3NSDjsYcJNezG77Cylsh PXkPYDgZk+b+Toj6+JsKZRRx24jpKHeTOR4I/nuSpaF0Via8d1NDwyKSs5QjuSLVm1r1plb9 I9LXuxUFsApxDanoHqL4cerwuOTTFdaszm4bGR31qU82EFHgSRUP95VDi+9Ycs68Zcw94rXE 8HUgqQyM0JNUzmIyZ94k6cbrmt+yOJegD2nDDP3BTEbmtDnKpKqD7E9kzhg0pZvoLJU+JgbH PQ0j44k9eVoEKp3izdrZtOEsAORdS2eFwhS3V7wZg4bIOP6qWLYTMdXrYa5S4uQy+HUskry7 QAhWDY8T5pZApv4H3Y7eVlJsP7aqZQdZPtdwox1beVnBnqKg7vmvjL7HSoVqATNpaMoMlXvK SKrmYcATmdZDhTPaI5OJ9shh/8ke0XH/DHuXtROnZChlXNPwrAmtqE8nIxpUhxj5fLFXpomg pi0tldiA42JT1c7XaHs/UECRkT9UQLes/pDZNVB9v/Ct6qzLM1W5x7QEe+3Mg8R8d6VpC/rd CcNuYNxl7fa4P512ez3PGT+qLdiT+NXGYG81hc/hcKhqCn32uxQ++4z3nHp7xeDRUthSJci8 /sw4pia9xoCFg1otZ9h5zWvTY/oPcxpEZEnCMKJMDBKIJkgXEu5Qft7SlEZy1Vq3SsTZ6a2t FgoznKI6tqBVkdDjB/qZlMMQROAo+3wbXCwvemyFaaFEIYIHzi2GMe7TYqVuyQO2RkZ+wzZI YVA1hFwE9G+cXRkp8LgYI9SpyKHymFBZXFQ2JSwvujuafbdWSti8mI4VMc+lGq0kibMhUaZ+ 2aiNBst8HeVxOeSrqtaGY0UqAX0WkfPKgUcJrYyT47dLPJUeXc2VuOLeZUh8+vSJID/KwpST 0/dIoyRj0NDMRVpUxIptCJOa+OiFSpyaFmHAxAZ2JO0kPUWix2eOgn+EIqwvDmlbhJZUSNKG 9yzB6iOVuCBL1KVQBeUblui5UsyYb26B9Gy/x7VaDxDNrQSqNGcF/NyTVM3HyNrbl7NV5G7V BnE7Nmw0QApxSfuo2pDcE4aBircLvfubeKWLyu6i31AYgpygN0EMGY/twYAw5L9oB2KUBVf1 bO00KpJ/0VV1288Vbu/ry+k7Fs/9N2Uh9XTRqM411A0xe8t5hjkHlVyYl9tPKStpenGfGEOK sSOQRIarI0yKSxVYogAvTyBQEufcrJjuAhSLoJu0fiaLwA8vdi20Q2Nb6jCfYcpEHBnhUqwR kuNNFaP0aTLPAQ+hAhlXfvFUmDpdWkI18TGKlKhdvRJISi+8vjajps+YQyciJKTfwcWERUic CX7qFTxsFDR0awhrJKnhM2Z4YTiT+W39XnFsArE6RzW7z4HDT0Vs42/NpuJifxl2nxOaKmB1 yp4jV8RQd9U+bCfPb/z3hmEZhc2bO8JzKVsO3hqm86I7rp5kJiUyczGC6nvnbpn7ASW2PVYO JGTdvLFh/z7KdV90y1GJcSNEPdkWookSvQwSxqpPNSftD7BT7SMp7Y+G9pQQgbyHBGQRebeP 56SgAaPoRQi2bp3kCvfeM+agyO+ovo2Amv6WFDKHiP9abE38MVZU4UFJHJWqZGRbhz35Qg2A 9KWXfLVLGBuUha8uZouLX8/nZ+/PTxuULdVxX+t/PiKpzDMuTj48ZRyC2/8/piB7fPxbfxZw H7PK8PDR4aE7ngwO3T7ycHfSG42nQW+wz8O/Ja3k4N9aqVKld4hdW5M+1FSOSHpCfxMu8qJI ne9s5Up/Yb68W5nCh07ee19lCntDWfSk0ckVIvz+7gBTIb/TXhV+BE3Ybvsexxkw1J9y5MOs al+LXWa1vXCLZqmU+BeyUx6Y15gelGRns/nZyWzxVuV8C6x+u48QYln/AERXj9JKIAAA --b7xqzbr3x5db4egn--