Date: Thu, 7 Jan 2021 20:35:37 -0600
From: Justin Pryzby <pryzby@telsasoft.com>
To: Tomas Vondra <tomas.vondra@enterprisedb.com>
Cc: Dean Rasheed <dean.a.rasheed@gmail.com>, pgsql-hackers@postgresql.org
Subject: Re: PoC/WIP: Extended statistics on expressions
Message-ID: <20210108023537.GA19743@telsasoft.com>
References: <4a2918bb-49b0-ccd5-3af7-f7795941a6ad@enterprisedb.com>
 <CAEZATCUiy-Vtj9RfVrTMCQpfVO_VaCD9Dk4jBj0j_eEFEBS1ow@mail.gmail.com>
 <958870c8-65e0-31b1-4591-b0b10e807dd9@enterprisedb.com>
 <CAEZATCXAPrrOKwEsyZKQ4uzzJQWBCt6QAvOcgqRGdWwT1zb+rQ@mail.gmail.com>
 <329203a4-7cad-a0e3-d5a8-ce70872c8156@enterprisedb.com>
 <CAEZATCWGYooC_Tzt8sQo7760ZEsguj0JGoMLhkANrDwJ9CY20Q@mail.gmail.com>
 <2567ae7e-e1dd-48b5-054d-ed6ff81c0335@enterprisedb.com>
 <CAEZATCW_7nfeHmLtV6TzSfmFG+i=io_Qo0NL8BiBnmEqq3CQnQ@mail.gmail.com>
 <CAEZATCUfMM43sv+RFDqyczQpYMP7SZLd4EexFEEUX4buWRNnrw@mail.gmail.com>
 <afcd65d7-7701-a469-b064-9c62d63ef437@enterprisedb.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <afcd65d7-7701-a469-b064-9c62d63ef437@enterprisedb.com>
User-Agent: Mutt/1.9.4 (2018-02-28)
Precedence: bulk

On Fri, Jan 08, 2021 at 01:57:29AM +0100, Tomas Vondra wrote:
> Attached is a patch fixing most of the issues. There are a couple
> exceptions:

In the docs:

+   &mdash; at the cost that its schema must be extended whenever the structure                                                                                                                                  
+   of statistics <link linkend="catalog-pg-statistic"><structname>pg_statistic</structname></link> changes.                                                                                                     

should say "of statistics *IN* pg_statistics changes" ?

+   to an expression index. The full variant allows defining statistics objects                                                                                                                                  
+   on multiple columns and expressions, and pick which statistics kinds will                                                                                                                                    
+   be built. The per-expression statistics are built automatically when there                                                                                                                                   

"and pick" is wrong - maybe say "and selecting which.."

+   and run a query using an expression on that column.  Without the                                                                                                                                             

remove "the" ?

+   extended statistics, the planner has no information about data                                                                                                                                               
+   distribution for reasults of those expression, and uses default                                                                                                                                              

*results

+   estimates as illustrated by the first query.  The planner also does                                                                                                                                          
+   not realize the value of the second column fully defines the value                                                                                                                                           
+   of the other column, because date truncated to day still identifies                                                                                                                                          
+   the month). Then expression and ndistinct statistics are built on                                                                                                                                            

The ")" is unbalanced

+                               /* all parts of thi expression are covered by this statistics */                                                                                                                 

this

+ * GrouExprInfos, but only if it's not known equal to any of the existing                                                                                                                                       

Group

+        * we don't allow specifying any statistis kinds.  The simple variant                                                                                                                                    

statistics

+        * If no statistic type was specified, build them all (but request                                                                                                                                       

Say "kind" not "type" ?

+ * expression is a simple Var. OTOH we check that there's at least one                                                                                                                                          
+ * statistics matching the expression.                                                                                                                                                                          

one statistic (singular) ?

+                * the future, we might consider                                                                                                                                                                 
+                */                                                                                                                                                                                              

consider ???

+-- (not it fails, when there are no simple column references)                                                                                                                                                   

note?

There's some remaining copy/paste stuff from index expressions:

errmsg("statistics expressions and predicates can refer only to the table being indexed")));
left behind by evaluating the predicate or index expressions.
Set up for predicate or expression evaluation
Need an EState for evaluation of index expressions and
/* Compute and save index expression values */
left behind by evaluating the predicate or index expressions.
Fetch function for analyzing index expressions.
partial-index predicates.  Create it in the per-index context to be
* When analyzing an expression index, believe the expression tree's type                                                                                                                                

-- 
Justin