Message-ID: <31d8c6d6-717c-45ae-a8f7-9947eb5e3401@enterprisedb.com>
Date: Sun, 7 Jan 2024 11:51:56 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Multidimensional Histograms
Content-Language: en-US
To: Andrei Lepikhov <a.lepikhov@postgrespro.ru>
Cc: pgsql-hackers@postgresql.org, Alexander Cheshev <alex.cheshev@gmail.com>
References: <CAN_hQmsYDhPMK4bSJHPw3RGnpKqPbx2Uk_7HRV+Dz9F5H7dj1A@mail.gmail.com>
 <6e92450c-8136-11d4-8f6e-501c693af5c8@enterprisedb.com>
 <e665a0c8-4a5c-4345-2234-4487138a0d6f@enterprisedb.com>
 <CAN_hQmvOzA0CRjjDzyoOh_LZwHeQQNnZTK=whF4sKh7DRWnQjQ@mail.gmail.com>
 <d091b383-8589-4970-83f4-93169a525df4@enterprisedb.com>
 <62435ae5-15f3-4485-a8e5-4f50d3bc8c8b@postgrespro.ru>
From: Tomas Vondra <tomas.vondra@enterprisedb.com>
In-Reply-To: <62435ae5-15f3-4485-a8e5-4f50d3bc8c8b@postgrespro.ru>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Archived-At: <https://www.postgresql.org/message-id/31d8c6d6-717c-45ae-a8f7-9947eb5e3401%40enterprisedb.com>
Precedence: bulk


On 1/7/24 11:22, Andrei Lepikhov wrote:
> On 7/1/2024 06:54, Tomas Vondra wrote:
>> It's an interesting are for experiments, no doubt about it. And if you
>> choose to explore it, that's fine. But it's better to be aware it may
>> not end with a commit.
>> For the multi-dimensional case, I propose we first try to experiment
>> with the various algorithms, and figure out what works etc. Maybe
>> implementing them in python or something would be easier than C.
> 
> Curiously, trying to utilize extended statistics for some problematic
> cases, I am experimenting with auto-generating such statistics by
> definition of indexes [1]. Doing that, I wanted to add some hand-made
> statistics like a multidimensional histogram or just a histogram which
> could help to perform estimation over a set of columns/expressions.
> I realized that current hooks get_relation_stats_hook and
> get_index_stats_hook are insufficient if I want to perform an estimation
> over a set of ANDed quals on different columns.
> In your opinion, is it possible to add a hook into the extended
> statistics to allow for an extension to propose alternative estimation?
> 
> [1] https://github.com/danolivo/pg_index_stats
> 

No idea, I haven't thought about that very much. Presumably the existing
hooks are insufficient because they're per-attnum? I guess it would make
sense to have a hook for all the attnums of the relation, but I'm not
sure it'd be enough to introduce a new extended statistics kind ...

regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company