Message-ID: <af4fd8c6-25e7-09ff-d5ef-076a6b18dd80@enterprisedb.com>
Date: Fri, 14 Jul 2023 16:47:33 +0200
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.12.0
Subject: Re: BRIN indexes vs. SK_SEARCHARRAY (and preprocessing scan keys)
Content-Language: en-US
From: Tomas Vondra <tomas.vondra@enterprisedb.com>
To: Heikki Linnakangas <hlinnaka@iki.fi>,
 PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
References: <f6aab55c-35eb-4118-a7ff-571c62e6cc39@enterprisedb.com>
 <0e1f3350-c9cf-ab62-43a5-5dae314de89c@enterprisedb.com>
 <57020b2e-d9c9-9bc7-4892-b36d9bb07563@enterprisedb.com>
 <327064f1-276e-cb45-88c2-1ff498037cf5@enterprisedb.com>
 <a312b665-79fb-4ee1-e9eb-e8ed9df5479f@iki.fi>
 <4fa33caf-b5d1-6887-65a9-82ce9d92765c@enterprisedb.com>
 <1701e986-fa32-f3fa-0ac8-39fae8e5db35@enterprisedb.com>
 <856edf2c-8e95-0a8f-5a37-c8564a770a82@enterprisedb.com>
 <2484032b-3e6d-ae6a-61e5-732e2136ad18@iki.fi>
 <677c95ad-513f-6702-3c59-0afa2790a149@enterprisedb.com>
 <6a220195-52f8-303c-d154-3f29ac2f078d@iki.fi>
 <4c512f2b-1f08-aa43-4087-78434f94ffeb@enterprisedb.com>
In-Reply-To: <4c512f2b-1f08-aa43-4087-78434f94ffeb@enterprisedb.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Archived-At: <https://www.postgresql.org/message-id/af4fd8c6-25e7-09ff-d5ef-076a6b18dd80%40enterprisedb.com>
Precedence: bulk

On 7/9/23 23:44, Tomas Vondra wrote:
> ...
>>> Yes, my previous message was mostly about backwards compatibility, and
>>> this may seem a bit like an argument against it. But that message was
>>> more a question "If we do this, is it actually backwards compatible the
>>> way we want/need?")
>>>
>>> Anyway, I think the BrinDesc scratch space is a neat idea, I'll try
>>> doing it that way and report back in a couple days.
>>
>> Cool. In 0005-Support-SK_SEARCHARRAY-in-BRIN-bloom-20230702.patch, you
>> used the preprocess function to pre-calculate the scankey's hash, even
>> for scalars. You could use the scratch space in BrinDesc for that,
>> before doing anything with SEARCHARRAYs.
>>
> 
> Yeah, that's a good idea.
> 

I started looking at this (the scratch space in BrinDesc), and it's not
as straightforward. The trouble is BrinDesc is "per attribute" but the
scratch space is "per scankey" (because we'd like to sort values from
the scankey array).

With the "new" consistent functions (that get all scan keys at once)
this probably is not an issue, because we know which scan key we're
processing and so we can map it to the scratch space. But with the old
consistent function that's not the case. Maybe we should support this
only with the "new" consistent function variant?

This would however conflict with the idea to have a separate consistent
function for arrays, which "splits" the scankeys into multiple groups
again. There could be multiple SAOP scan keys, and then what?

I wonder if the scratch space should be in the ScanKey instead?


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company