Message-ID: <78098996-ca5e-a2ce-ae33-68ec8648ccc7@enterprisedb.com>
Date: Thu, 17 Feb 2022 17:29:37 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.5.0
Subject: Re: BUG #17406: Segmentation fault on GiST index after 14.2 upgrade
Content-Language: en-US
From: Tomas Vondra <tomas.vondra@enterprisedb.com>
To: Peter Geoghegan <pg@bowt.ie>
Cc: Victor Yegorov <vyegorov@gmail.com>,
 PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>
References: <17406-71e02820ae79bb40@postgresql.org>
 <cb3a777a-99c7-f671-532d-b828a2223502@enterprisedb.com>
 <CAGnEbojzf-uObyWeu1MPNKaYsnL_U2kg48K6Ec43waPRBu3DrQ@mail.gmail.com>
 <788a0a51-b1e3-3601-5783-3f6488796ec4@enterprisedb.com>
 <CAH2-WzkTvnAfAMLfo5Rs+xn0Jnhf4riRt0Q6gu=_H1NO1XJJxQ@mail.gmail.com>
 <c0c903ee-d53f-9e78-7e3b-1fee2e69922f@enterprisedb.com>
In-Reply-To: <c0c903ee-d53f-9e78-7e3b-1fee2e69922f@enterprisedb.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Archived-At: 
 <https://www.postgresql.org/message-id/78098996-ca5e-a2ce-ae33-68ec8648ccc7%40enterprisedb.com>
Precedence: bulk


On 2/17/22 13:26, Tomas Vondra wrote:
> On 2/17/22 03:52, Peter Geoghegan wrote:
>> On Wed, Feb 16, 2022 at 4:16 PM Tomas Vondra
>> <tomas.vondra@enterprisedb.com> wrote:
>>> test=# SELECT * FROM gist_page_items(get_raw_page('path_gist_idx', 0),
>>> 'path_gist_idx');
>>
>> Maybe this is obvious, but just in case: gist_page_items is *very*
>> trusting here. It's necessary to use gist_page_items_bytea() if you're
>> not 100% sure about the index definition for the index page you
>> provide. The fact that you can display the contents of each tuple
>> using the underlying type's own output function is very handy. But
>> potentially hazardous.
>>
> 
> Sure, but I used this to inspect the index from example in our docs:
> 
>   https://www.postgresql.org/docs/14/ltree.html#id-1.11.7.30.8
> 
> I doubt we can be even more sure about the index definition.
> 
> 
> I don't know what the root cause is yet, but the memory corruption
> happens simply because ltree_out() sees entirely bogus data when
> executed from gist_page_items.
> 
> For example it gets 16B varlena, allocates 16B string for it, and starts
> iterating over the ltree data. The it finds the first element is 60B
> long and happily copies it "into" the 16B buffer. That can't end well,
> obviously.
> 
> Clearly, the ltree values are mangled somehow / somewhere. The funny
> thing is the bytea always looks 8B longer and with bogus data. So for
> example if you select this ltree from the table directly
> 
>   Top.Collections.Pictures.Astronomy.Astronauts
> 
> then ltree_out sees it as 80B varlena, with 5 levels. But if ltree_out
> gets called from gist_page_items, then it sees 88B with 1 level, which
> is completely bogus.
> 
> And funnily enough, if you do this at the beginning of ltree_out:
> 
>   in = (ltree *) ((char *) in + 8);
> 
> then gist_page_items() starts working just fine.
> 
> So, I guess gist_page_items() is broken/confused in some way. I'm not
> sure if this is related to the crash reported by Victor or a separate,
> independent issue.
> 

I may be wrong, but it seems gist_page_items() is entirely oblivious to
the fact that GiST does "compression", and fails to decompress the
values before passing them to out function. So instead of "ltree" it
extracts "ltree_gist" and passes that to "ltree_out".

This is consistent with the 8B offset I mentioned before, because
ltree_gist has 8B header and then stores the original ltree value.

But I may be missing something - this seems like such an obvious gap
that I don't know how this could have worked with anything but the most
trivial cases without GiST compression.

Furthermore, it seems more like a pageinspect issue, unrelated to the
original issue reported by Victor.


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company