Content-Type: multipart/mixed; boundary="------------TW77kWB0SwYT2VsJJ9OBf4km"
Message-ID: <5cd74597-f555-d4e1-4dc2-d399180bb8cc@enterprisedb.com>
Date: Thu, 24 Feb 2022 19:36:40 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.5.0
Subject: Re: BUG #17406: Segmentation fault on GiST index after 14.2 upgrade
Content-Language: en-US
From: Tomas Vondra <tomas.vondra@enterprisedb.com>
To: Victor Yegorov <vyegorov@gmail.com>
Cc: PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>,
 Alexander Korotkov <aekorotkov@gmail.com>
References: <17406-71e02820ae79bb40@postgresql.org>
 <cb3a777a-99c7-f671-532d-b828a2223502@enterprisedb.com>
 <CAGnEbojzf-uObyWeu1MPNKaYsnL_U2kg48K6Ec43waPRBu3DrQ@mail.gmail.com>
 <788a0a51-b1e3-3601-5783-3f6488796ec4@enterprisedb.com>
 <CAGnEboiQ-cnhXceFa9WfXk7y06fuVVsf9WcmEdPerjqth8QJxA@mail.gmail.com>
 <c733dae0-15ba-bcdd-e465-1a9f2fdefebf@enterprisedb.com>
 <e544c795-2c7a-e662-19d2-e34da12bfa66@enterprisedb.com>
 <14ac4567-3285-f549-67df-debaeab7719f@enterprisedb.com>
In-Reply-To: <14ac4567-3285-f549-67df-debaeab7719f@enterprisedb.com>
Archived-At: 
 <https://www.postgresql.org/message-id/5cd74597-f555-d4e1-4dc2-d399180bb8cc%40enterprisedb.com>
Precedence: bulk

This is a multi-part message in MIME format.
--------------TW77kWB0SwYT2VsJJ9OBf4km
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 2/24/22 16:18, Tomas Vondra wrote:
> On 2/23/22 16:41, Tomas Vondra wrote:
>> On 2/23/22 16:36, Tomas Vondra wrote:
>>>>
>>>> I got approval to send a table with its data and index in the subject
>>>> datafile.
>>>> Taken from the v14 database.
>>>>
>>>
>>> Thanks for the data! I've been able to reproduce the issue (load into
>>> 12, pg_upgrade to 14 and run the query). After bisecting this for a
>>> while, this seems like a bug in PG13 commit
>>>
>>> commit 911e70207703799605f5a0e8aad9f06cff067c63 (HEAD)
>>> Author: Alexander Korotkov <akorotkov@postgresql.org>
>>> Date:   Mon Mar 30 19:17:11 2020 +0300
>>>
>>>     Implement operator class parameters
>>>
>>>     ...
>>>
>>> It works fine when upgrading to an earlier build, and crashes since this
>>> commit. I haven't investigated this further, but I guess there's some
>>> thinko in gist_ltree_ops, affecting how we interpret existing indexes.
>>>
>>> Alexander, any ideas?
>>>
>>
>> Sorry, I accidentally used Alexander's old address, so let me CC him
>> with the correct/current one.
>>
> 
> I tried investigating this a bit further, but my gist-foo is rather
> limited so I haven't made much progress so far. But as Victor already
> mentioned, REINDEX fixes this - so I tried comparing the broken index
> with a new one built on PG13.
> 
> And the it seems the indexes are mostly the same, with the exception of
> LSN in the page header. The only page that substantially differs is the
> first page, which is also storing less data (the broken version):
> 
> test=# SELECT * FROM
> page_header(get_raw_page('v3_region_copy_ltree_path_idx', 0));
>     lsn    | checksum | flags | lower | upper | special | pagesize | ...
> ----------+----------+-------+-------+-------+---------+----------+- ...
>  0/1CD80C8 |        0 |     0 |    40 |  7856 |    8176 |     8192 | ...
>   (1 row)
> 
> test=# SELECT * FROM
> page_header(get_raw_page('v3_region_copy_13_ltree_path_idx', 0));
>     lsn    | checksum | flags | lower | upper | special | pagesize | ...
> -----------+----------+-------+-------+-------+---------+----------+-...
>  0/7029760 |        0 |     0 |    40 |  7760 |    8176 |     8192 | ...
> (1 row)
> 

It took me a bit of digging and debugging, but I think I got it. The
problem is that to determine siglen value, some of the functions use

  LTREE_GET_ASIGLEN

macro, which inspects the opclass options and returns the value. But if
there are no explicit options for the particular index (as happens for
indexes created in older versions), it uses LTREE_ASIGLEN_DEFAULT, which
is 28 bytes. But that's wrong - this value for ltree[] gist opclass, not
for plain ltree - that uses just 8B. So we're off by 20B, which messes
all kinds of stuff, of course.

My guess this is a simple mistake, because the macro is in section
labeled as

  /* GiST support for ltree[] */

and the "A" in the name probably refers to "array" too.

Attached is a patch that fixes this, but I wonder what will happen to
new indexes created on 13+. Will check.

Victor, can you try with this patch applied?


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--------------TW77kWB0SwYT2VsJJ9OBf4km
Content-Type: text/x-patch; charset=UTF-8;
 name="0001-ltree-gist-siglen-fix.patch"
Content-Disposition: attachment; filename="0001-ltree-gist-siglen-fix.patch"
Content-Transfer-Encoding: base64

RnJvbSBjNmM1NDcxYWViZjQ5ZGE5NmU0N2RlYjUyODZhNmY3MDY1MDY4YmQ3IE1vbiBTZXAg
MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBUb21hcyBWb25kcmEgPHRvbWFzLnZvbmRyYUBwb3N0
Z3Jlc3FsLm9yZz4KRGF0ZTogVGh1LCAyNCBGZWIgMjAyMiAxOToyMjozMiArMDEwMApTdWJq
ZWN0OiBbUEFUQ0hdIGx0cmVlIGdpc3Qgc2lnbGVuIGZpeAoKLS0tCiBjb250cmliL2x0cmVl
L19sdHJlZV9naXN0LmMgfCAxMiArKysrKystLS0tLS0KIGNvbnRyaWIvbHRyZWUvbHRyZWUu
aCAgICAgICB8ICA0ICsrLS0KIGNvbnRyaWIvbHRyZWUvbHRyZWVfZ2lzdC5jICB8IDEwICsr
KysrLS0tLS0KIDMgZmlsZXMgY2hhbmdlZCwgMTMgaW5zZXJ0aW9ucygrKSwgMTMgZGVsZXRp
b25zKC0pCgpkaWZmIC0tZ2l0IGEvY29udHJpYi9sdHJlZS9fbHRyZWVfZ2lzdC5jIGIvY29u
dHJpYi9sdHJlZS9fbHRyZWVfZ2lzdC5jCmluZGV4IDcyNTE2YzNiNmI5Li4xZjM2MTQxNmFj
OSAxMDA2NDQKLS0tIGEvY29udHJpYi9sdHJlZS9fbHRyZWVfZ2lzdC5jCisrKyBiL2NvbnRy
aWIvbHRyZWUvX2x0cmVlX2dpc3QuYwpAQCAtNDksNyArNDksNyBAQCBfbHRyZWVfY29tcHJl
c3MoUEdfRlVOQ1RJT05fQVJHUykKIHsKIAlHSVNURU5UUlkgICplbnRyeSA9IChHSVNURU5U
UlkgKikgUEdfR0VUQVJHX1BPSU5URVIoMCk7CiAJR0lTVEVOVFJZICAqcmV0dmFsID0gZW50
cnk7Ci0JaW50CQkJc2lnbGVuID0gTFRSRUVfR0VUX0FTSUdMRU4oKTsKKwlpbnQJCQlzaWds
ZW4gPSBMVFJFRV9HRVRfQVNJR0xFTihMVFJFRV9BU0lHTEVOX0RFRkFVTFQpOwogCiAJaWYg
KGVudHJ5LT5sZWFma2V5KQogCXsJCQkJCQkJLyogbHRyZWUgKi8KQEAgLTEwOCw3ICsxMDgs
NyBAQCBfbHRyZWVfc2FtZShQR19GVU5DVElPTl9BUkdTKQogCWx0cmVlX2dpc3QgKmEgPSAo
bHRyZWVfZ2lzdCAqKSBQR19HRVRBUkdfUE9JTlRFUigwKTsKIAlsdHJlZV9naXN0ICpiID0g
KGx0cmVlX2dpc3QgKikgUEdfR0VUQVJHX1BPSU5URVIoMSk7CiAJYm9vbAkgICAqcmVzdWx0
ID0gKGJvb2wgKikgUEdfR0VUQVJHX1BPSU5URVIoMik7Ci0JaW50CQkJc2lnbGVuID0gTFRS
RUVfR0VUX0FTSUdMRU4oKTsKKwlpbnQJCQlzaWdsZW4gPSBMVFJFRV9HRVRfQVNJR0xFTihM
VFJFRV9BU0lHTEVOX0RFRkFVTFQpOwogCiAJaWYgKExUR19JU0FMTFRSVUUoYSkgJiYgTFRH
X0lTQUxMVFJVRShiKSkKIAkJKnJlc3VsdCA9IHRydWU7CkBAIC0xNTQsNyArMTU0LDcgQEAg
X2x0cmVlX3VuaW9uKFBHX0ZVTkNUSU9OX0FSR1MpCiB7CiAJR2lzdEVudHJ5VmVjdG9yICpl
bnRyeXZlYyA9IChHaXN0RW50cnlWZWN0b3IgKikgUEdfR0VUQVJHX1BPSU5URVIoMCk7CiAJ
aW50CQkgICAqc2l6ZSA9IChpbnQgKikgUEdfR0VUQVJHX1BPSU5URVIoMSk7Ci0JaW50CQkJ
c2lnbGVuID0gTFRSRUVfR0VUX0FTSUdMRU4oKTsKKwlpbnQJCQlzaWdsZW4gPSBMVFJFRV9H
RVRfQVNJR0xFTihMVFJFRV9BU0lHTEVOX0RFRkFVTFQpOwogCWludDMyCQlpOwogCWx0cmVl
X2dpc3QgKnJlc3VsdCA9IGx0cmVlX2dpc3RfYWxsb2MoZmFsc2UsIE5VTEwsIHNpZ2xlbiwg
TlVMTCwgTlVMTCk7CiAJQklUVkVDUAkJYmFzZSA9IExUR19TSUdOKHJlc3VsdCk7CkBAIC0y
MTksNyArMjE5LDcgQEAgX2x0cmVlX3BlbmFsdHkoUEdfRlVOQ1RJT05fQVJHUykKIAlsdHJl
ZV9naXN0ICpvcmlndmFsID0gKGx0cmVlX2dpc3QgKikgRGF0dW1HZXRQb2ludGVyKCgoR0lT
VEVOVFJZICopIFBHX0dFVEFSR19QT0lOVEVSKDApKS0+a2V5KTsKIAlsdHJlZV9naXN0ICpu
ZXd2YWwgPSAobHRyZWVfZ2lzdCAqKSBEYXR1bUdldFBvaW50ZXIoKChHSVNURU5UUlkgKikg
UEdfR0VUQVJHX1BPSU5URVIoMSkpLT5rZXkpOwogCWZsb2F0CSAgICpwZW5hbHR5ID0gKGZs
b2F0ICopIFBHX0dFVEFSR19QT0lOVEVSKDIpOwotCWludAkJCXNpZ2xlbiA9IExUUkVFX0dF
VF9BU0lHTEVOKCk7CisJaW50CQkJc2lnbGVuID0gTFRSRUVfR0VUX0FTSUdMRU4oTFRSRUVf
QVNJR0xFTl9ERUZBVUxUKTsKIAogCSpwZW5hbHR5ID0gaGVtZGlzdChvcmlndmFsLCBuZXd2
YWwsIHNpZ2xlbik7CiAJUEdfUkVUVVJOX1BPSU5URVIocGVuYWx0eSk7CkBAIC0yNDIsNyAr
MjQyLDcgQEAgX2x0cmVlX3BpY2tzcGxpdChQR19GVU5DVElPTl9BUkdTKQogewogCUdpc3RF
bnRyeVZlY3RvciAqZW50cnl2ZWMgPSAoR2lzdEVudHJ5VmVjdG9yICopIFBHX0dFVEFSR19Q
T0lOVEVSKDApOwogCUdJU1RfU1BMSVRWRUMgKnYgPSAoR0lTVF9TUExJVFZFQyAqKSBQR19H
RVRBUkdfUE9JTlRFUigxKTsKLQlpbnQJCQlzaWdsZW4gPSBMVFJFRV9HRVRfQVNJR0xFTigp
OworCWludAkJCXNpZ2xlbiA9IExUUkVFX0dFVF9BU0lHTEVOKExUUkVFX0FTSUdMRU5fREVG
QVVMVCk7CiAJT2Zmc2V0TnVtYmVyIGssCiAJCQkJajsKIAlsdHJlZV9naXN0ICpkYXR1bV9s
LApAQCAtNTA4LDcgKzUwOCw3IEBAIF9sdHJlZV9jb25zaXN0ZW50KFBHX0ZVTkNUSU9OX0FS
R1MpCiAKIAkvKiBPaWQJCXN1YnR5cGUgPSBQR19HRVRBUkdfT0lEKDMpOyAqLwogCWJvb2wJ
ICAgKnJlY2hlY2sgPSAoYm9vbCAqKSBQR19HRVRBUkdfUE9JTlRFUig0KTsKLQlpbnQJCQlz
aWdsZW4gPSBMVFJFRV9HRVRfQVNJR0xFTigpOworCWludAkJCXNpZ2xlbiA9IExUUkVFX0dF
VF9BU0lHTEVOKExUUkVFX0FTSUdMRU5fREVGQVVMVCk7CiAJbHRyZWVfZ2lzdCAqa2V5ID0g
KGx0cmVlX2dpc3QgKikgRGF0dW1HZXRQb2ludGVyKGVudHJ5LT5rZXkpOwogCWJvb2wJCXJl
cyA9IGZhbHNlOwogCmRpZmYgLS1naXQgYS9jb250cmliL2x0cmVlL2x0cmVlLmggYi9jb250
cmliL2x0cmVlL2x0cmVlLmgKaW5kZXggNWI0YmU1ZTY4MGEuLmMyZmYxNWY3MmU2IDEwMDY0
NAotLS0gYS9jb250cmliL2x0cmVlL2x0cmVlLmgKKysrIGIvY29udHJpYi9sdHJlZS9sdHJl
ZS5oCkBAIC0yOTEsOSArMjkxLDkgQEAgZXh0ZXJuIGx0cmVlX2dpc3QgKmx0cmVlX2dpc3Rf
YWxsb2MoYm9vbCBpc2FsbHRydWUsIEJJVFZFQ1Agc2lnbiwgaW50IHNpZ2xlbiwKIAogI2Rl
ZmluZSBMVFJFRV9BU0lHTEVOX0RFRkFVTFQJKDcgKiBzaXplb2YoaW50MzIpKQogI2RlZmlu
ZSBMVFJFRV9BU0lHTEVOX01BWAkJR0lTVE1heEluZGV4S2V5U2l6ZQotI2RlZmluZSBMVFJF
RV9HRVRfQVNJR0xFTigpCQkoUEdfSEFTX09QQ0xBU1NfT1BUSU9OUygpID8gXAorI2RlZmlu
ZSBMVFJFRV9HRVRfQVNJR0xFTihzaWdsZW5fZGVmYXVsdCkJCShQR19IQVNfT1BDTEFTU19P
UFRJT05TKCkgPyBcCiAJCQkJCQkJCSAoKEx0cmVlR2lzdE9wdGlvbnMgKikgUEdfR0VUX09Q
Q0xBU1NfT1BUSU9OUygpKS0+c2lnbGVuIDogXAotCQkJCQkJCQkgTFRSRUVfQVNJR0xFTl9E
RUZBVUxUKQorCQkJCQkJCQkgKHNpZ2xlbl9kZWZhdWx0KSkKICNkZWZpbmUgQVNJR0xFTkJJ
VChzaWdsZW4pCQkoKHNpZ2xlbikgKiBCSVRCWVRFKQogCiAjZGVmaW5lIEFMT09QQllURShz
aWdsZW4pIFwKZGlmZiAtLWdpdCBhL2NvbnRyaWIvbHRyZWUvbHRyZWVfZ2lzdC5jIGIvY29u
dHJpYi9sdHJlZS9sdHJlZV9naXN0LmMKaW5kZXggN2MzOWVkNDI5OGUuLmU5NWMyZjVkNjY2
IDEwMDY0NAotLS0gYS9jb250cmliL2x0cmVlL2x0cmVlX2dpc3QuYworKysgYi9jb250cmli
L2x0cmVlL2x0cmVlX2dpc3QuYwpAQCAtMTMwLDcgKzEzMCw3IEBAIGx0cmVlX3NhbWUoUEdf
RlVOQ1RJT05fQVJHUykKIAlsdHJlZV9naXN0ICphID0gKGx0cmVlX2dpc3QgKikgUEdfR0VU
QVJHX1BPSU5URVIoMCk7CiAJbHRyZWVfZ2lzdCAqYiA9IChsdHJlZV9naXN0ICopIFBHX0dF
VEFSR19QT0lOVEVSKDEpOwogCWJvb2wJICAgKnJlc3VsdCA9IChib29sICopIFBHX0dFVEFS
R19QT0lOVEVSKDIpOwotCWludAkJCXNpZ2xlbiA9IExUUkVFX0dFVF9BU0lHTEVOKCk7CisJ
aW50CQkJc2lnbGVuID0gTFRSRUVfR0VUX0FTSUdMRU4oU0lHTEVOX0RFRkFVTFQpOwogCiAJ
KnJlc3VsdCA9IGZhbHNlOwogCWlmIChMVEdfSVNPTkVOT0RFKGEpICE9IExUR19JU09ORU5P
REUoYikpCkBAIC0xOTAsNyArMTkwLDcgQEAgbHRyZWVfdW5pb24oUEdfRlVOQ1RJT05fQVJH
UykKIHsKIAlHaXN0RW50cnlWZWN0b3IgKmVudHJ5dmVjID0gKEdpc3RFbnRyeVZlY3RvciAq
KSBQR19HRVRBUkdfUE9JTlRFUigwKTsKIAlpbnQJCSAgICpzaXplID0gKGludCAqKSBQR19H
RVRBUkdfUE9JTlRFUigxKTsKLQlpbnQJCQlzaWdsZW4gPSBMVFJFRV9HRVRfQVNJR0xFTigp
OworCWludAkJCXNpZ2xlbiA9IExUUkVFX0dFVF9BU0lHTEVOKFNJR0xFTl9ERUZBVUxUKTsK
IAlCSVRWRUNQCQliYXNlID0gcGFsbG9jMChzaWdsZW4pOwogCWludDMyCQlpLAogCQkJCWo7
CkBAIC0yNjAsNyArMjYwLDcgQEAgbHRyZWVfcGVuYWx0eShQR19GVU5DVElPTl9BUkdTKQog
CWx0cmVlX2dpc3QgKm9yaWd2YWwgPSAobHRyZWVfZ2lzdCAqKSBEYXR1bUdldFBvaW50ZXIo
KChHSVNURU5UUlkgKikgUEdfR0VUQVJHX1BPSU5URVIoMCkpLT5rZXkpOwogCWx0cmVlX2dp
c3QgKm5ld3ZhbCA9IChsdHJlZV9naXN0ICopIERhdHVtR2V0UG9pbnRlcigoKEdJU1RFTlRS
WSAqKSBQR19HRVRBUkdfUE9JTlRFUigxKSktPmtleSk7CiAJZmxvYXQJICAgKnBlbmFsdHkg
PSAoZmxvYXQgKikgUEdfR0VUQVJHX1BPSU5URVIoMik7Ci0JaW50CQkJc2lnbGVuID0gTFRS
RUVfR0VUX0FTSUdMRU4oKTsKKwlpbnQJCQlzaWdsZW4gPSBMVFJFRV9HRVRfQVNJR0xFTihT
SUdMRU5fREVGQVVMVCk7CiAJaW50MzIJCWNtcHIsCiAJCQkJY21wbDsKIApAQCAtMjkyLDcg
KzI5Miw3IEBAIGx0cmVlX3BpY2tzcGxpdChQR19GVU5DVElPTl9BUkdTKQogewogCUdpc3RF
bnRyeVZlY3RvciAqZW50cnl2ZWMgPSAoR2lzdEVudHJ5VmVjdG9yICopIFBHX0dFVEFSR19Q
T0lOVEVSKDApOwogCUdJU1RfU1BMSVRWRUMgKnYgPSAoR0lTVF9TUExJVFZFQyAqKSBQR19H
RVRBUkdfUE9JTlRFUigxKTsKLQlpbnQJCQlzaWdsZW4gPSBMVFJFRV9HRVRfQVNJR0xFTigp
OworCWludAkJCXNpZ2xlbiA9IExUUkVFX0dFVF9BU0lHTEVOKFNJR0xFTl9ERUZBVUxUKTsK
IAlPZmZzZXROdW1iZXIgajsKIAlpbnQzMgkJaTsKIAlSSVgJCSAgICphcnJheTsKQEAgLTYx
OCw3ICs2MTgsNyBAQCBsdHJlZV9jb25zaXN0ZW50KFBHX0ZVTkNUSU9OX0FSR1MpCiAKIAkv
KiBPaWQJCXN1YnR5cGUgPSBQR19HRVRBUkdfT0lEKDMpOyAqLwogCWJvb2wJICAgKnJlY2hl
Y2sgPSAoYm9vbCAqKSBQR19HRVRBUkdfUE9JTlRFUig0KTsKLQlpbnQJCQlzaWdsZW4gPSBM
VFJFRV9HRVRfQVNJR0xFTigpOworCWludAkJCXNpZ2xlbiA9IExUUkVFX0dFVF9BU0lHTEVO
KFNJR0xFTl9ERUZBVUxUKTsKIAlsdHJlZV9naXN0ICprZXkgPSAobHRyZWVfZ2lzdCAqKSBE
YXR1bUdldFBvaW50ZXIoZW50cnktPmtleSk7CiAJdm9pZAkgICAqcXVlcnkgPSBOVUxMOwog
CWJvb2wJCXJlcyA9IGZhbHNlOwotLSAKMi4zNC4xCgo=

--------------TW77kWB0SwYT2VsJJ9OBf4km--