Subject: Re: IRe: BUG #16792: silent corruption of GIN index resulting in
 SELECTs returning non-matching rows
To: Pawel Kudzia <kudzia@gmail.com>
Cc: Alexander Korotkov <aekorotkov@gmail.com>,
 pgsql-bugs@lists.postgresql.org, Tom Lane <tgl@sss.pgh.pa.us>
References: <16792-b1913b6b4e098331@postgresql.org>
 <CAJYBUS9AaEgFTXnZnAWir993-MOVLnXefQmyLd9erXewZhXiHA@mail.gmail.com>
 <CAJYBUS_5Z+9aHyNQ7wkTEkZF+c_RMn7hMjPkTGU0Uc7OHMU3sQ@mail.gmail.com>
 <CAJYBUS8p3aYh5ZWJHHk1QGnGkUepMHQMMm87kODoXOFS+VWVGQ@mail.gmail.com>
 <970432.1623938874@sss.pgh.pa.us> <1039976.1623959387@sss.pgh.pa.us>
 <efe09893-8e6d-9a34-8351-3ea3c6ab202a@iki.fi>
 <CAJYBUS9mmVvVPWKM4rNKJqaq85bQG0Rw20LvT05Fc3zOWt2MaQ@mail.gmail.com>
 <CAJYBUS84aSgCNhcusOJQfeWFy1V3U55AOmRUWcpd+SsF9_qp=A@mail.gmail.com>
 <920041ab-e975-afe3-77d0-934a7c50965d@iki.fi>
 <CAJYBUS8S-k7r-VQ3Vv7Asz9KiW1u50VT5k7fq5BSgVXJ_W4KYA@mail.gmail.com>
From: Heikki Linnakangas <hlinnaka@iki.fi>
Message-ID: <06cedb17-6cbe-6586-caa6-2ff618a35517@iki.fi>
Date: Wed, 14 Jul 2021 01:25:13 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: 
 <CAJYBUS8S-k7r-VQ3Vv7Asz9KiW1u50VT5k7fq5BSgVXJ_W4KYA@mail.gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Archived-At: 
 <https://www.postgresql.org/message-id/06cedb17-6cbe-6586-caa6-2ff618a35517%40iki.fi>
Precedence: bulk

On 12/07/2021 20:24, Pawel Kudzia wrote:
> i've run these two commands:
> 
> SET enable_seqscan=off;
> SELECT entity_id FROM entity WHERE ( attribute_name_ids && '{1737}' )
> AND NOT ( (attribute_name_ids||0) && '{1737}') LIMIT 10;
> 
> and got this in the output:
> 
> 2021-07-12 07:41:05 UTC LOG:  GINBUG: startScanKey called: excludeOnly
> 0 nentries 1
> 
> 2021-07-12 07:41:05 UTC STATEMENT:  SELECT entity_id FROM entity WHERE
> ( attribute_name_ids && '{1737}' ) AND NOT ( (attribute_name_ids||0)
> && '{1737}') LIMIT 10;
> 2021-07-12 07:41:05 UTC LOG:  GINBUG: called triConsistentFn(1): 1 0
> 
> 
> the "STATEMENT" line is repeated 79586 times, the "GINBUG: called
> triConsistentFn" - 79585 times.
> 
> there's nothing else in the log besides information about server startup.
> 
> i'll be happy to help with the follow up checks.

Ok, that confirms that it's taking the simple path through 
triConsistentFn(), with a single non-lossy key. Also, there don't seem 
to be any pending inserts. Now that we've ruled those out, I have no 
clue what the problem might be.

Is the data sensitive, or you could you share it? I realize it's a 
pretty large database, but if I had a copy I could poke around to look 
at the index pages involved here, to see if they're corrupt and how. If 
that's possible, you can contact me directly, off-list, to figure the 
logistics to send over a copy of the data directory.

- Heikki