Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1xMgsi-000000029Tu-0Thc for pgsql-bugs@arkaria.postgresql.org; Fri, 30 Oct 2026 07:18:32 +0000 Received: from localhost ([127.0.0.1]:51196 helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xMgsf-00000008qJy-2PF7 for pgsql-bugs@arkaria.postgresql.org; Fri, 30 Oct 2026 07:18:29 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]:55266) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1xMfuO-00000008BdL-1KmY for pgsql-bugs@lists.postgresql.org; Fri, 30 Oct 2026 06:16:12 +0000 Received: from mx3.ctgameinfo.com ([198.50.206.250]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1xMfuK-00000001hzE-3Lq2 for pgsql-bugs@lists.postgresql.org; Fri, 30 Oct 2026 06:16:11 +0000 Received: from [192.168.2.187] (S01068c6a8dfaafeb.vs.shawcable.net [70.71.68.2]) by mx3.ctgameinfo.com (Postfix) with ESMTPSA id 74AE884058; Thu, 29 Oct 2026 23:16:05 -0700 (PDT) Message-ID: <1990fc2d-c56a-4111-8444-6f8f1d66d1fd@ctgameinfo.com> Date: Thu, 29 Oct 2026 23:16:04 -0700 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: crash in ltree_compare_distance() in 14.24 To: David Geier , pgsql-bugs@lists.postgresql.org References: <043a5139-1f0c-49df-b423-1cd129839aa9@ctgameinfo.com> <497fe0b6-2f8c-4237-b749-bed21de36c01@gmail.com> <3527d5c4-0ce4-4412-b433-2e1d36517139@ctgameinfo.com> <23946400-3809-499d-b0bc-5f859df8ef50@gmail.com> Content-Language: en-GB From: Chris St Denis In-Reply-To: <23946400-3809-499d-b0bc-5f859df8ef50@gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On 2026-10-28 12:19 p.m., David Geier wrote: >>> In case you cannot reproduce it, can you share at least the query you >>> were running and the relevant parts of the schema? >> It happened in the middle of the night, has not happened since. >> >> The ltree column id_tree contains a ltree of comment ID's showing the >> tree of comment replies. >> >> Logs show this >> >>      2026-10-26 02:26:35.914 PDT [61206] LOG:  server process (PID >>    50023) was terminated by signal 11: Segmentation fault >>      2026-10-26 02:26:35.914 PDT [61206] DETAIL:  Failed process was >>    running: INSERT INTO "story_comments" ("comment", "reply_to", >>    "deleted", "id_tree", "sid", "user_id", "ipaddr") VALUES ($1, $2, >>    $3, $4, $5, $6, $7) >> >> Due to parameterised queries, I do not have the data values that were >> inserted. >> >> Here is the table structure. > Thanks for sharing the details. > > What can be concluded from the core dump information you shared in your > first mail is that the read of b->numlevel in > > float > ltree_compare_distance(const ltree *a, const ltree *b) > { > ltree_level *al = LTREE_FIRST(a); > ltree_level *bl = LTREE_FIRST(b); > int an = a->numlevel; > int bn = b->numlevel; > > causes the segfault, likely because "b" is a corrupted pointer. The > value of b is 0x127a2f2a13, which is unaligned and far away from "a" > which is 0x2fa39243b1d8 and aligned as expected. Beyond that, the "a" > ltree is empty / has 0 levels as can be inferred from edx = 0. > > I tried to create a failing test case based on your schema details but > failed. I'm assuming that the order of INSERTs and the data matters. > > What you could try is open your core dump and go down the call stack to > see where "b" gets messed up. It would also be interesting which of the > two calls to ltree_compare_distance() in ltree_penalty() causes the > segfault. > > Do you use data checksums? If not, maybe the issue stems from a data > corruption on disk? > > FYI: PostgreSQL 14 reaches end of life on November 12, 2026. Though, I > think it's not unlikely that this bug is also on master. Data corruption is possible, I recently had an incident with it, however my thinking was that if this was caused by corruption that the pg_dump would also crash the server which is not happening. That assumption may be incorrect. If corruption is the cause, can you give guidance on what I should do to repair/validate that data? Which index should be rebuilt/etc? I will be doing a full dump/restore to upgrade to latest version soonish (need to validate that new version won't break anything in the application) which should hopefully remove any issues there. I do not know gdb very well, so I will need guidance for commands to 'go down the call stack tosee where "b" gets messed up'. I recompiled postgres to try to get debugging symbols, and now backtrace is corrupt so I don't know if I can still get useful data out of the core. I guess I can go back to the old binary if the data is useful, but as you said it's an old version. Unfortunately not using checksums.