Received: from malur.postgresql.org ([217.196.149.56])
	by arkaria.postgresql.org with esmtps  (TLS1.3) tls
 TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
	(Exim 4.94.2)
	(envelope-from <pgsql-hackers-owner+archive@lists.postgresql.org>)
	id 1sMX5L-002qT4-Qs
	for pgsql-hackers@arkaria.postgresql.org; Wed, 26 Jun 2024 18:09:35 +0000
Received: from localhost ([127.0.0.1] helo=malur.postgresql.org)
	by malur.postgresql.org with esmtp (Exim 4.94.2)
	(envelope-from <pgsql-hackers-owner+archive@lists.postgresql.org>)
	id 1sMX5I-005pbc-3h
	for pgsql-hackers@arkaria.postgresql.org; Wed, 26 Jun 2024 18:09:32 +0000
Received: from makus.postgresql.org ([2001:4800:3e1:1::229])
	by malur.postgresql.org with esmtps  (TLS1.3) tls
 TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
	(Exim 4.94.2)
	(envelope-from <tgl@sss.pgh.pa.us>)
	id 1sMX5H-005pbT-Q5
	for pgsql-hackers@lists.postgresql.org; Wed, 26 Jun 2024 18:09:32 +0000
Received: from sss.pgh.pa.us ([68.162.161.243])
	by makus.postgresql.org with esmtps  (TLS1.3) tls
 TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
	(Exim 4.94.2)
	(envelope-from <tgl@sss.pgh.pa.us>)
	id 1sMX5F-003GNY-1K
	for pgsql-hackers@lists.postgresql.org; Wed, 26 Jun 2024 18:09:30 +0000
Received: from sss1.sss.pgh.pa.us (localhost [127.0.0.1])
	by sss.pgh.pa.us (8.15.2/8.15.2) with ESMTP id 45QI9St11565655
	for <pgsql-hackers@lists.postgresql.org>; Wed, 26 Jun 2024 14:09:28 -0400
From: Tom Lane <tgl@sss.pgh.pa.us>
To: pgsql-hackers@lists.postgresql.org
Subject: JIT causes core dump during error recovery
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <1565653.1719425368.1@sss.pgh.pa.us>
Content-Transfer-Encoding: quoted-printable
Date: Wed, 26 Jun 2024 14:09:28 -0400
Message-ID: <1565654.1719425368@sss.pgh.pa.us>
List-Id: <pgsql-hackers.lists.postgresql.org>
List-Help: <https://lists.postgresql.org/manage/>
List-Subscribe: <https://lists.postgresql.org/manage/>
List-Post: <mailto:pgsql-hackers@lists.postgresql.org>
List-Owner: <mailto:pgsql-hackers-owner@lists.postgresql.org>
List-Archive: <https://www.postgresql.org/list/pgsql-hackers>
Archived-At: 
 <https://www.postgresql.org/message-id/1565654.1719425368%40sss.pgh.pa.us>
Precedence: bulk

I initially ran into this while trying to reproduce the recent
reports of trouble with LLVM 14 on ARM.  However, it also reproduces
with LLVM 17 on x86_64, and I see no reason to think it's at all
arch-specific.  I also reproduced it in back branches (only tried
v14, but it's definitely not new in HEAD).

To reproduce:

1. Build with --with-llvm

2. Create a config file containing

$ cat $HOME/tmp/temp_config
# enable jit at max
jit_above_cost =3D 1
jit_inline_above_cost =3D 1
jit_optimize_above_cost =3D 1

and do
export TEMP_CONFIG=3D$HOME/tmp/temp_config

3. cd to .../src/pl/plpgsql/src/, and do "make check".

It gets a SIGSEGV in plpgsql_transaction.sql's
cursor_fail_during_commit test.  The stack trace looks like

(gdb) bt
#0  __strlen_evex () at ../sysdeps/x86_64/multiarch/strlen-evex.S:77
#1  0x0000000000735c58 in pq_sendstring (buf=3D0x7ffd80f8eeb0, =

    str=3D0x7f77cffdf000 <error: Cannot access memory at address 0x7f77cff=
df000>)
    at pqformat.c:197
#2  0x00000000009ca09c in err_sendstring (buf=3D0x7ffd80f8eeb0, =

    str=3D0x7f77cffdf000 <error: Cannot access memory at address 0x7f77cff=
df000>)
    at elog.c:3449
#3  0x00000000009ca4ba in send_message_to_frontend (edata=3D0xf786a0 <erro=
rdata>)
    at elog.c:3568
#4  0x00000000009c73a3 in EmitErrorReport () at elog.c:1715
#5  0x00000000008987e7 in PostgresMain (dbname=3D<optimized out>, =

    username=3D0x29fdb00 "postgres") at postgres.c:4378
#6  0x0000000000893c5d in BackendMain (startup_data=3D<optimized out>, =

    startup_data_len=3D<optimized out>) at backend_startup.c:105

The errordata structure it's trying to print out contains

(gdb) p *edata
$1 =3D {elevel =3D 21, output_to_server =3D true, output_to_client =3D tru=
e, =

  hide_stmt =3D false, hide_ctx =3D false, =

  filename =3D 0x7f77cffdf000 <error: Cannot access memory at address 0x7f=
77cffdf000>, lineno =3D 843, =

  funcname =3D 0x7f77cffdf033 <error: Cannot access memory at address 0x7f=
77cffdf033>, domain =3D 0xbd3baa "postgres-17", =

  context_domain =3D 0x7f77c3343320 "plpgsql-17", sqlerrcode =3D 33816706,=
 =

  message =3D 0x29fdc20 "division by zero", detail =3D 0x0, detail_log =3D=
 0x0, =

  hint =3D 0x0, =

  context =3D 0x29fdc50 "PL/pgSQL function cursor_fail_during_commit() lin=
e 6 at COMMIT", backtrace =3D 0x0, =

  message_id =3D 0x7f77cffdf022 <error: Cannot access memory at address 0x=
7f77cffdf022>, schema_name =3D 0x0, table_name =3D 0x0, column_name =3D 0x=
0, =

  datatype_name =3D 0x0, constraint_name =3D 0x0, cursorpos =3D 0, interna=
lpos =3D 0, =

  internalquery =3D 0x0, saved_errno =3D 2, assoc_context =3D 0x29fdb20}

lineno =3D 843 matches the expected error location in int4_div().
The three string fields containing obviously-garbage pointers
are ones that elog.c expects to point at compile-time constants,
so it just records the caller's pointers without strdup'ing them.

Perhaps somebody else will know better, but what I think is happening
here is

A. Thanks to the low jit cost settings, we choose to jit-compile
the "1/(x-1000)" expression inside cursor_fail_during_commit().

B. When x reaches 1000, the division-by-zero error that the test
intends to provoke is thrown from jit-compiled code.

C. Somewhere between there and EmitErrorReport(), something decided
it could unmap the jit-compiled code.

D. Now filename/funcname are pointing into the void, and =

send_message_to_frontend() dumps core while trying to send them.

One way to fix this could be to pstrdup those strings even
though they should be constants.  I don't especially like
the amount of overhead that'd add though.

What I think is the right solution is to fix things so that
seemingly-no-longer-used jit compilations are not thrown away
until transaction cleanup.  I don't know the JIT code nearly
well enough to take point on fixing it like that, though.

			regards, tom lane