From: Daniel Gustafsson <dgustafsson@postgresql.org>
To: pgsql-committers@lists.postgresql.org
Subject: pgsql: llvmjit: Use explicit LLVMContextRef for inlining
Date: Wed, 27 Sep 2023 11:08:38 +0000
Message-ID: <E1qlSPG-006GNj-Dj@gemulon.postgresql.org> (raw)
llvmjit: Use explicit LLVMContextRef for inlining
When performing inlining LLVM unfortunately "leaks" types (the
types survive and are usable, but a new round of inlining will
recreate new structurally equivalent types). This accumulation
will over time amount to a memory leak which for some queries
can be large enough to trigger the OOM process killer.
To avoid accumulation of types, all IR related data is stored
in an LLVMContextRef which is dropped and recreated in order
to release all types. Dropping and recreating incurs overhead,
so it will be done only after 100 queries. This is a heuristic
which might be revisited, but until we can get the size of the
context from LLVM we are flying a bit blind.
This issue has been reported several times, there may be more
references to it in the archives on top of the threads linked
below.
Backpatching of this fix will be handled once it has matured
in master for a bit.
Reported-By: Justin Pryzby <pryzby@telsasoft.com>
Reported-By: Kurt Roeckx <kurt@roeckx.be>
Reported-By: Jaime Casanova <jcasanov@systemguards.com.ec>
Reported-By: Lauri Laanmets <pcspets@gmail.com>
Author: Andres Freund and Daniel Gustafsson
Discussion: https://postgr.es/m/7acc8678-df5f-4923-9cf6-e843131ae89d@www.fastmail.com
Discussion: https://postgr.es/m/20201218235607.GC30237@telsasoft.com
Discussion: https://postgr.es/m/CAPH-tTxLf44s3CvUUtQpkDr1D8Hxqc2NGDzGXS1ODsfiJ6WSqA@mail.gmail.com
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/9dce22033d5d2813e9f8e7d595f57ee5a38b3f8e
Modified Files
--------------
src/backend/jit/llvm/llvmjit.c | 139 ++++++++++++++++++++++++++++----
src/backend/jit/llvm/llvmjit_deform.c | 60 +++++++-------
src/backend/jit/llvm/llvmjit_expr.c | 58 +++++++------
src/backend/jit/llvm/llvmjit_inline.cpp | 33 +++++---
src/include/jit/llvmjit.h | 8 ++
src/include/jit/llvmjit_emit.h | 30 ++++---
6 files changed, 234 insertions(+), 94 deletions(-)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-committers@postgresql.org
Cc: dgustafsson@postgresql.org, pgsql-committers@lists.postgresql.org
Subject: Re: pgsql: llvmjit: Use explicit LLVMContextRef for inlining
In-Reply-To: <E1qlSPG-006GNj-Dj@gemulon.postgresql.org>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox