pg.ddx.io pgsql-committers@postgresql.org mailing list archive
help / color / mirror / Atom feedpgsql: Monkey-patch LLVM code to fix ARM relocation bug.
7+ messages / 1 participants
[nested] [flat]
* pgsql: Monkey-patch LLVM code to fix ARM relocation bug.
@ 2024-11-06 10:27 Thomas Munro <tmunro@postgresql.org>
0 siblings, 0 replies; 7+ messages in thread
From: Thomas Munro @ 2024-11-06 10:27 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Monkey-patch LLVM code to fix ARM relocation bug.
Supply a new memory manager for RuntimeDyld, to avoid crashes in
generated code caused by memory placement that can overflow a 32 bit
data type. This is a drop-in replacement for the
llvm::SectionMemoryManager class in the LLVM library, with Michael
Smith's proposed fix from
https://www.github.com/llvm/llvm-project/pull/71968.
We hereby slurp it into our own source tree, after moving into a new
namespace llvm::backport and making some minor adjustments so that it
can be compiled with older LLVM versions as far back as 12. It's harder
to make it work on even older LLVM versions, but it doesn't seem likely
that people are really using them so that is not investigated for now.
The problem could also be addressed by switching to JITLink instead of
RuntimeDyld, and that is the LLVM project's recommended solution as
the latter is about to be deprecated. We'll have to do that soon enough
anyway, and then when the LLVM version support window advances far
enough in a few years we'll be able to delete this code. Unfortunately
that wouldn't be enough for PostgreSQL today: in most relevant versions
of LLVM, JITLink is missing or incomplete.
Several other projects have already back-ported this fix into their fork
of LLVM, which is a vote of confidence despite the lack of commit into
LLVM as of today. We don't have our own copy of LLVM so we can't do
exactly what they've done; instead we have a copy of the whole patched
class so we can pass an instance of it to RuntimeDyld.
The LLVM project hasn't chosen to commit the fix yet, and even if it
did, it wouldn't be back-ported into the releases of LLVM that most of
our users care about, so there is not much point in waiting any longer
for that. If they make further changes and commit it to LLVM 19 or 20,
we'll still need this for older versions, but we may want to
resynchronize our copy and update some comments.
The changes that we've had to make to our copy can be seen by diffing
our SectionMemoryManager.{h,cpp} files against the ones in the tree of
the pull request. Per the LLVM project's license requirements, a copy
is in SectionMemoryManager.LICENSE.
This should fix the spate of crash reports we've been receiving lately
from users on large memory ARM systems.
Back-patch to all supported releases.
Co-authored-by: Thomas Munro <thomas.munro@gmail.com>
Co-authored-by: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Reviewed-by: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se> (license aspects)
Reported-by: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Discussion: https://postgr.es/m/CAO6_Xqr63qj%3DSx7HY6ZiiQ6R_JbX%2B-p6sTPwDYwTWZjUmjsYBg%40mail.gmail.com
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/9044fc1d45a0212fd123bd8f364eac058f60fed7
Modified Files
--------------
src/backend/jit/llvm/Makefile | 3 +-
src/backend/jit/llvm/SectionMemoryManager.LICENSE | 279 +++++++++++++++
src/backend/jit/llvm/SectionMemoryManager.cpp | 412 ++++++++++++++++++++++
src/backend/jit/llvm/llvmjit.c | 7 +
src/backend/jit/llvm/llvmjit_wrap.cpp | 20 ++
src/backend/jit/llvm/meson.build | 1 +
src/include/jit/SectionMemoryManager.h | 226 ++++++++++++
src/include/jit/llvmjit.h | 8 +
src/include/jit/llvmjit_backport.h | 25 ++
src/tools/pginclude/headerscheck | 3 +
src/tools/pgindent/exclude_file_patterns | 3 +-
11 files changed, 985 insertions(+), 2 deletions(-)
^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Monkey-patch LLVM code to fix ARM relocation bug.
@ 2024-11-06 10:28 Thomas Munro <tmunro@postgresql.org>
0 siblings, 0 replies; 7+ messages in thread
From: Thomas Munro @ 2024-11-06 10:28 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Monkey-patch LLVM code to fix ARM relocation bug.
Supply a new memory manager for RuntimeDyld, to avoid crashes in
generated code caused by memory placement that can overflow a 32 bit
data type. This is a drop-in replacement for the
llvm::SectionMemoryManager class in the LLVM library, with Michael
Smith's proposed fix from
https://www.github.com/llvm/llvm-project/pull/71968.
We hereby slurp it into our own source tree, after moving into a new
namespace llvm::backport and making some minor adjustments so that it
can be compiled with older LLVM versions as far back as 12. It's harder
to make it work on even older LLVM versions, but it doesn't seem likely
that people are really using them so that is not investigated for now.
The problem could also be addressed by switching to JITLink instead of
RuntimeDyld, and that is the LLVM project's recommended solution as
the latter is about to be deprecated. We'll have to do that soon enough
anyway, and then when the LLVM version support window advances far
enough in a few years we'll be able to delete this code. Unfortunately
that wouldn't be enough for PostgreSQL today: in most relevant versions
of LLVM, JITLink is missing or incomplete.
Several other projects have already back-ported this fix into their fork
of LLVM, which is a vote of confidence despite the lack of commit into
LLVM as of today. We don't have our own copy of LLVM so we can't do
exactly what they've done; instead we have a copy of the whole patched
class so we can pass an instance of it to RuntimeDyld.
The LLVM project hasn't chosen to commit the fix yet, and even if it
did, it wouldn't be back-ported into the releases of LLVM that most of
our users care about, so there is not much point in waiting any longer
for that. If they make further changes and commit it to LLVM 19 or 20,
we'll still need this for older versions, but we may want to
resynchronize our copy and update some comments.
The changes that we've had to make to our copy can be seen by diffing
our SectionMemoryManager.{h,cpp} files against the ones in the tree of
the pull request. Per the LLVM project's license requirements, a copy
is in SectionMemoryManager.LICENSE.
This should fix the spate of crash reports we've been receiving lately
from users on large memory ARM systems.
Back-patch to all supported releases.
Co-authored-by: Thomas Munro <thomas.munro@gmail.com>
Co-authored-by: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Reviewed-by: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se> (license aspects)
Reported-by: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Discussion: https://postgr.es/m/CAO6_Xqr63qj%3DSx7HY6ZiiQ6R_JbX%2B-p6sTPwDYwTWZjUmjsYBg%40mail.gmail.com
Branch
------
REL_17_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/b7467ab71c1982c120bb53811be4341730061212
Modified Files
--------------
src/backend/jit/llvm/Makefile | 3 +-
src/backend/jit/llvm/SectionMemoryManager.LICENSE | 279 +++++++++++++++
src/backend/jit/llvm/SectionMemoryManager.cpp | 412 ++++++++++++++++++++++
src/backend/jit/llvm/llvmjit.c | 7 +
src/backend/jit/llvm/llvmjit_wrap.cpp | 20 ++
src/backend/jit/llvm/meson.build | 1 +
src/include/jit/SectionMemoryManager.h | 226 ++++++++++++
src/include/jit/llvmjit.h | 8 +
src/include/jit/llvmjit_backport.h | 25 ++
src/tools/pginclude/headerscheck | 3 +
src/tools/pgindent/exclude_file_patterns | 3 +-
11 files changed, 985 insertions(+), 2 deletions(-)
^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Monkey-patch LLVM code to fix ARM relocation bug.
@ 2024-11-06 10:28 Thomas Munro <tmunro@postgresql.org>
0 siblings, 0 replies; 7+ messages in thread
From: Thomas Munro @ 2024-11-06 10:28 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Monkey-patch LLVM code to fix ARM relocation bug.
Supply a new memory manager for RuntimeDyld, to avoid crashes in
generated code caused by memory placement that can overflow a 32 bit
data type. This is a drop-in replacement for the
llvm::SectionMemoryManager class in the LLVM library, with Michael
Smith's proposed fix from
https://www.github.com/llvm/llvm-project/pull/71968.
We hereby slurp it into our own source tree, after moving into a new
namespace llvm::backport and making some minor adjustments so that it
can be compiled with older LLVM versions as far back as 12. It's harder
to make it work on even older LLVM versions, but it doesn't seem likely
that people are really using them so that is not investigated for now.
The problem could also be addressed by switching to JITLink instead of
RuntimeDyld, and that is the LLVM project's recommended solution as
the latter is about to be deprecated. We'll have to do that soon enough
anyway, and then when the LLVM version support window advances far
enough in a few years we'll be able to delete this code. Unfortunately
that wouldn't be enough for PostgreSQL today: in most relevant versions
of LLVM, JITLink is missing or incomplete.
Several other projects have already back-ported this fix into their fork
of LLVM, which is a vote of confidence despite the lack of commit into
LLVM as of today. We don't have our own copy of LLVM so we can't do
exactly what they've done; instead we have a copy of the whole patched
class so we can pass an instance of it to RuntimeDyld.
The LLVM project hasn't chosen to commit the fix yet, and even if it
did, it wouldn't be back-ported into the releases of LLVM that most of
our users care about, so there is not much point in waiting any longer
for that. If they make further changes and commit it to LLVM 19 or 20,
we'll still need this for older versions, but we may want to
resynchronize our copy and update some comments.
The changes that we've had to make to our copy can be seen by diffing
our SectionMemoryManager.{h,cpp} files against the ones in the tree of
the pull request. Per the LLVM project's license requirements, a copy
is in SectionMemoryManager.LICENSE.
This should fix the spate of crash reports we've been receiving lately
from users on large memory ARM systems.
Back-patch to all supported releases.
Co-authored-by: Thomas Munro <thomas.munro@gmail.com>
Co-authored-by: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Reviewed-by: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se> (license aspects)
Reported-by: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Discussion: https://postgr.es/m/CAO6_Xqr63qj%3DSx7HY6ZiiQ6R_JbX%2B-p6sTPwDYwTWZjUmjsYBg%40mail.gmail.com
Branch
------
REL_16_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/ee67b73f5902178fba3b825013c4777ffdd0e7af
Modified Files
--------------
src/backend/jit/llvm/Makefile | 3 +-
src/backend/jit/llvm/SectionMemoryManager.LICENSE | 279 +++++++++++++++
src/backend/jit/llvm/SectionMemoryManager.cpp | 412 ++++++++++++++++++++++
src/backend/jit/llvm/llvmjit.c | 7 +
src/backend/jit/llvm/llvmjit_wrap.cpp | 20 ++
src/backend/jit/llvm/meson.build | 1 +
src/include/jit/SectionMemoryManager.h | 226 ++++++++++++
src/include/jit/llvmjit.h | 8 +
src/include/jit/llvmjit_backport.h | 25 ++
src/tools/pginclude/headerscheck | 3 +
src/tools/pgindent/exclude_file_patterns | 3 +-
11 files changed, 985 insertions(+), 2 deletions(-)
^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Monkey-patch LLVM code to fix ARM relocation bug.
@ 2024-11-06 10:28 Thomas Munro <tmunro@postgresql.org>
0 siblings, 0 replies; 7+ messages in thread
From: Thomas Munro @ 2024-11-06 10:28 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Monkey-patch LLVM code to fix ARM relocation bug.
Supply a new memory manager for RuntimeDyld, to avoid crashes in
generated code caused by memory placement that can overflow a 32 bit
data type. This is a drop-in replacement for the
llvm::SectionMemoryManager class in the LLVM library, with Michael
Smith's proposed fix from
https://www.github.com/llvm/llvm-project/pull/71968.
We hereby slurp it into our own source tree, after moving into a new
namespace llvm::backport and making some minor adjustments so that it
can be compiled with older LLVM versions as far back as 12. It's harder
to make it work on even older LLVM versions, but it doesn't seem likely
that people are really using them so that is not investigated for now.
The problem could also be addressed by switching to JITLink instead of
RuntimeDyld, and that is the LLVM project's recommended solution as
the latter is about to be deprecated. We'll have to do that soon enough
anyway, and then when the LLVM version support window advances far
enough in a few years we'll be able to delete this code. Unfortunately
that wouldn't be enough for PostgreSQL today: in most relevant versions
of LLVM, JITLink is missing or incomplete.
Several other projects have already back-ported this fix into their fork
of LLVM, which is a vote of confidence despite the lack of commit into
LLVM as of today. We don't have our own copy of LLVM so we can't do
exactly what they've done; instead we have a copy of the whole patched
class so we can pass an instance of it to RuntimeDyld.
The LLVM project hasn't chosen to commit the fix yet, and even if it
did, it wouldn't be back-ported into the releases of LLVM that most of
our users care about, so there is not much point in waiting any longer
for that. If they make further changes and commit it to LLVM 19 or 20,
we'll still need this for older versions, but we may want to
resynchronize our copy and update some comments.
The changes that we've had to make to our copy can be seen by diffing
our SectionMemoryManager.{h,cpp} files against the ones in the tree of
the pull request. Per the LLVM project's license requirements, a copy
is in SectionMemoryManager.LICENSE.
This should fix the spate of crash reports we've been receiving lately
from users on large memory ARM systems.
Back-patch to all supported releases.
Co-authored-by: Thomas Munro <thomas.munro@gmail.com>
Co-authored-by: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Reviewed-by: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se> (license aspects)
Reported-by: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Discussion: https://postgr.es/m/CAO6_Xqr63qj%3DSx7HY6ZiiQ6R_JbX%2B-p6sTPwDYwTWZjUmjsYBg%40mail.gmail.com
Branch
------
REL_15_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/19bf81c06a372a138215d25e9844e68ddde35313
Modified Files
--------------
src/backend/jit/llvm/Makefile | 3 +-
src/backend/jit/llvm/SectionMemoryManager.LICENSE | 279 +++++++++++++++
src/backend/jit/llvm/SectionMemoryManager.cpp | 412 ++++++++++++++++++++++
src/backend/jit/llvm/llvmjit.c | 8 +-
src/backend/jit/llvm/llvmjit_wrap.cpp | 20 ++
src/include/jit/SectionMemoryManager.h | 226 ++++++++++++
src/include/jit/llvmjit.h | 8 +
src/include/jit/llvmjit_backport.h | 25 ++
src/tools/pginclude/headerscheck | 3 +
src/tools/pgindent/exclude_file_patterns | 3 +-
10 files changed, 984 insertions(+), 3 deletions(-)
^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Monkey-patch LLVM code to fix ARM relocation bug.
@ 2024-11-06 10:28 Thomas Munro <tmunro@postgresql.org>
0 siblings, 0 replies; 7+ messages in thread
From: Thomas Munro @ 2024-11-06 10:28 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Monkey-patch LLVM code to fix ARM relocation bug.
Supply a new memory manager for RuntimeDyld, to avoid crashes in
generated code caused by memory placement that can overflow a 32 bit
data type. This is a drop-in replacement for the
llvm::SectionMemoryManager class in the LLVM library, with Michael
Smith's proposed fix from
https://www.github.com/llvm/llvm-project/pull/71968.
We hereby slurp it into our own source tree, after moving into a new
namespace llvm::backport and making some minor adjustments so that it
can be compiled with older LLVM versions as far back as 12. It's harder
to make it work on even older LLVM versions, but it doesn't seem likely
that people are really using them so that is not investigated for now.
The problem could also be addressed by switching to JITLink instead of
RuntimeDyld, and that is the LLVM project's recommended solution as
the latter is about to be deprecated. We'll have to do that soon enough
anyway, and then when the LLVM version support window advances far
enough in a few years we'll be able to delete this code. Unfortunately
that wouldn't be enough for PostgreSQL today: in most relevant versions
of LLVM, JITLink is missing or incomplete.
Several other projects have already back-ported this fix into their fork
of LLVM, which is a vote of confidence despite the lack of commit into
LLVM as of today. We don't have our own copy of LLVM so we can't do
exactly what they've done; instead we have a copy of the whole patched
class so we can pass an instance of it to RuntimeDyld.
The LLVM project hasn't chosen to commit the fix yet, and even if it
did, it wouldn't be back-ported into the releases of LLVM that most of
our users care about, so there is not much point in waiting any longer
for that. If they make further changes and commit it to LLVM 19 or 20,
we'll still need this for older versions, but we may want to
resynchronize our copy and update some comments.
The changes that we've had to make to our copy can be seen by diffing
our SectionMemoryManager.{h,cpp} files against the ones in the tree of
the pull request. Per the LLVM project's license requirements, a copy
is in SectionMemoryManager.LICENSE.
This should fix the spate of crash reports we've been receiving lately
from users on large memory ARM systems.
Back-patch to all supported releases.
Co-authored-by: Thomas Munro <thomas.munro@gmail.com>
Co-authored-by: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Reviewed-by: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se> (license aspects)
Reported-by: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Discussion: https://postgr.es/m/CAO6_Xqr63qj%3DSx7HY6ZiiQ6R_JbX%2B-p6sTPwDYwTWZjUmjsYBg%40mail.gmail.com
Branch
------
REL_14_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/0b022ddf3e30c292a5246150492af15590e7f477
Modified Files
--------------
src/backend/jit/llvm/Makefile | 3 +-
src/backend/jit/llvm/SectionMemoryManager.LICENSE | 279 +++++++++++++++
src/backend/jit/llvm/SectionMemoryManager.cpp | 412 ++++++++++++++++++++++
src/backend/jit/llvm/llvmjit.c | 8 +-
src/backend/jit/llvm/llvmjit_wrap.cpp | 20 ++
src/include/jit/SectionMemoryManager.h | 226 ++++++++++++
src/include/jit/llvmjit.h | 8 +
src/include/jit/llvmjit_backport.h | 25 ++
src/tools/pginclude/headerscheck | 3 +
src/tools/pgindent/exclude_file_patterns | 3 +-
10 files changed, 984 insertions(+), 3 deletions(-)
^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Monkey-patch LLVM code to fix ARM relocation bug.
@ 2024-11-06 10:28 Thomas Munro <tmunro@postgresql.org>
0 siblings, 0 replies; 7+ messages in thread
From: Thomas Munro @ 2024-11-06 10:28 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Monkey-patch LLVM code to fix ARM relocation bug.
Supply a new memory manager for RuntimeDyld, to avoid crashes in
generated code caused by memory placement that can overflow a 32 bit
data type. This is a drop-in replacement for the
llvm::SectionMemoryManager class in the LLVM library, with Michael
Smith's proposed fix from
https://www.github.com/llvm/llvm-project/pull/71968.
We hereby slurp it into our own source tree, after moving into a new
namespace llvm::backport and making some minor adjustments so that it
can be compiled with older LLVM versions as far back as 12. It's harder
to make it work on even older LLVM versions, but it doesn't seem likely
that people are really using them so that is not investigated for now.
The problem could also be addressed by switching to JITLink instead of
RuntimeDyld, and that is the LLVM project's recommended solution as
the latter is about to be deprecated. We'll have to do that soon enough
anyway, and then when the LLVM version support window advances far
enough in a few years we'll be able to delete this code. Unfortunately
that wouldn't be enough for PostgreSQL today: in most relevant versions
of LLVM, JITLink is missing or incomplete.
Several other projects have already back-ported this fix into their fork
of LLVM, which is a vote of confidence despite the lack of commit into
LLVM as of today. We don't have our own copy of LLVM so we can't do
exactly what they've done; instead we have a copy of the whole patched
class so we can pass an instance of it to RuntimeDyld.
The LLVM project hasn't chosen to commit the fix yet, and even if it
did, it wouldn't be back-ported into the releases of LLVM that most of
our users care about, so there is not much point in waiting any longer
for that. If they make further changes and commit it to LLVM 19 or 20,
we'll still need this for older versions, but we may want to
resynchronize our copy and update some comments.
The changes that we've had to make to our copy can be seen by diffing
our SectionMemoryManager.{h,cpp} files against the ones in the tree of
the pull request. Per the LLVM project's license requirements, a copy
is in SectionMemoryManager.LICENSE.
This should fix the spate of crash reports we've been receiving lately
from users on large memory ARM systems.
Back-patch to all supported releases.
Co-authored-by: Thomas Munro <thomas.munro@gmail.com>
Co-authored-by: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Reviewed-by: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se> (license aspects)
Reported-by: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Discussion: https://postgr.es/m/CAO6_Xqr63qj%3DSx7HY6ZiiQ6R_JbX%2B-p6sTPwDYwTWZjUmjsYBg%40mail.gmail.com
Branch
------
REL_13_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/e88d824a41278af98c208df320051bc1834d3937
Modified Files
--------------
src/backend/jit/llvm/Makefile | 3 +-
src/backend/jit/llvm/SectionMemoryManager.LICENSE | 279 +++++++++++++++
src/backend/jit/llvm/SectionMemoryManager.cpp | 412 ++++++++++++++++++++++
src/backend/jit/llvm/llvmjit.c | 8 +-
src/backend/jit/llvm/llvmjit_wrap.cpp | 20 ++
src/include/jit/SectionMemoryManager.h | 226 ++++++++++++
src/include/jit/llvmjit.h | 8 +
src/include/jit/llvmjit_backport.h | 25 ++
src/tools/pginclude/headerscheck | 3 +
9 files changed, 982 insertions(+), 2 deletions(-)
^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Monkey-patch LLVM code to fix ARM relocation bug.
@ 2024-11-06 10:29 Thomas Munro <tmunro@postgresql.org>
0 siblings, 0 replies; 7+ messages in thread
From: Thomas Munro @ 2024-11-06 10:29 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Monkey-patch LLVM code to fix ARM relocation bug.
Supply a new memory manager for RuntimeDyld, to avoid crashes in
generated code caused by memory placement that can overflow a 32 bit
data type. This is a drop-in replacement for the
llvm::SectionMemoryManager class in the LLVM library, with Michael
Smith's proposed fix from
https://www.github.com/llvm/llvm-project/pull/71968.
We hereby slurp it into our own source tree, after moving into a new
namespace llvm::backport and making some minor adjustments so that it
can be compiled with older LLVM versions as far back as 12. It's harder
to make it work on even older LLVM versions, but it doesn't seem likely
that people are really using them so that is not investigated for now.
The problem could also be addressed by switching to JITLink instead of
RuntimeDyld, and that is the LLVM project's recommended solution as
the latter is about to be deprecated. We'll have to do that soon enough
anyway, and then when the LLVM version support window advances far
enough in a few years we'll be able to delete this code. Unfortunately
that wouldn't be enough for PostgreSQL today: in most relevant versions
of LLVM, JITLink is missing or incomplete.
Several other projects have already back-ported this fix into their fork
of LLVM, which is a vote of confidence despite the lack of commit into
LLVM as of today. We don't have our own copy of LLVM so we can't do
exactly what they've done; instead we have a copy of the whole patched
class so we can pass an instance of it to RuntimeDyld.
The LLVM project hasn't chosen to commit the fix yet, and even if it
did, it wouldn't be back-ported into the releases of LLVM that most of
our users care about, so there is not much point in waiting any longer
for that. If they make further changes and commit it to LLVM 19 or 20,
we'll still need this for older versions, but we may want to
resynchronize our copy and update some comments.
The changes that we've had to make to our copy can be seen by diffing
our SectionMemoryManager.{h,cpp} files against the ones in the tree of
the pull request. Per the LLVM project's license requirements, a copy
is in SectionMemoryManager.LICENSE.
This should fix the spate of crash reports we've been receiving lately
from users on large memory ARM systems.
Back-patch to all supported releases.
Co-authored-by: Thomas Munro <thomas.munro@gmail.com>
Co-authored-by: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Reviewed-by: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se> (license aspects)
Reported-by: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Discussion: https://postgr.es/m/CAO6_Xqr63qj%3DSx7HY6ZiiQ6R_JbX%2B-p6sTPwDYwTWZjUmjsYBg%40mail.gmail.com
Branch
------
REL_12_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/50c1453a308dc535255a0048fce7cfdb0fc5219d
Modified Files
--------------
src/backend/jit/llvm/Makefile | 3 +-
src/backend/jit/llvm/SectionMemoryManager.LICENSE | 279 +++++++++++++++
src/backend/jit/llvm/SectionMemoryManager.cpp | 412 ++++++++++++++++++++++
src/backend/jit/llvm/llvmjit.c | 8 +-
src/backend/jit/llvm/llvmjit_wrap.cpp | 20 ++
src/include/jit/SectionMemoryManager.h | 226 ++++++++++++
src/include/jit/llvmjit.h | 8 +
src/include/jit/llvmjit_backport.h | 25 ++
8 files changed, 979 insertions(+), 2 deletions(-)
^ permalink raw reply [nested|flat] 7+ messages in thread
end of thread, other threads:[~2024-11-06 10:29 UTC | newest]
Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2024-11-06 10:27 pgsql: Monkey-patch LLVM code to fix ARM relocation bug. Thomas Munro <tmunro@postgresql.org>
2024-11-06 10:28 pgsql: Monkey-patch LLVM code to fix ARM relocation bug. Thomas Munro <tmunro@postgresql.org>
2024-11-06 10:28 pgsql: Monkey-patch LLVM code to fix ARM relocation bug. Thomas Munro <tmunro@postgresql.org>
2024-11-06 10:28 pgsql: Monkey-patch LLVM code to fix ARM relocation bug. Thomas Munro <tmunro@postgresql.org>
2024-11-06 10:28 pgsql: Monkey-patch LLVM code to fix ARM relocation bug. Thomas Munro <tmunro@postgresql.org>
2024-11-06 10:28 pgsql: Monkey-patch LLVM code to fix ARM relocation bug. Thomas Munro <tmunro@postgresql.org>
2024-11-06 10:29 pgsql: Monkey-patch LLVM code to fix ARM relocation bug. Thomas Munro <tmunro@postgresql.org>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox