public inbox for [email protected]help / color / mirror / Atom feed
Re: meson vs. llvm bitcode files 10+ messages / 2 participants [nested] [flat]
* Re: meson vs. llvm bitcode files @ 2024-09-05 09:24 Nazir Bilal Yavuz <[email protected]> 0 siblings, 1 reply; 10+ messages in thread From: Nazir Bilal Yavuz @ 2024-09-05 09:24 UTC (permalink / raw) To: Peter Eisentraut <[email protected]>; +Cc: pgsql-hackers Hi, On Thu, 5 Sept 2024 at 11:56, Peter Eisentraut <[email protected]> wrote: > > The meson build currently does not produce llvm bitcode (.bc) files. > AFAIK, this is the last major regression for using meson for production > builds. > > Is anyone working on that? I vaguely recall that some in-progress code > was shared a couple of years ago, but I haven't seen anything since. It > would be great if we could collect any existing code and notes to maybe > get this moving again. I found that Andres shared a patch (v17-0021-meson-Add-LLVM-bitcode-emission.patch) a while ago [1]. [1] https://www.postgresql.org/message-id/20220927011951.j3h4o7n6bhf7dwau%40awork3.anarazel.de -- Regards, Nazir Bilal Yavuz Microsoft ^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: meson vs. llvm bitcode files @ 2025-03-07 10:52 Nazir Bilal Yavuz <[email protected]> parent: Nazir Bilal Yavuz <[email protected]> 0 siblings, 1 reply; 10+ messages in thread From: Nazir Bilal Yavuz @ 2025-03-07 10:52 UTC (permalink / raw) To: Peter Eisentraut <[email protected]>; +Cc: pgsql-hackers; Andres Freund <[email protected]> Hi, On Thu, 5 Sept 2024 at 12:24, Nazir Bilal Yavuz <[email protected]> wrote: > > I found that Andres shared a patch > (v17-0021-meson-Add-LLVM-bitcode-emission.patch) a while ago [1]. Andres and I continued to work on that. I think the patches are in sharable state now and I wanted to hear opinions before proceeding further. After applying the patches, bitcode files should be installed into $pkglibdir/bitcode/ directory if the llvm is found. There are 6 patches attached: v1-0001-meson-Add-generated-header-stamps: This patch is trivial. Instead of having targets depending directly on the generated headers, have them depend on a stamp file. The benefit of using a stamp file is that it makes ninja.build smaller and meson setup faster. ---------- v1-0002-meson-Add-postgresql-extension.pc-for-building-extension-libraries: This patch is for generating postgresql-extension.pc file which can be used for building extensions libraries. Normally, there is no need to use this .pc file for generating bitcode files. However, since there is no clear way to get all include paths for building bitcode files, this .pc file is later used for this purpose (by running pkg-config --cflags-only-I postgresql-extension-uninstalled.pc) [1]. ---------- v1-0003-meson-Test-building-extensions-by-using-postgresql-extension.pc: [Not needed for generating bitcode files] This is a patch for testing if extensions can be built by using postgresql-extension.pc. I added that commit as an example of using postgresql-extension.pc to build extensions. ---------- v1-0004-meson-WIP-Add-docs-for-postgresql-extension.pc: [Not needed for generating bitcode files] I added this patch in case we recommend people to use postgresql-extension.pc to build extension libraries. I am not sure if we want to do that because there are still TODOs about postgresql-extension.pc like running test suites. I just wanted to show my plan, dividing 'Extension Building Infrastructure' into two, 'PGXS' and 'postgresql-extension.pc'. ---------- v1-0005-meson-Add-LLVM-bitcode-emission: This patch adds required infrastructure to generate bitcode files and uses postgresql-extension-uninstalled.pc to get include paths for generating bitcode files [1]. ---------- v1-0006-meson-Generate-bitcode-files-of-contrib-extension.patch: This patch adds manually selected contrib libraries to generate their bitcode files. These libraries are selected manually, depending on - If they have SQL callable functions - If the library functions are short enough (the performance gain from bitcode files is too minimal compared to the function's run time, so this type of libraries are omitted). Any kind of feedback would be appreciated. -- Regards, Nazir Bilal Yavuz Microsoft Attachments: [text/x-patch] v1-0001-meson-Add-generated-header-stamps.patch (5.0K, 2-v1-0001-meson-Add-generated-header-stamps.patch) download | inline diff: From 5f9a9d208bb7e308733b194b75e8b0229797ee4f Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Thu, 24 Oct 2024 07:23:05 -0400 Subject: [PATCH v1 1/6] meson: Add generated header stamps Otherwise build commands become too long and this has visible effect on creation time of meson build files. Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/include/meson.build | 18 ++++++++++++++++++ src/backend/meson.build | 2 +- src/fe_utils/meson.build | 2 +- meson.build | 16 +++++++++------- 4 files changed, 29 insertions(+), 9 deletions(-) diff --git a/src/include/meson.build b/src/include/meson.build index 2e4b7aa529e..c488a5dc4c9 100644 --- a/src/include/meson.build +++ b/src/include/meson.build @@ -177,3 +177,21 @@ install_subdir('catalog', # autoconf generates the file there, ensure we get a conflict generated_sources_ac += {'src/include': ['stamp-h']} + +# Instead of having targets depending directly on the generated headers, have +# them depend on a stamp files for all of them. Dependencies on headers are +# implemented as order-only dependencies in meson (later using compiler +# generated dependencies). The benefit of using a stamp file is that it makes +# ninja.build smaller and meson setup faster. +generated_headers_stamp = custom_target('generated-headers-stamp.h', + output: 'generated-headers-stamp.h', + input: generated_headers, + command: stamp_cmd, +) + +generated_backend_headers_stamp = custom_target('generated-backend-headers-stamp.h', + output: 'generated-backend-headers-stamp.h', + input: generated_backend_headers, + depends: generated_headers_stamp, + command: stamp_cmd, +) diff --git a/src/backend/meson.build b/src/backend/meson.build index 2b0db214804..7fc649c3ebd 100644 --- a/src/backend/meson.build +++ b/src/backend/meson.build @@ -169,7 +169,7 @@ backend_mod_code = declare_dependency( compile_args: pg_mod_c_args, include_directories: postgres_inc, link_args: pg_mod_link_args, - sources: generated_headers + generated_backend_headers, + sources: [generated_backend_headers_stamp], dependencies: backend_mod_deps, ) diff --git a/src/fe_utils/meson.build b/src/fe_utils/meson.build index a18cbc939e4..5a9ddb73463 100644 --- a/src/fe_utils/meson.build +++ b/src/fe_utils/meson.build @@ -29,7 +29,7 @@ generated_sources += psqlscan fe_utils_sources += psqlscan fe_utils = static_library('libpgfeutils', - fe_utils_sources + generated_headers, + fe_utils_sources, c_pch: pch_postgres_fe_h, include_directories: [postgres_inc, libpq_inc], c_args: host_system == 'windows' ? ['-DFD_SETSIZE=1024'] : [], diff --git a/meson.build b/meson.build index 13c13748e5d..9bfa96ad255 100644 --- a/meson.build +++ b/meson.build @@ -2973,6 +2973,8 @@ gen_export_kwargs = { 'install': false, } +# command to create stamp files on all OSs +stamp_cmd = [python, '-c', 'import sys; open(sys.argv[1], "w")', '@OUTPUT0@'] ### @@ -3090,14 +3092,14 @@ subdir('src/port') frontend_common_code = declare_dependency( compile_args: ['-DFRONTEND'], include_directories: [postgres_inc], - sources: generated_headers, + sources: generated_headers_stamp, dependencies: [os_deps, zlib, zstd, lz4], ) backend_common_code = declare_dependency( compile_args: ['-DBUILDING_DLL'], include_directories: [postgres_inc], - sources: generated_headers, + sources: generated_headers_stamp, dependencies: [os_deps, zlib, zstd], ) @@ -3112,7 +3114,7 @@ shlib_code = declare_dependency( frontend_stlib_code = declare_dependency( include_directories: [postgres_inc], link_with: [common_static, pgport_static], - sources: generated_headers, + sources: generated_headers_stamp, dependencies: [os_deps, libintl], ) @@ -3120,7 +3122,7 @@ frontend_stlib_code = declare_dependency( frontend_shlib_code = declare_dependency( include_directories: [postgres_inc], link_with: [common_shlib, pgport_shlib], - sources: generated_headers, + sources: generated_headers_stamp, dependencies: [shlib_code, os_deps, libintl], ) @@ -3130,7 +3132,7 @@ frontend_shlib_code = declare_dependency( frontend_no_fe_utils_code = declare_dependency( include_directories: [postgres_inc], link_with: [common_static, pgport_static], - sources: generated_headers, + sources: generated_headers_stamp, dependencies: [os_deps, libintl], ) @@ -3156,7 +3158,7 @@ subdir('src/fe_utils') frontend_code = declare_dependency( include_directories: [postgres_inc], link_with: [fe_utils, common_static, pgport_static], - sources: generated_headers, + sources: generated_headers_stamp, dependencies: [os_deps, libintl], ) @@ -3184,7 +3186,7 @@ backend_code = declare_dependency( include_directories: [postgres_inc], link_args: ldflags_be, link_with: [], - sources: generated_headers + generated_backend_headers, + sources: [generated_backend_headers_stamp], dependencies: os_deps + backend_both_deps + backend_deps, ) -- 2.47.2 [text/x-patch] v1-0002-meson-Add-postgresql-extension.pc-for-building-ex.patch (4.5K, 3-v1-0002-meson-Add-postgresql-extension.pc-for-building-ex.patch) download | inline diff: From 2a7f61ac7951eaf4e64ec61c8559d4df57364694 Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Sat, 27 Aug 2022 09:52:03 -0700 Subject: [PATCH v1 2/6] meson: Add postgresql-extension.pc for building extension libraries This should work with several other buildsystems. TODO: Docs Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/backend/meson.build | 110 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 110 insertions(+) diff --git a/src/backend/meson.build b/src/backend/meson.build index 7fc649c3ebd..9d79d4d058c 100644 --- a/src/backend/meson.build +++ b/src/backend/meson.build @@ -188,6 +188,116 @@ pg_test_mod_args = pg_mod_args + { +############################################################### +# Define a .pc file that can be used to build server extensions +############################################################### + +pg_ext_vars = [] +pg_ext_vars_inst = [] +pg_ext_vars_uninst = [] + +pg_ext_cflags = pg_mod_c_args + cppflags +pg_ext_libs = [backend_mod_deps, thread_dep, ldflags, ldflags_mod] +pg_ext_subdirs = [''] + +# Compute directories to add include directories to the .pc files for. +# This is a bit more complicated due to port/win32 etc. +i = 0 +foreach incdir : postgres_inc_d + if fs.is_absolute(incdir) + # an absolute path from -Dextra_include_dirs + pg_ext_cflags += '-I@0@'.format(incdir) + continue + elif incdir.startswith('src/include') + subincdir = dir_include_pkg_rel / 'server' / incdir.split('src/include/').get(1, '') + else + subincdir = '' + endif + pg_ext_subdirs += subincdir + + # Add directories in source / build dir containing headers to cflags for the + # -uninstalled.pc. Older versions of pkg-config complain if a referenced + # variable is not defined, so we emit an empty one for the installed .pc + # file. + pg_ext_vars += [ + 'build_inc@0@=""'.format(i), + 'src_inc@0@=""'.format(i), + ] + pg_ext_vars_uninst += [ + 'build_inc@0@=-I${prefix}/@1@'.format(i, incdir), + 'src_inc@0@=-I${srcdir}/@1@'.format(i, incdir), + ] + pg_ext_cflags += [ + '${build_inc@0@}'.format(i), + '${src_inc@0@}'.format(i) + ] + + i += 1 +endforeach + + +# Extension modules should likely also use -fwrapv etc. But it it's a bit odd +# to expose it to a .pc file? +pg_ext_cflags_warn = pg_ext_cflags + cflags_warn +pg_ext_cflags += cflags + +# Directories for extensions to install into +# XXX: more needed +pg_ext_vars += 'pkglibdir=${prefix}/@0@'.format(dir_lib_pkg) +pg_ext_vars += 'dir_mod=${pkglibdir}' +pg_ext_vars += 'dir_data=${prefix}/@0@'.format(dir_data_extension) +pg_ext_vars += 'dir_include=${prefix}/@0@'.format(dir_include_extension) +pg_ext_vars += 'dir_doc=${prefix}/@0@'.format(dir_doc_extension) +pg_ext_vars += 'dir_bitcode=${prefix}/@0@'.format(dir_bitcode) +# referenced on some platforms, via mod_link_with_dir +pg_ext_vars += 'bindir=${prefix}/@0@'.format(dir_bin) + +# XXX: Define variables making it easy to define tests, too + +# Some platforms need linker flags to link with binary, they are the same +# between building with meson and .pc file, except that we have have to +# reference a variable to make it work for both normal and -uninstalled .pc +# files. +if mod_link_args_fmt.length() != 0 + assert(link_with_inst != '') + assert(link_with_uninst != '') + + pg_ext_vars_inst += 'mod_link_with=@0@'.format(link_with_inst) + pg_ext_vars_uninst += 'mod_link_with=@0@'.format(link_with_uninst) + + foreach el : mod_link_args_fmt + pg_ext_libs += el.format('${mod_link_with}') + endforeach +endif + +# main .pc to build extensions +pkgconfig.generate( + name: 'postgresql-extension', + description: 'PostgreSQL Extension Support', + url: pg_url, + + subdirs: pg_ext_subdirs, + libraries: pg_ext_libs, + extra_cflags: pg_ext_cflags, + + variables: pg_ext_vars + pg_ext_vars_inst, + uninstalled_variables: pg_ext_vars + pg_ext_vars_uninst, +) + +# a .pc depending on the above, but with all our warnings enabled +pkgconfig.generate( + name: 'postgresql-extension-warnings', + description: 'PostgreSQL Extension Support - Compiler Warnings', + requires: 'postgresql-extension', + url: pg_url, + extra_cflags: pg_ext_cflags_warn, + + variables: pg_ext_vars + pg_ext_vars_inst, + uninstalled_variables: pg_ext_vars + pg_ext_vars_uninst, +) + + + # Shared modules that, on some system, link against the server binary. Only # enter these after we defined the server build. -- 2.47.2 [text/x-patch] v1-0003-meson-Test-building-extensions-by-using-postgresq.patch (10.7K, 4-v1-0003-meson-Test-building-extensions-by-using-postgresq.patch) download | inline diff: From 5a7e25dd04b6825da5d50c91ccc63526d0fa4d8c Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Thu, 27 Feb 2025 17:45:31 +0300 Subject: [PATCH v1 3/6] meson: Test building extensions by using postgresql-extension.pc The 'test_meson_extensions' pyton wrapper is added to run these tests. It compiles and builds extensions at ${build}/testrun/meson_extensions/${extension_name} path. The tests for building amcheck, auth_delay and postgres_fdw extensions are added. These are also examples of how to build extensions by using postgresql-extension.pc. Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/test/modules/meson.build | 1 + .../modules/test_meson_extensions/meson.build | 3 + .../amcheck/meson.build | 28 ++++++++ .../auth_delay/meson.build | 17 +++++ .../test_pkg_config_extensions/meson.build | 24 +++++++ .../postgres_fdw/meson.build | 31 ++++++++ meson.build | 32 +++++++++ src/tools/ci/test_meson_extensions | 70 +++++++++++++++++++ 8 files changed, 206 insertions(+) create mode 100644 src/test/modules/test_meson_extensions/meson.build create mode 100644 src/test/modules/test_meson_extensions/test_pkg_config_extensions/amcheck/meson.build create mode 100644 src/test/modules/test_meson_extensions/test_pkg_config_extensions/auth_delay/meson.build create mode 100644 src/test/modules/test_meson_extensions/test_pkg_config_extensions/meson.build create mode 100644 src/test/modules/test_meson_extensions/test_pkg_config_extensions/postgres_fdw/meson.build create mode 100644 src/tools/ci/test_meson_extensions diff --git a/src/test/modules/meson.build b/src/test/modules/meson.build index 2b057451473..22ec5e969ae 100644 --- a/src/test/modules/meson.build +++ b/src/test/modules/meson.build @@ -25,6 +25,7 @@ subdir('test_ginpostinglist') subdir('test_integerset') subdir('test_json_parser') subdir('test_lfind') +subdir('test_meson_extensions') subdir('test_misc') subdir('test_oat_hooks') subdir('test_parser') diff --git a/src/test/modules/test_meson_extensions/meson.build b/src/test/modules/test_meson_extensions/meson.build new file mode 100644 index 00000000000..06cf5d555df --- /dev/null +++ b/src/test/modules/test_meson_extensions/meson.build @@ -0,0 +1,3 @@ +# Copyright (c) 2022-2025, PostgreSQL Global Development Group + +subdir('test_pkg_config_extensions') diff --git a/src/test/modules/test_meson_extensions/test_pkg_config_extensions/amcheck/meson.build b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/amcheck/meson.build new file mode 100644 index 00000000000..482a543eb86 --- /dev/null +++ b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/amcheck/meson.build @@ -0,0 +1,28 @@ +# Copyright (c) 2022-2025, PostgreSQL Global Development Group + +project('amcheck', 'c') + +amcheck_path = '../../../../../../contrib/amcheck/' + +amcheck_sources = files( + amcheck_path / 'verify_heapam.c', + amcheck_path / 'verify_nbtree.c', +) + +pg_ext = dependency('postgresql-extension-warnings') + +amcheck = shared_module('amcheck', + amcheck_sources, + dependencies: pg_ext, + install_dir: pg_ext.get_variable(pkgconfig: 'dir_mod'), +) + +install_data( + amcheck_path / 'amcheck.control', + amcheck_path / 'amcheck--1.0.sql', + amcheck_path / 'amcheck--1.0--1.1.sql', + amcheck_path / 'amcheck--1.1--1.2.sql', + amcheck_path / 'amcheck--1.2--1.3.sql', + amcheck_path / 'amcheck--1.3--1.4.sql', + install_dir: pg_ext.get_variable(pkgconfig: 'dir_data'), +) diff --git a/src/test/modules/test_meson_extensions/test_pkg_config_extensions/auth_delay/meson.build b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/auth_delay/meson.build new file mode 100644 index 00000000000..98ad24cc183 --- /dev/null +++ b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/auth_delay/meson.build @@ -0,0 +1,17 @@ +# Copyright (c) 2022-2025, PostgreSQL Global Development Group + +project('auth_delay', 'c') + +auth_delay_path = '../../../../../../contrib/auth_delay/' + +auth_delay_sources = files( + auth_delay_path / 'auth_delay.c', +) + +pg_ext = dependency('postgresql-extension-warnings') + +auth_delay = shared_module('auth_delay', + auth_delay_sources, + dependencies: pg_ext, + install_dir: pg_ext.get_variable(pkgconfig: 'dir_mod'), +) diff --git a/src/test/modules/test_meson_extensions/test_pkg_config_extensions/meson.build b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/meson.build new file mode 100644 index 00000000000..dae94a384a1 --- /dev/null +++ b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/meson.build @@ -0,0 +1,24 @@ +# Copyright (c) 2022-2025, PostgreSQL Global Development Group + +# pkgconfig is not available on Windows, so skip it. +if host_machine.system() == 'windows' + subdir_done() +endif + +meson_extension_tests += { + 'name': 'amcheck', + 'kind': 'pkg_config', + 'sd': meson.current_source_dir() / 'amcheck', +} + +meson_extension_tests += { + 'name': 'auth_delay', + 'kind': 'pkg_config', + 'sd': meson.current_source_dir() / 'auth_delay', +} + +meson_extension_tests += { + 'name': 'postgres_fdw', + 'kind': 'pkg_config', + 'sd': meson.current_source_dir() / 'postgres_fdw', +} diff --git a/src/test/modules/test_meson_extensions/test_pkg_config_extensions/postgres_fdw/meson.build b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/postgres_fdw/meson.build new file mode 100644 index 00000000000..d49625e0611 --- /dev/null +++ b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/postgres_fdw/meson.build @@ -0,0 +1,31 @@ +# Copyright (c) 2022-2025, PostgreSQL Global Development Group + +project('auth_delay', 'c') + +postgres_fdw_path = '../../../../../../contrib/postgres_fdw/' + +postgres_fdw_sources = files( + postgres_fdw_path / 'connection.c', + postgres_fdw_path / 'deparse.c', + postgres_fdw_path / 'option.c', + postgres_fdw_path / 'postgres_fdw.c', + postgres_fdw_path / 'shippable.c', +) + +pg_ext = dependency('postgresql-extension-warnings') +libpq = dependency('libpq') + +postgres_fdw = shared_module('postgres_fdw', + postgres_fdw_sources, + dependencies: [pg_ext, libpq], + install_dir: pg_ext.get_variable(pkgconfig: 'dir_mod'), +) + +install_data( + postgres_fdw_path / 'postgres_fdw.control', + postgres_fdw_path / 'postgres_fdw--1.0.sql', + postgres_fdw_path / 'postgres_fdw--1.0--1.1.sql', + postgres_fdw_path / 'postgres_fdw--1.1--1.2.sql', + install_dir: pg_ext.get_variable(pkgconfig: 'dir_data'), +) + diff --git a/meson.build b/meson.build index 9bfa96ad255..765b7f6a207 100644 --- a/meson.build +++ b/meson.build @@ -2891,6 +2891,7 @@ nls_targets = [] # Define the tests to distribute them to the correct test styles later test_deps = [] tests = [] +meson_extension_tests = [] # Default options for targets @@ -3443,6 +3444,37 @@ sys.exit(sp.returncode) suite: ['setup']) +# it seems freebsd doesn't use libdir for pkgconfig path +if host_system == 'freebsd' + pkgconf_installdir = dir_prefix / 'libdata' / 'pkgconfig' +else + pkgconf_installdir = dir_prefix / dir_lib / 'pkgconfig' +endif +test_pkg_conf_file = files('src/tools/ci/test_meson_extensions') + +foreach test : meson_extension_tests + if test['kind'] not in ['pkg_config'] + error('unknown kind @0@ of test in @1@'.format(test['kind'], test['sd'])) + endif + + test_group = 'meson_@0@_extensions'.format(test['kind']) + + test(test_group / test['name'], + test_pkg_conf_file, + args: [ + '--meson', meson_bin.path(), + '--meson_args', meson_args, + '--test_dir', test['sd'], + '--test_out_dir', test_result_dir / 'meson_extensions' / test['name'], + '--builddir', meson.build_root(), + '--pkg_conf_path', get_option('pkg_config_path'), + '--', + cc.cmd_array(), + ], + suite: test_group, + ) + +endforeach ############################################################### # Test Generation diff --git a/src/tools/ci/test_meson_extensions b/src/tools/ci/test_meson_extensions new file mode 100644 index 00000000000..50358121f49 --- /dev/null +++ b/src/tools/ci/test_meson_extensions @@ -0,0 +1,70 @@ +#!/usr/bin/env python3 + +import argparse +import os +import shutil +import subprocess + +parser = argparse.ArgumentParser() + +parser.add_argument('--meson', help='path to meson binary', + type=str, required=True) +parser.add_argument('--meson_args', help='args of meson binary', + type=str, nargs='*', required=False) +parser.add_argument('--test_dir', help='test source directory', + type=str, required=True) +parser.add_argument('--test_out_dir', help='test output directory', + type=str, required=True) +parser.add_argument('--builddir', help='meson build directory', + type=str, required=True) +parser.add_argument('--pkg_conf_path', + help='PKG_CONF_PATH from surrounding meson build', + type=str, nargs='?', const='', required=False) +parser.add_argument('c_args', help='c_args from surrounding meson build', + nargs='*') + +args = parser.parse_args() + +meson_bin = args.meson +meson_args = ' '.join(args.meson_args) +test_source_dir = args.test_dir +test_out_dir = args.test_out_dir +build_dir = args.builddir +pkg_conf_path = args.pkg_conf_path +c_args = ' '.join(args.c_args) + +exit_code = 0 + +def remove_duplicates(duplicate_str): + words = duplicate_str.split() + return ' '.join(sorted(set(words), key=words.index)) + + +def run_tests(pkg_conf_path_local, message): + print('\n{}\n{}\n'.format('#' * 60, message), flush=True) + + env = {**os.environ, } + env['PKG_CONFIG_PATH'] = '{}:{}:{}'.format( + pkg_conf_path_local, pkg_conf_path, env.get('PKG_CONFIG_PATH', ''), + ).strip(': ') + env['CC'] = '{} {}'.format( + c_args, env.get('CC', ''), + ) + env['CC'] = remove_duplicates(env['CC']) + + # Clear the build directory beforehand. + if os.path.exists(test_out_dir): + shutil.rmtree(test_out_dir) + + if meson_args: + meson_setup_command = [meson_bin, meson_args, 'setup', test_out_dir] + else: + meson_setup_command = [meson_bin, 'setup', test_out_dir] + + meson_compile_command = ['meson', 'compile', '-C', test_out_dir, '-v'] + + subprocess.run(meson_setup_command, env=env, cwd=test_source_dir, check=True) + subprocess.run(meson_compile_command, cwd=test_source_dir, check=True) + +run_tests(os.path.join(build_dir, 'meson-uninstalled'), + message='Testing postgresql-extension-warnings-uninstalled') -- 2.47.2 [text/x-patch] v1-0004-meson-WIP-Add-docs-for-postgresql-extension.pc.patch (12.3K, 5-v1-0004-meson-WIP-Add-docs-for-postgresql-extension.pc.patch) download | inline diff: From 4b53335dcc2d26a134783f9070929f1b4a639e13 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Thu, 6 Mar 2025 17:46:57 +0300 Subject: [PATCH v1 4/6] meson: [WIP] Add docs for postgresql-extension.pc Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- doc/src/sgml/acronyms.sgml | 2 +- doc/src/sgml/extend.sgml | 101 +++++++++++++++++++++++-------------- doc/src/sgml/jit.sgml | 2 +- 3 files changed, 66 insertions(+), 39 deletions(-) diff --git a/doc/src/sgml/acronyms.sgml b/doc/src/sgml/acronyms.sgml index 58d0d90fece..240db5cb827 100644 --- a/doc/src/sgml/acronyms.sgml +++ b/doc/src/sgml/acronyms.sgml @@ -561,7 +561,7 @@ <term><acronym>PGXS</acronym></term> <listitem> <para> - <link linkend="extend-pgxs"><productname>PostgreSQL</productname> Extension System</link> + <link linkend="extend-postgres-pgxs"><productname>PostgreSQL</productname> Extension System</link> </para> </listitem> </varlistentry> diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml index ba492ca27c0..4a8c19b3f53 100644 --- a/doc/src/sgml/extend.sgml +++ b/doc/src/sgml/extend.sgml @@ -1421,7 +1421,7 @@ include $(PGXS) </programlisting> This makefile relies on <acronym>PGXS</acronym>, which is described - in <xref linkend="extend-pgxs"/>. The command <literal>make install</literal> + in <xref linkend="extend-postgres-pgxs"/>. The command <literal>make install</literal> will install the control and script files into the correct directory as reported by <application>pg_config</application>. </para> @@ -1434,21 +1434,26 @@ include $(PGXS) </sect2> </sect1> - <sect1 id="extend-pgxs"> + <sect1 id="extend-postgres"> <title>Extension Building Infrastructure</title> - <indexterm zone="extend-pgxs"> - <primary>pgxs</primary> - </indexterm> - <para> If you are thinking about distributing your <productname>PostgreSQL</productname> extension modules, setting up a portable build system for them can be fairly difficult. Therefore the <productname>PostgreSQL</productname> installation provides a build - infrastructure for extensions, called <acronym>PGXS</acronym>, so - that simple extension modules can be built simply against an - already installed server. <acronym>PGXS</acronym> is mainly intended + infrastructure for extensions, called <literal>PGXS</literal> + (<xref linkend="extend-postgres-pgxs"/>) and + its meson counterpart <literal>postgresql-extension.pc</literal> + (<xref linkend="extend-postgres-meson"/>). + </para> + + </sect1> + + <sect1 id="extend-postgres-pgxs"> + <title>PGXS</title> + + <para> <acronym>PGXS</acronym> is mainly intended for extensions that include C code, although it can be used for pure-SQL extensions too. Note that <acronym>PGXS</acronym> is not intended to be a universal build system framework that can be used @@ -1488,7 +1493,7 @@ include $(PGXS) Set one of these three variables to specify what is built: <variablelist> - <varlistentry id="extend-pgxs-modules"> + <varlistentry id="extend-postgres-pgxs-modules"> <term><varname>MODULES</varname></term> <listitem> <para> @@ -1498,7 +1503,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-module-big"> + <varlistentry id="extend-postgres-pgxs-module-big"> <term><varname>MODULE_big</varname></term> <listitem> <para> @@ -1508,7 +1513,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-program"> + <varlistentry id="extend-postgres-pgxs-program"> <term><varname>PROGRAM</varname></term> <listitem> <para> @@ -1522,7 +1527,7 @@ include $(PGXS) The following variables can also be set: <variablelist> - <varlistentry id="extend-pgxs-extension"> + <varlistentry id="extend-postgres-pgxs-extension"> <term><varname>EXTENSION</varname></term> <listitem> <para> @@ -1534,7 +1539,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-moduledir"> + <varlistentry id="extend-postgres-pgxs-moduledir"> <term><varname>MODULEDIR</varname></term> <listitem> <para> @@ -1547,7 +1552,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-data"> + <varlistentry id="extend-postgres-pgxs-data"> <term><varname>DATA</varname></term> <listitem> <para> @@ -1556,7 +1561,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-data-built"> + <varlistentry id="extend-postgres-pgxs-data-built"> <term><varname>DATA_built</varname></term> <listitem> <para> @@ -1567,7 +1572,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-data-tsearch"> + <varlistentry id="extend-postgres-pgxs-data-tsearch"> <term><varname>DATA_TSEARCH</varname></term> <listitem> <para> @@ -1577,7 +1582,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-docs"> + <varlistentry id="extend-postgres-pgxs-docs"> <term><varname>DOCS</varname></term> <listitem> <para> @@ -1587,7 +1592,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-headers"> + <varlistentry id="extend-postgres-pgxs-headers"> <term><varname>HEADERS</varname></term> <term><varname>HEADERS_built</varname></term> <listitem> @@ -1603,7 +1608,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-headers-module"> + <varlistentry id="extend-postgres-pgxs-headers-module"> <term><varname>HEADERS_$MODULE</varname></term> <term><varname>HEADERS_built_$MODULE</varname></term> <listitem> @@ -1629,7 +1634,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-scripts"> + <varlistentry id="extend-postgres-pgxs-scripts"> <term><varname>SCRIPTS</varname></term> <listitem> <para> @@ -1639,7 +1644,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-scripts-built"> + <varlistentry id="extend-postgres-pgxs-scripts-built"> <term><varname>SCRIPTS_built</varname></term> <listitem> <para> @@ -1650,7 +1655,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-regress"> + <varlistentry id="extend-postgres-pgxs-regress"> <term><varname>REGRESS</varname></term> <listitem> <para> @@ -1659,7 +1664,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-regress-opts"> + <varlistentry id="extend-postgres-pgxs-regress-opts"> <term><varname>REGRESS_OPTS</varname></term> <listitem> <para> @@ -1668,7 +1673,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-isolation"> + <varlistentry id="extend-postgres-pgxs-isolation"> <term><varname>ISOLATION</varname></term> <listitem> <para> @@ -1677,7 +1682,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-isolation-opts"> + <varlistentry id="extend-postgres-pgxs-isolation-opts"> <term><varname>ISOLATION_OPTS</varname></term> <listitem> <para> @@ -1687,7 +1692,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-tap-tests"> + <varlistentry id="extend-postgres-pgxs-tap-tests"> <term><varname>TAP_TESTS</varname></term> <listitem> <para> @@ -1696,7 +1701,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-no-install"> + <varlistentry id="extend-postgres-pgxs-no-install"> <term><varname>NO_INSTALL</varname></term> <listitem> <para> @@ -1706,7 +1711,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-no-installcheck"> + <varlistentry id="extend-postgres-pgxs-no-installcheck"> <term><varname>NO_INSTALLCHECK</varname></term> <listitem> <para> @@ -1715,7 +1720,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-extra-clean"> + <varlistentry id="extend-postgres-pgxs-extra-clean"> <term><varname>EXTRA_CLEAN</varname></term> <listitem> <para> @@ -1724,7 +1729,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-cppflags"> + <varlistentry id="extend-postgres-pgxs-pg-cppflags"> <term><varname>PG_CPPFLAGS</varname></term> <listitem> <para> @@ -1733,7 +1738,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-cflags"> + <varlistentry id="extend-postgres-pgxs-pg-cflags"> <term><varname>PG_CFLAGS</varname></term> <listitem> <para> @@ -1742,7 +1747,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-cxxflags"> + <varlistentry id="extend-postgres-pgxs-pg-cxxflags"> <term><varname>PG_CXXFLAGS</varname></term> <listitem> <para> @@ -1751,7 +1756,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-ldflags"> + <varlistentry id="extend-postgres-pgxs-pg-ldflags"> <term><varname>PG_LDFLAGS</varname></term> <listitem> <para> @@ -1760,7 +1765,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-libs"> + <varlistentry id="extend-postgres-pgxs-pg-libs"> <term><varname>PG_LIBS</varname></term> <listitem> <para> @@ -1769,7 +1774,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-shlib-link"> + <varlistentry id="extend-postgres-pgxs-shlib-link"> <term><varname>SHLIB_LINK</varname></term> <listitem> <para> @@ -1778,7 +1783,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-config"> + <varlistentry id="extend-postgres-pgxs-pg-config"> <term><varname>PG_CONFIG</varname></term> <listitem> <para> @@ -1894,4 +1899,26 @@ make VPATH=/path/to/extension/source/tree install </tip> </sect1> + <sect1 id="extend-postgres-meson"> + <title>postgresql-extension.pc</title> + + <para> + When Postgres is built by using meson, it generates + <literal>postgresql-extension.pc</literal> pkg-config file. Extension + libraries can use this file like <literal>PGXS</literal> + (<xref linkend="extend-postgres-pgxs"/>). + + To use the <literal>postgresql-extension.pc</literal> infrastructure for + your extension, you must write a simple meson.build file. In the + meson.build file, you need to include the + <literal>postgresql-extension.pc</literal> pkg-config file. Here is an + example that builds an extension module named isbn_issn, consisting of a + shared library containing some C code, an extension control file, an SQL + script, an include file (only needed if other modules might need to access + the extension functions without going via SQL), and a documentation text + file: + </para> + + </sect1> + </chapter> diff --git a/doc/src/sgml/jit.sgml b/doc/src/sgml/jit.sgml index 44e18bf1a6f..81a4644a97d 100644 --- a/doc/src/sgml/jit.sgml +++ b/doc/src/sgml/jit.sgml @@ -223,7 +223,7 @@ SET of types <literal>C</literal> and <literal>internal</literal>, as well as operators based on such functions. To do so for functions in extensions, the definitions of those functions need to be made available. - When using <link linkend="extend-pgxs">PGXS</link> to build an extension + When using <link linkend="extend-postgres-pgxs">PGXS</link> to build an extension against a server that has been compiled with LLVM JIT support, the relevant files will be built and installed automatically. </para> -- 2.47.2 [text/x-patch] v1-0005-meson-Add-LLVM-bitcode-emission.patch (7.3K, 6-v1-0005-meson-Add-LLVM-bitcode-emission.patch) download | inline diff: From 4deec61ecfbf5b0de304e82d1785d5c26fc28cb3 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Fri, 7 Mar 2025 12:10:58 +0300 Subject: [PATCH v1 5/6] meson: Add LLVM bitcode emission Bitcode files are installed into $pkglibdir/bitcode/ directory. Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/backend/jit/llvm/bitcode/meson.build | 48 ++++++++++++++++++++++++ src/backend/jit/llvm/meson.build | 31 +++++++++------ src/backend/meson.build | 6 +++ meson.build | 21 +++++++++++ src/tools/irlink | 25 ++++++++++++ 5 files changed, 120 insertions(+), 11 deletions(-) create mode 100644 src/backend/jit/llvm/bitcode/meson.build create mode 100644 src/tools/irlink diff --git a/src/backend/jit/llvm/bitcode/meson.build b/src/backend/jit/llvm/bitcode/meson.build new file mode 100644 index 00000000000..577b4ddb210 --- /dev/null +++ b/src/backend/jit/llvm/bitcode/meson.build @@ -0,0 +1,48 @@ +# Copyright (c) 2022-2024, PostgreSQL Global Development Group +# +# emit LLVM bitcode for JIT inlining + +assert(llvm.found()) + +foreach bitcode_module : bitcode_modules + bitcode_targets = [] + bitcode_obj = bitcode_module['target'] + bitcode_cflags_local = bitcode_cflags + bitcode_module.get('additional_flags', []) + + if 'name' not in bitcode_module + bitcode_name = bitcode_obj.name() + else + bitcode_name = bitcode_module['name'] + endif + + foreach srcfile : bitcode_module['srcfiles'] + if meson.version().version_compare('>=0.59') + srcfilename = fs.parent(srcfile) / fs.name(srcfile) + else + srcfilename = '@0@'.format(srcfile) + endif + targetname = '@0@_@[email protected]'.format( + bitcode_name, + srcfilename.underscorify(), + ) + bitcode_targets += custom_target( + targetname, + depends: [bitcode_module['target']], + input: [srcfile], + output: targetname, + command: [llvm_irgen_command, bitcode_cflags_local], + install: true, + install_dir: dir_bitcode, + ) + endforeach + + index_name = '@[email protected]'.format(bitcode_name) + bitcode_index = custom_target('@0@'.format(bitcode_name), + output: index_name, + input: bitcode_targets, + command: [irlink, '--lto', llvm_lto, '--outdir', '@OUTDIR@', '--index', index_name, '@INPUT@'], + install: true, + install_dir: dir_bitcode, + ) + backend_targets += bitcode_index +endforeach diff --git a/src/backend/jit/llvm/meson.build b/src/backend/jit/llvm/meson.build index c8e06dfbe35..2dd922e573b 100644 --- a/src/backend/jit/llvm/meson.build +++ b/src/backend/jit/llvm/meson.build @@ -42,21 +42,22 @@ backend_targets += llvmjit # Define a few bits and pieces used here and elsewhere to generate bitcode -llvm_irgen_args = [ - '-c', '-o', '@OUTPUT@', '@INPUT@', +llvm_irgen_command = [] +if ccache.found() + llvm_irgen_command += ccache +endif + +llvm_irgen_command += [ + clang, + '-c', '-o', '@OUTPUT0@', '@INPUT0@', '-flto=thin', '-emit-llvm', - '-MD', '-MQ', '@OUTPUT@', '-MF', '@DEPFILE@', '-O2', '-Wno-ignored-attributes', '-Wno-empty-body', + '-Wno-unknown-warning-option', + '-Wno-compound-token-split-by-macro', ] - -if ccache.found() - llvm_irgen_command = ccache - llvm_irgen_args = [clang.path()] + llvm_irgen_args -else - llvm_irgen_command = clang -endif +llvm_irgen_dep_args = ['-MD', '-MQ', '@OUTPUT0@', '-MF', '@DEPFILE@'] # XXX: Need to determine proper version of the function cflags for clang @@ -73,7 +74,7 @@ bitcode_cflags += '-I@SOURCE_ROOT@/src/include' # Note this is intentionally not installed to bitcodedir, as it's not for # inlining llvmjit_types = custom_target('llvmjit_types.bc', - command: [llvm_irgen_command] + llvm_irgen_args + bitcode_cflags, + command: llvm_irgen_command + llvm_irgen_dep_args + bitcode_cflags, input: 'llvmjit_types.c', output: 'llvmjit_types.bc', depends: [postgres], @@ -82,3 +83,11 @@ llvmjit_types = custom_target('llvmjit_types.bc', depfile: '@[email protected]', ) backend_targets += llvmjit_types + +# Figure out -I's needed to build all postgres code, including all its +# dependencies +pkg_config = find_program(['pkg-config', 'pkgconf'], required: true) +r = run_command(pkg_config, + ['--cflags-only-I', meson.build_root() / 'meson-uninstalled/postgresql-extension-uninstalled.pc'], + check: true) +bitcode_cflags += r.stdout().split() diff --git a/src/backend/meson.build b/src/backend/meson.build index 9d79d4d058c..5fb33660d3d 100644 --- a/src/backend/meson.build +++ b/src/backend/meson.build @@ -140,6 +140,12 @@ postgres = executable('postgres', backend_targets += postgres +bitcode_modules += { + 'name': 'postgres', + 'target': postgres_lib, + 'srcfiles': backend_sources, +} + pg_mod_c_args = cflags_mod pg_mod_cpp_args = cxxflags_mod pg_mod_link_args = ldflags_sl + ldflags_mod diff --git a/meson.build b/meson.build index 765b7f6a207..a01dbca66d1 100644 --- a/meson.build +++ b/meson.build @@ -814,6 +814,8 @@ if add_languages('cpp', required: llvmopt, native: false) # Some distros put LLVM and clang in different paths, so fallback to # find via PATH, too. clang = find_program(llvm_binpath / 'clang', 'clang', required: true) + llvm_lto = find_program(llvm_binpath / 'llvm-lto', required: true) + irlink = find_program('src/tools/irlink', native: true) endif elif llvmopt.auto() message('llvm requires a C++ compiler') @@ -2893,6 +2895,11 @@ test_deps = [] tests = [] meson_extension_tests = [] +# List of object files + source files to generated LLVM IR for inlining. +# Each element is a hash of: +# {'target': target, 'srcfiles': ..., 'additional_flags': ...}. +bitcode_modules = [] + # Default options for targets @@ -3212,6 +3219,11 @@ subdir('src/interfaces/ecpg/test') subdir('doc/src/sgml') +# generate bitcode for JIT inlining after giving contrib modules etc a chance +# to add themselves to bitcode_modules[] +subdir('src/backend/jit/llvm/bitcode', if_found: llvm) + + generated_sources_ac += {'': ['GNUmakefile']} # After processing src/test, add test_install_libs to the testprep_targets @@ -3847,6 +3859,15 @@ if meson.version().version_compare('>=0.57') section: 'Programs', ) + if llvm.found() + summary( + { + 'clang': clang, + }, + section: 'Programs', + ) + endif + summary( { 'bonjour': bonjour, diff --git a/src/tools/irlink b/src/tools/irlink new file mode 100644 index 00000000000..793c0abf91a --- /dev/null +++ b/src/tools/irlink @@ -0,0 +1,25 @@ +#!/usr/bin/env python3 + +import argparse +import os +import shutil +import subprocess +import sys + +parser = argparse.ArgumentParser( + description='generate PostgreSQL JIT IR module') + +parser.add_argument('--index', type=str, required=True) +parser.add_argument('--lto', type=str, required=True) +parser.add_argument('--outdir', type=str, required=True) +parser.add_argument('INPUT', type=str, nargs='+') + +args = parser.parse_args() + +file_names = [os.path.basename(f) for f in args.INPUT] +command = [args.lto, + '-thinlto', '-thinlto-action=thinlink', + '-o', args.index] + file_names +res = subprocess.run(command, cwd=args.outdir) + +exit(res.returncode) -- 2.47.2 [text/x-patch] v1-0006-meson-Generate-bitcode-files-of-contrib-extension.patch (21.5K, 7-v1-0006-meson-Generate-bitcode-files-of-contrib-extension.patch) download | inline diff: From 0ee14cab08c778ca49c438ae3d80a695f0d68f84 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Fri, 7 Mar 2025 10:34:34 +0300 Subject: [PATCH v1 6/6] meson: Generate bitcode files of contrib extensions The extensions which the bitcode files will be generated in are selected manually. Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/interfaces/libpq/meson.build | 3 +++ contrib/bloom/meson.build | 5 +++++ contrib/bool_plperl/meson.build | 9 +++++++++ contrib/btree_gin/meson.build | 5 +++++ contrib/btree_gist/meson.build | 5 +++++ contrib/citext/meson.build | 5 +++++ contrib/cube/meson.build | 10 ++++++++++ contrib/dict_int/meson.build | 5 +++++ contrib/dict_xsyn/meson.build | 5 +++++ contrib/earthdistance/meson.build | 6 ++++++ contrib/fuzzystrmatch/meson.build | 9 +++++++++ contrib/hstore/meson.build | 7 +++++++ contrib/hstore_plperl/meson.build | 10 ++++++++++ contrib/hstore_plpython/meson.build | 12 ++++++++++++ contrib/intarray/meson.build | 5 +++++ contrib/isn/meson.build | 5 +++++ contrib/jsonb_plperl/meson.build | 8 ++++++++ contrib/jsonb_plpython/meson.build | 10 ++++++++++ contrib/lo/meson.build | 5 +++++ contrib/ltree/meson.build | 10 ++++++++++ contrib/ltree_plpython/meson.build | 11 +++++++++++ contrib/pg_buffercache/meson.build | 5 +++++ contrib/pg_freespacemap/meson.build | 5 +++++ contrib/pg_logicalinspect/meson.build | 5 +++++ contrib/pg_surgery/meson.build | 4 ++++ contrib/pg_trgm/meson.build | 5 +++++ contrib/pgcrypto/meson.build | 4 ++++ contrib/seg/meson.build | 9 +++++++++ contrib/spi/meson.build | 20 ++++++++++++++++++++ contrib/sslinfo/meson.build | 5 +++++ contrib/tablefunc/meson.build | 5 +++++ contrib/tcn/meson.build | 5 +++++ contrib/tsm_system_rows/meson.build | 5 +++++ contrib/tsm_system_time/meson.build | 5 +++++ contrib/unaccent/meson.build | 5 +++++ contrib/uuid-ossp/meson.build | 5 +++++ contrib/xml2/meson.build | 5 +++++ src/pl/plperl/meson.build | 1 + src/pl/plpython/meson.build | 1 + meson.build | 1 + 40 files changed, 250 insertions(+) diff --git a/src/interfaces/libpq/meson.build b/src/interfaces/libpq/meson.build index 19f4a52a97a..4742489e0ce 100644 --- a/src/interfaces/libpq/meson.build +++ b/src/interfaces/libpq/meson.build @@ -50,6 +50,9 @@ export_file = custom_target('libpq.exports', libpq_inc = include_directories('.', '../../port') libpq_c_args = ['-DSO_MAJOR_VERSION=5'] +# libpq-fe.h is needed to generate bitcode for some extensions +libpq_dir = meson.current_source_dir() + # Not using both_libraries() here as # 1) resource files should only be in the shared library # 2) we want the .pc file to include a dependency to {pgport,common}_static for diff --git a/contrib/bloom/meson.build b/contrib/bloom/meson.build index 695712a455e..1090c281532 100644 --- a/contrib/bloom/meson.build +++ b/contrib/bloom/meson.build @@ -28,6 +28,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': bloom, + 'srcfiles': bloom_sources, +} + tests += { 'name': 'bloom', 'sd': meson.current_source_dir(), diff --git a/contrib/bool_plperl/meson.build b/contrib/bool_plperl/meson.build index f489d99044c..1758adb7489 100644 --- a/contrib/bool_plperl/meson.build +++ b/contrib/bool_plperl/meson.build @@ -37,6 +37,15 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': bool_plperl, + 'srcfiles': bool_plperl_sources, + 'additional_flags': [ + '-I@0@'.format(plperl_dir), + perl_ccflags + ] +} + tests += { 'name': 'bool_plperl', 'sd': meson.current_source_dir(), diff --git a/contrib/btree_gin/meson.build b/contrib/btree_gin/meson.build index b2749f6e669..26d49d005ab 100644 --- a/contrib/btree_gin/meson.build +++ b/contrib/btree_gin/meson.build @@ -25,6 +25,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': btree_gin, + 'srcfiles': btree_gin_sources, +} + tests += { 'name': 'btree_gin', 'sd': meson.current_source_dir(), diff --git a/contrib/btree_gist/meson.build b/contrib/btree_gist/meson.build index f4fa9574f1f..2649d5a7ffb 100644 --- a/contrib/btree_gist/meson.build +++ b/contrib/btree_gist/meson.build @@ -54,6 +54,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': btree_gist, + 'srcfiles': btree_gist_sources, +} + tests += { 'name': 'btree_gist', 'sd': meson.current_source_dir(), diff --git a/contrib/citext/meson.build b/contrib/citext/meson.build index 7fff34f2368..fa07ff72be4 100644 --- a/contrib/citext/meson.build +++ b/contrib/citext/meson.build @@ -30,6 +30,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': citext, + 'srcfiles': citext_sources, +} + tests += { 'name': 'citext', 'sd': meson.current_source_dir(), diff --git a/contrib/cube/meson.build b/contrib/cube/meson.build index fd3c057f469..52bd6a9fb81 100644 --- a/contrib/cube/meson.build +++ b/contrib/cube/meson.build @@ -3,6 +3,7 @@ cube_sources = files( 'cube.c', ) +bc_cube_sources = cube_sources cube_scan = custom_target('cubescan', input: 'cubescan.l', @@ -48,6 +49,15 @@ install_headers( install_dir: dir_include_extension / 'cube', ) +cube_dir = meson.current_source_dir() +bitcode_modules += { + 'target': cube, + 'srcfiles': bc_cube_sources, + 'additional_flags': [ + '-I@0@'.format(cube_dir), + ] +} + tests += { 'name': 'cube', 'sd': meson.current_source_dir(), diff --git a/contrib/dict_int/meson.build b/contrib/dict_int/meson.build index ab41588547b..894c8a9c94d 100644 --- a/contrib/dict_int/meson.build +++ b/contrib/dict_int/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': dict_int, + 'srcfiles': dict_int_sources, +} + tests += { 'name': 'dict_int', 'sd': meson.current_source_dir(), diff --git a/contrib/dict_xsyn/meson.build b/contrib/dict_xsyn/meson.build index 93f41b1f963..d4bd4a6ef5a 100644 --- a/contrib/dict_xsyn/meson.build +++ b/contrib/dict_xsyn/meson.build @@ -29,6 +29,11 @@ install_data( } ) +bitcode_modules += { + 'target': dict_xsyn, + 'srcfiles': dict_xsyn_sources, +} + tests += { 'name': 'dict_xsyn', 'sd': meson.current_source_dir(), diff --git a/contrib/earthdistance/meson.build b/contrib/earthdistance/meson.build index a5f8d1a3ffa..c101359b118 100644 --- a/contrib/earthdistance/meson.build +++ b/contrib/earthdistance/meson.build @@ -24,6 +24,12 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': earthdistance, + 'srcfiles': earthdistance_sources, +} + + tests += { 'name': 'earthdistance', 'sd': meson.current_source_dir(), diff --git a/contrib/fuzzystrmatch/meson.build b/contrib/fuzzystrmatch/meson.build index f52daa4ea1c..fc5e88d0c92 100644 --- a/contrib/fuzzystrmatch/meson.build +++ b/contrib/fuzzystrmatch/meson.build @@ -5,6 +5,7 @@ fuzzystrmatch_sources = files( 'dmetaphone.c', 'fuzzystrmatch.c', ) +bc_fuzzystrmatch_sources = fuzzystrmatch_sources daitch_mokotoff_h = custom_target('daitch_mokotoff', input: 'daitch_mokotoff_header.pl', @@ -35,6 +36,14 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': fuzzystrmatch, + 'srcfiles': bc_fuzzystrmatch_sources, + 'additional_flags': [ + '-I@0@'.format(meson.current_build_dir()) + ] +} + tests += { 'name': 'fuzzystrmatch', 'sd': meson.current_source_dir(), diff --git a/contrib/hstore/meson.build b/contrib/hstore/meson.build index 39622d1024d..344a620273f 100644 --- a/contrib/hstore/meson.build +++ b/contrib/hstore/meson.build @@ -43,6 +43,13 @@ install_headers( install_dir: dir_include_extension / 'hstore', ) +# some libraries include "hstore/hstore.h" instead of "hstore.h" +hstore_dir_up = join_paths(meson.current_source_dir(), '..') +bitcode_modules += { + 'target': hstore, + 'srcfiles': hstore_sources, +} + tests += { 'name': 'hstore', 'sd': meson.current_source_dir(), diff --git a/contrib/hstore_plperl/meson.build b/contrib/hstore_plperl/meson.build index 60b8ad23569..b689b55e1c9 100644 --- a/contrib/hstore_plperl/meson.build +++ b/contrib/hstore_plperl/meson.build @@ -37,6 +37,16 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': hstore_plperl, + 'srcfiles': hstore_plperl_sources, + 'additional_flags': [ + '-I@0@'.format(hstore_dir_up), + '-I@0@'.format(plperl_dir), + perl_ccflags, + ] +} + tests += { 'name': 'hstore_plperl', 'sd': meson.current_source_dir(), diff --git a/contrib/hstore_plpython/meson.build b/contrib/hstore_plpython/meson.build index ea8e20a377b..d68de3dc46b 100644 --- a/contrib/hstore_plpython/meson.build +++ b/contrib/hstore_plpython/meson.build @@ -30,6 +30,18 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': hstore_plpython, + 'srcfiles': hstore_plpython_sources, + 'additional_flags': [ + '-I@0@'.format(hstore_dir_up), + '-DPLPYTHON_LIBNAME="plpython3"', + '-I@0@'.format(python3_inc_dir), + '-I@0@'.format(plpython_dir), + perl_ccflags, + ] +} + hstore_plpython_regress = [ 'hstore_plpython' ] diff --git a/contrib/intarray/meson.build b/contrib/intarray/meson.build index fae9add981d..3d7da0c9c9a 100644 --- a/contrib/intarray/meson.build +++ b/contrib/intarray/meson.build @@ -33,6 +33,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': intarray, + 'srcfiles': intarray_sources, +} + tests += { 'name': 'intarray', 'sd': meson.current_source_dir(), diff --git a/contrib/isn/meson.build b/contrib/isn/meson.build index fbbeeff01bb..d42473134ea 100644 --- a/contrib/isn/meson.build +++ b/contrib/isn/meson.build @@ -29,6 +29,11 @@ install_headers( install_dir: dir_include_extension / 'isn', ) +bitcode_modules += { + 'target': isn, + 'srcfiles': isn_sources, +} + tests += { 'name': 'isn', 'sd': meson.current_source_dir(), diff --git a/contrib/jsonb_plperl/meson.build b/contrib/jsonb_plperl/meson.build index 95a9a7bc082..febafc73a99 100644 --- a/contrib/jsonb_plperl/meson.build +++ b/contrib/jsonb_plperl/meson.build @@ -37,6 +37,14 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': jsonb_plperl, + 'srcfiles': jsonb_plperl_sources, + 'additional_flags': [ + '-I@0@'.format(plperl_dir), + perl_ccflags, + ] +} tests += { 'name': 'jsonb_plperl', diff --git a/contrib/jsonb_plpython/meson.build b/contrib/jsonb_plpython/meson.build index 5fe80483e58..6be92221272 100644 --- a/contrib/jsonb_plpython/meson.build +++ b/contrib/jsonb_plpython/meson.build @@ -30,6 +30,16 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': jsonb_plpython, + 'srcfiles': jsonb_plpython_sources, + 'additional_flags': [ + '-DPLPYTHON_LIBNAME="plpython3"', + '-I@0@'.format(python3_inc_dir), + '-I@0@'.format(plpython_dir), + ] +} + jsonb_plpython_regress = [ 'jsonb_plpython' ] diff --git a/contrib/lo/meson.build b/contrib/lo/meson.build index 2d78907ba12..1222faa9169 100644 --- a/contrib/lo/meson.build +++ b/contrib/lo/meson.build @@ -24,6 +24,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': lo, + 'srcfiles': lo_sources, +} + tests += { 'name': 'lo', 'sd': meson.current_source_dir(), diff --git a/contrib/ltree/meson.build b/contrib/ltree/meson.build index f9b06302839..cda86935544 100644 --- a/contrib/ltree/meson.build +++ b/contrib/ltree/meson.build @@ -41,6 +41,16 @@ install_headers( install_dir: dir_include_extension / 'ltree', ) +ltree_dir = meson.current_source_dir() +ltree_dir_up = join_paths(ltree_dir, '..') +bitcode_modules += { + 'target': ltree, + 'srcfiles': ltree_sources, + 'additional_flags': [ + '-I@0@'.format(ltree_dir) + ] +} + tests += { 'name': 'ltree', 'sd': meson.current_source_dir(), diff --git a/contrib/ltree_plpython/meson.build b/contrib/ltree_plpython/meson.build index a37732c486b..74b285da05c 100644 --- a/contrib/ltree_plpython/meson.build +++ b/contrib/ltree_plpython/meson.build @@ -30,6 +30,17 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': ltree_plpython, + 'srcfiles': ltree_plpython_sources, + 'additional_flags': [ + '-I@0@'.format(ltree_dir_up), + '-DPLPYTHON_LIBNAME="plpython3"', + '-I@0@'.format(python3_inc_dir), + '-I@0@'.format(plpython_dir), + ] +} + ltree_plpython_regress = [ 'ltree_plpython' ] diff --git a/contrib/pg_buffercache/meson.build b/contrib/pg_buffercache/meson.build index 12d1fe48717..5385318fdc9 100644 --- a/contrib/pg_buffercache/meson.build +++ b/contrib/pg_buffercache/meson.build @@ -27,6 +27,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pg_buffercache, + 'srcfiles': pg_buffercache_sources, +} + tests += { 'name': 'pg_buffercache', 'sd': meson.current_source_dir(), diff --git a/contrib/pg_freespacemap/meson.build b/contrib/pg_freespacemap/meson.build index ff8eda3580e..120def1ad2e 100644 --- a/contrib/pg_freespacemap/meson.build +++ b/contrib/pg_freespacemap/meson.build @@ -25,6 +25,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pg_freespacemap, + 'srcfiles': pg_freespacemap_sources, +} + tests += { 'name': 'pg_freespacemap', 'sd': meson.current_source_dir(), diff --git a/contrib/pg_logicalinspect/meson.build b/contrib/pg_logicalinspect/meson.build index 5c0528a8c63..1b4f46277ed 100644 --- a/contrib/pg_logicalinspect/meson.build +++ b/contrib/pg_logicalinspect/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pg_logicalinspect, + 'srcfiles': pg_logicalinspect_sources, +} + tests += { 'name': 'pg_logicalinspect', 'sd': meson.current_source_dir(), diff --git a/contrib/pg_surgery/meson.build b/contrib/pg_surgery/meson.build index c6cfa9c4694..9d7199392c7 100644 --- a/contrib/pg_surgery/meson.build +++ b/contrib/pg_surgery/meson.build @@ -22,6 +22,10 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pg_surgery, + 'srcfiles': pg_surgery_sources, +} tests += { 'name': 'pg_surgery', diff --git a/contrib/pg_trgm/meson.build b/contrib/pg_trgm/meson.build index a31aa5c574d..408e287172e 100644 --- a/contrib/pg_trgm/meson.build +++ b/contrib/pg_trgm/meson.build @@ -32,6 +32,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pg_trgm, + 'srcfiles': pg_trgm_sources, +} + tests += { 'name': 'pg_trgm', 'sd': meson.current_source_dir(), diff --git a/contrib/pgcrypto/meson.build b/contrib/pgcrypto/meson.build index 7a4e8e76d64..d6b0559f232 100644 --- a/contrib/pgcrypto/meson.build +++ b/contrib/pgcrypto/meson.build @@ -98,6 +98,10 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pgcrypto, + 'srcfiles': pgcrypto_sources, +} tests += { 'name': 'pgcrypto', diff --git a/contrib/seg/meson.build b/contrib/seg/meson.build index e331e097230..c9ed585e504 100644 --- a/contrib/seg/meson.build +++ b/contrib/seg/meson.build @@ -3,6 +3,7 @@ seg_sources = files( 'seg.c', ) +bc_seg_sources = seg_sources seg_scan = custom_target('segscan', input: 'segscan.l', @@ -47,6 +48,14 @@ install_headers( install_dir: dir_include_extension / 'seg', ) +bitcode_modules += { + 'target': seg, + 'srcfiles': bc_seg_sources, + 'additional_flags': [ + '-I@0@'.format(meson.current_source_dir()), + ] +} + tests += { 'name': 'seg', 'sd': meson.current_source_dir(), diff --git a/contrib/spi/meson.build b/contrib/spi/meson.build index eeab1ab210b..b16c4384bb0 100644 --- a/contrib/spi/meson.build +++ b/contrib/spi/meson.build @@ -24,6 +24,11 @@ install_data('autoinc.example', kwargs: contrib_doc_args, ) +bitcode_modules += { + 'target': autoinc, + 'srcfiles': autoinc_sources, +} + insert_username_sources = files( 'insert_username.c', @@ -51,6 +56,11 @@ install_data('insert_username.example', kwargs: contrib_doc_args, ) +bitcode_modules += { + 'target': insert_username, + 'srcfiles': insert_username_sources, +} + moddatetime_sources = files( 'moddatetime.c', @@ -78,6 +88,11 @@ install_data('moddatetime.example', kwargs: contrib_doc_args, ) +bitcode_modules += { + 'target': moddatetime, + 'srcfiles': moddatetime_sources, +} + # this is needed for the regression tests; # comment out if you want a quieter refint package for other uses @@ -107,3 +122,8 @@ install_data('refint.control', 'refint--1.0.sql', install_data('refint.example', kwargs: contrib_doc_args, ) + +bitcode_modules += { + 'target': refint, + 'srcfiles': refint_sources, +} diff --git a/contrib/sslinfo/meson.build b/contrib/sslinfo/meson.build index 4c513759200..5750d783b92 100644 --- a/contrib/sslinfo/meson.build +++ b/contrib/sslinfo/meson.build @@ -29,3 +29,8 @@ install_data( 'sslinfo.control', kwargs: contrib_data_args, ) + +bitcode_modules += { + 'target': sslinfo, + 'srcfiles': sslinfo_sources, +} diff --git a/contrib/tablefunc/meson.build b/contrib/tablefunc/meson.build index ee67272ec0a..6c26ffafddf 100644 --- a/contrib/tablefunc/meson.build +++ b/contrib/tablefunc/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': tablefunc, + 'srcfiles': tablefunc_sources, +} + tests += { 'name': 'tablefunc', 'sd': meson.current_source_dir(), diff --git a/contrib/tcn/meson.build b/contrib/tcn/meson.build index 6ffb136af90..df71e8a91f9 100644 --- a/contrib/tcn/meson.build +++ b/contrib/tcn/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': tcn, + 'srcfiles': tcn_sources, +} + tests += { 'name': 'tcn', 'sd': meson.current_source_dir(), diff --git a/contrib/tsm_system_rows/meson.build b/contrib/tsm_system_rows/meson.build index b8cece3d80f..1ea74cce949 100644 --- a/contrib/tsm_system_rows/meson.build +++ b/contrib/tsm_system_rows/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': tsm_system_rows, + 'srcfiles': tsm_system_rows_sources, +} + tests += { 'name': 'tsm_system_rows', 'sd': meson.current_source_dir(), diff --git a/contrib/tsm_system_time/meson.build b/contrib/tsm_system_time/meson.build index 8a143a8f8e6..1f6b98d3ec4 100644 --- a/contrib/tsm_system_time/meson.build +++ b/contrib/tsm_system_time/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': tsm_system_time, + 'srcfiles': tsm_system_time_sources, +} + tests += { 'name': 'tsm_system_time', 'sd': meson.current_source_dir(), diff --git a/contrib/unaccent/meson.build b/contrib/unaccent/meson.build index 33d4649bae1..1f5d120e3d8 100644 --- a/contrib/unaccent/meson.build +++ b/contrib/unaccent/meson.build @@ -28,6 +28,11 @@ install_data( install_dir: dir_data / 'tsearch_data' ) +bitcode_modules += { + 'target': unaccent, + 'srcfiles': unaccent_sources, +} + # XXX: Implement downlo tests += { 'name': 'unaccent', diff --git a/contrib/uuid-ossp/meson.build b/contrib/uuid-ossp/meson.build index 982f27c085f..8d21bb0a996 100644 --- a/contrib/uuid-ossp/meson.build +++ b/contrib/uuid-ossp/meson.build @@ -29,6 +29,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': uuid_ossp, + 'srcfiles': uuid_ossp_sources, +} + tests += { 'name': 'uuid-ossp', 'sd': meson.current_source_dir(), diff --git a/contrib/xml2/meson.build b/contrib/xml2/meson.build index 08d3c3b8e30..b23a8fe0cd7 100644 --- a/contrib/xml2/meson.build +++ b/contrib/xml2/meson.build @@ -32,6 +32,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': xml2, + 'srcfiles': xml2_sources, +} + tests += { 'name': 'xml2', 'sd': meson.current_source_dir(), diff --git a/src/pl/plperl/meson.build b/src/pl/plperl/meson.build index b463d4d56c5..279277dee73 100644 --- a/src/pl/plperl/meson.build +++ b/src/pl/plperl/meson.build @@ -37,6 +37,7 @@ foreach n : ['SPI', 'Util'] plperl_sources += xs_c endforeach +plperl_dir = meson.current_source_dir() plperl_inc = include_directories('.') if host_system == 'windows' diff --git a/src/pl/plpython/meson.build b/src/pl/plpython/meson.build index 709e5932a93..e2c19771d85 100644 --- a/src/pl/plpython/meson.build +++ b/src/pl/plpython/meson.build @@ -29,6 +29,7 @@ plpython_sources += custom_target('spiexceptions.h', # FIXME: need to duplicate import library ugliness? plpython_inc = include_directories('.') +plpython_dir = meson.current_source_dir() if host_system == 'windows' plpython_sources += rc_lib_gen.process(win32ver_rc, extra_args: [ diff --git a/meson.build b/meson.build index a01dbca66d1..9e20be4590d 100644 --- a/meson.build +++ b/meson.build @@ -1244,6 +1244,7 @@ if not pyopt.disabled() python3_inst = pm.find_installation(python.path(), required: pyopt) if python3_inst.found() python3_dep = python3_inst.dependency(embed: true, required: pyopt) + python3_inc_dir = python3_inst.get_variable('INCLUDEPY') # Remove this check after we depend on Meson >= 1.1.0 if not cc.check_header('Python.h', dependencies: python3_dep, required: pyopt, include_directories: postgres_inc) python3_dep = not_found_dep -- 2.47.2 ^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: meson vs. llvm bitcode files @ 2025-03-10 22:04 Diego Fronza <[email protected]> parent: Nazir Bilal Yavuz <[email protected]> 0 siblings, 1 reply; 10+ messages in thread From: Diego Fronza @ 2025-03-10 22:04 UTC (permalink / raw) To: Nazir Bilal Yavuz <[email protected]>; +Cc: Peter Eisentraut <[email protected]>; pgsql-hackers; Andres Freund <[email protected]> Hello, I did a full review on the provided patches plus some tests, I was able to validate that the loading of bitcode modules is working also JIT works for both backend and contrib modules. To test JIT on contrib modules I just lowered the costs for all jit settings and used the intarray extension, using the data/test__int.data: CREATE EXTENSION intarray; CREATE TABLE test__int( a int[] );1 \copy test__int from 'data/test__int.data' For queries any from line 98+ on contrib/intarray/sql/_int.sql will work. Then I added extra debug messages to llvmjit_inline.cpp on add_module_to_inline_search_path() function, also on llvm_build_inline_plan(), I was able to see many functions in this module being successfully inlined. I'm attaching a new patch based on your original work which add further support for generating bitcode from: - Generated backend sources: processed by flex, bison, etc. - Generated contrib module sources, On this patch I just included fmgrtab.c and src/backend/parser for the backend generated code. For contrib generated sources I added contrib/cube as an example. All relevant details about the changes are included in the patch itself. As you may know already I also created a PR focused on llvm bitcode emission on meson, it generates bitcode for all backend and contribution modules, currently under review by some colleagues at Percona: https://github.com/percona/postgres/pull/103 I'm curious if we should get all or some of the generated backend sources compiled to bitcode, similar to contrib modules. Please let me know your thoughts and how we can proceed to get this feature included, thank you. Regards, Diego Fronza Percona On Fri, Mar 7, 2025 at 7:52 AM Nazir Bilal Yavuz <[email protected]> wrote: > Hi, > > On Thu, 5 Sept 2024 at 12:24, Nazir Bilal Yavuz <[email protected]> > wrote: > > > > I found that Andres shared a patch > > (v17-0021-meson-Add-LLVM-bitcode-emission.patch) a while ago [1]. > > Andres and I continued to work on that. I think the patches are in > sharable state now and I wanted to hear opinions before proceeding > further. After applying the patches, bitcode files should be installed > into $pkglibdir/bitcode/ directory if the llvm is found. > > There are 6 patches attached: > > v1-0001-meson-Add-generated-header-stamps: > > This patch is trivial. Instead of having targets depending directly on > the generated headers, have them depend on a stamp file. The benefit > of using a stamp file is that it makes ninja.build smaller and meson > setup faster. > ---------- > > v1-0002-meson-Add-postgresql-extension.pc-for-building-extension-libraries: > > This patch is for generating postgresql-extension.pc file which can be > used for building extensions libraries. > > Normally, there is no need to use this .pc file for generating bitcode > files. However, since there is no clear way to get all include paths > for building bitcode files, this .pc file is later used for this > purpose (by running pkg-config --cflags-only-I > postgresql-extension-uninstalled.pc) [1]. > ---------- > > v1-0003-meson-Test-building-extensions-by-using-postgresql-extension.pc: > [Not needed for generating bitcode files] > > This is a patch for testing if extensions can be built by using > postgresql-extension.pc. I added that commit as an example of using > postgresql-extension.pc to build extensions. > ---------- > > v1-0004-meson-WIP-Add-docs-for-postgresql-extension.pc: [Not needed > for generating bitcode files] > > I added this patch in case we recommend people to use > postgresql-extension.pc to build extension libraries. I am not sure if > we want to do that because there are still TODOs about > postgresql-extension.pc like running test suites. I just wanted to > show my plan, dividing 'Extension Building Infrastructure' into two, > 'PGXS' and 'postgresql-extension.pc'. > ---------- > > v1-0005-meson-Add-LLVM-bitcode-emission: > > This patch adds required infrastructure to generate bitcode files and > uses postgresql-extension-uninstalled.pc to get include paths for > generating bitcode files [1]. > ---------- > > v1-0006-meson-Generate-bitcode-files-of-contrib-extension.patch: > > This patch adds manually selected contrib libraries to generate their > bitcode files. These libraries are selected manually, depending on > - If they have SQL callable functions > - If the library functions are short enough (the performance gain from > bitcode files is too minimal compared to the function's run time, so > this type of libraries are omitted). > > Any kind of feedback would be appreciated. > > -- > Regards, > Nazir Bilal Yavuz > Microsoft > Attachments: [text/x-patch] v2-0001-meson-Add-LLVM-bitcode-emission-for-generated-source.patch (6.7K, 3-v2-0001-meson-Add-LLVM-bitcode-emission-for-generated-source.patch) download | inline diff: From be09b1f38d107789fc9ffe1cd2b2470552689a20 Mon Sep 17 00:00:00 2001 From: Diego Fronza <[email protected]> Date: Mon, 10 Mar 2025 18:19:49 -0300 Subject: [PATCH] meson: Add LLVM bitcode emission for generated sources This commit adds suport for bitcode emission for generated source files (processed by bison, flex, etc). Since generated sources are created with custom_target, we must handle the source files differently when iterating over the list, as fs.parent expects str or file object. The way to handle it is cast the object as string `@[email protected](src)` then check if it's a custom target (a string starting with `<CustomTarget`), if that's the case we can safely call full_path() on it. But we also want to generate bitcode files with path separators replaced by underscore, relative to their source location, for that we strip meson.build_root() from it. Since generated backend sources may have their own compilation flags and must also be included in the postgres.index.bc, the way to make it work with current code was to create a new variable, called `bc_generated_backend_sources`, which is a list of dictionaries, each one having an optional 'additional_flags' and a `srclist` pointing to the list of custom_target generated sources. This list then has one dictionary entry for each generated backend sources which share a separate compilation flag. An example of a possible structure of bitcode_modules which is processed by the main meson llvm bitcode emission file src/backend/jit/llvm/bitcode/meson.build: ``` bitcode_modules = [ { 'name': 'postgres', 'target': postgres_lib, 'src_file': backend_sources, 'gen_srcfiles': [ { 'additional_flags': [ '-I/path/postgresl/src/backend/parser', '-I/path/postgresl/build/src/backend/parser', ], 'srcfiles': [ <custom_target for scan.c>, <custom_target for gram.c> ] } ] } ] ``` --- contrib/cube/meson.build | 2 ++ src/backend/jit/llvm/bitcode/meson.build | 37 ++++++++++++++++++++---- src/backend/meson.build | 2 ++ src/backend/parser/meson.build | 8 +++++ src/backend/utils/fmgr/meson.build | 1 + 5 files changed, 44 insertions(+), 6 deletions(-) diff --git a/contrib/cube/meson.build b/contrib/cube/meson.build index 52bd6a9fb8..bbc03dd319 100644 --- a/contrib/cube/meson.build +++ b/contrib/cube/meson.build @@ -12,6 +12,7 @@ cube_scan = custom_target('cubescan', ) generated_sources += cube_scan cube_sources += cube_scan +bc_cube_sources += [cube_scan] cube_parse = custom_target('cubeparse', input: 'cubeparse.y', @@ -19,6 +20,7 @@ cube_parse = custom_target('cubeparse', ) generated_sources += cube_parse.to_list() cube_sources += cube_parse +bc_cube_sources += cube_parse[0] if host_system == 'windows' cube_sources += rc_lib_gen.process(win32ver_rc, extra_args: [ diff --git a/src/backend/jit/llvm/bitcode/meson.build b/src/backend/jit/llvm/bitcode/meson.build index 577b4ddb21..acc13a1d16 100644 --- a/src/backend/jit/llvm/bitcode/meson.build +++ b/src/backend/jit/llvm/bitcode/meson.build @@ -9,18 +9,18 @@ foreach bitcode_module : bitcode_modules bitcode_obj = bitcode_module['target'] bitcode_cflags_local = bitcode_cflags + bitcode_module.get('additional_flags', []) - if 'name' not in bitcode_module - bitcode_name = bitcode_obj.name() - else - bitcode_name = bitcode_module['name'] - endif + bitcode_name = bitcode_module.get('name', bitcode_obj.name()) foreach srcfile : bitcode_module['srcfiles'] - if meson.version().version_compare('>=0.59') + srcfilename = '@0@'.format(srcfile) + if srcfilename.startswith('<CustomTarget') + srcfilename = srcfile.full_path().split(meson.build_root() + '/')[1] + elif meson.version().version_compare('>=0.59') srcfilename = fs.parent(srcfile) / fs.name(srcfile) else srcfilename = '@0@'.format(srcfile) endif + targetname = '@0@_@[email protected]'.format( bitcode_name, srcfilename.underscorify(), @@ -36,6 +36,31 @@ foreach bitcode_module : bitcode_modules ) endforeach + # process generated sources, which may include custom compilation flags. + foreach gen_srcfiles: bitcode_module.get('gen_srcfiles', []) + bitcode_cflags_gen_local = bitcode_cflags_local + gen_srcfiles.get('additional_flags', []) + + foreach srcfile: gen_srcfiles['srcfiles'] + # generated sources are stored in some folder under meson.build_root()/** + # remove the build prefix from the string. + srcfilename = srcfile.full_path().split(meson.build_root() + '/')[1] + + targetname = '@0@_@[email protected]'.format( + bitcode_name, + srcfilename.underscorify(), + ) + bitcode_targets += custom_target( + targetname, + depends: [bitcode_module['target']], + input: [srcfile], + output: targetname, + command: [llvm_irgen_command, bitcode_cflags_gen_local], + install: true, + install_dir: dir_bitcode, + ) + endforeach + endforeach + index_name = '@[email protected]'.format(bitcode_name) bitcode_index = custom_target('@0@'.format(bitcode_name), output: index_name, diff --git a/src/backend/meson.build b/src/backend/meson.build index 5fb33660d3..76e5939fec 100644 --- a/src/backend/meson.build +++ b/src/backend/meson.build @@ -5,6 +5,7 @@ backend_sources = [] backend_link_with = [pgport_srv, common_srv] generated_backend_sources = [] +bc_generated_backend_sources = [] post_export_backend_sources = [] subdir('access') @@ -144,6 +145,7 @@ bitcode_modules += { 'name': 'postgres', 'target': postgres_lib, 'srcfiles': backend_sources, + 'gen_srcfiles': bc_generated_backend_sources, } pg_mod_c_args = cflags_mod diff --git a/src/backend/parser/meson.build b/src/backend/parser/meson.build index 874aa749aa..add472a0cd 100644 --- a/src/backend/parser/meson.build +++ b/src/backend/parser/meson.build @@ -42,6 +42,14 @@ backend_parser = custom_target('gram', generated_sources += backend_parser.to_list() parser_sources += backend_parser +bc_generated_backend_sources += { + 'additional_flags': [ + '-I@0@'.format(meson.current_build_dir()), + '-I@0@'.format(meson.current_source_dir()), + ], + 'srcfiles': [backend_scanner, backend_parser[0]], +} + parser = static_library('parser', parser_sources, dependencies: [backend_code], diff --git a/src/backend/utils/fmgr/meson.build b/src/backend/utils/fmgr/meson.build index b1dcab93e7..0119bc40cf 100644 --- a/src/backend/utils/fmgr/meson.build +++ b/src/backend/utils/fmgr/meson.build @@ -8,3 +8,4 @@ backend_sources += files( # fmgrtab.c generated_backend_sources += fmgrtab_target[2] +bc_generated_backend_sources += {'srcfiles': [fmgrtab_target[2]]} \ No newline at end of file -- 2.43.0 ^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: meson vs. llvm bitcode files @ 2025-03-12 10:26 Nazir Bilal Yavuz <[email protected]> parent: Diego Fronza <[email protected]> 0 siblings, 1 reply; 10+ messages in thread From: Nazir Bilal Yavuz @ 2025-03-12 10:26 UTC (permalink / raw) To: Diego Fronza <[email protected]>; +Cc: Peter Eisentraut <[email protected]>; pgsql-hackers; Andres Freund <[email protected]> Hi, On Tue, 11 Mar 2025 at 01:04, Diego Fronza <[email protected]> wrote: > I did a full review on the provided patches plus some tests, I was able to validate that the loading of bitcode modules is working also JIT works for both backend and contrib modules. Thank you! > To test JIT on contrib modules I just lowered the costs for all jit settings and used the intarray extension, using the data/test__int.data: > CREATE EXTENSION intarray; > CREATE TABLE test__int( a int[] );1 > \copy test__int from 'data/test__int.data' > > For queries any from line 98+ on contrib/intarray/sql/_int.sql will work. > > Then I added extra debug messages to llvmjit_inline.cpp on add_module_to_inline_search_path() function, also on llvm_build_inline_plan(), I was able to see many functions in this module being successfully inlined. > > I'm attaching a new patch based on your original work which add further support for generating bitcode from: Thanks for doing that! > - Generated backend sources: processed by flex, bison, etc. > - Generated contrib module sources, I think we do not need to separate these two. foreach srcfile : bitcode_module['srcfiles'] - if meson.version().version_compare('>=0.59') + srcfilename = '@0@'.format(srcfile) + if srcfilename.startswith('<CustomTarget') + srcfilename = srcfile.full_path().split(meson.build_root() + '/')[1] + elif meson.version().version_compare('>=0.59') Also, checking if the string starts with '<CustomTarget' is a bit hacky, and 'srcfilename = '@0@'.format(srcfile)' causes a deprecation warning. So, instead of this we can process all generated sources like how generated backend sources are processed. I updated the patch with that. > On this patch I just included fmgrtab.c and src/backend/parser for the backend generated code. > For contrib generated sources I added contrib/cube as an example. I applied your contrib/cube example and did the same thing for the contrib/seg. > All relevant details about the changes are included in the patch itself. > > As you may know already I also created a PR focused on llvm bitcode emission on meson, it generates bitcode for all backend and contribution modules, currently under review by some colleagues at Percona: https://github.com/percona/postgres/pull/103 > I'm curious if we should get all or some of the generated backend sources compiled to bitcode, similar to contrib modules. I think we can do this. I added other backend sources like you did in the PR but attached it as another patch (0007) because I wanted to hear other people's opinions on that first. v3 is attached. -- Regards, Nazir Bilal Yavuz Microsoft Attachments: [text/x-patch] v3-0001-meson-Add-generated-header-stamps.patch (5.0K, 2-v3-0001-meson-Add-generated-header-stamps.patch) download | inline diff: From 62c7c05b19476939941b656a6eab43dc3661ef09 Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Thu, 24 Oct 2024 07:23:05 -0400 Subject: [PATCH v3 1/7] meson: Add generated header stamps Otherwise build commands become too long and this has visible effect on creation time of meson build files. Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/include/meson.build | 18 ++++++++++++++++++ src/backend/meson.build | 2 +- src/fe_utils/meson.build | 2 +- meson.build | 16 +++++++++------- 4 files changed, 29 insertions(+), 9 deletions(-) diff --git a/src/include/meson.build b/src/include/meson.build index 2e4b7aa529e..c488a5dc4c9 100644 --- a/src/include/meson.build +++ b/src/include/meson.build @@ -177,3 +177,21 @@ install_subdir('catalog', # autoconf generates the file there, ensure we get a conflict generated_sources_ac += {'src/include': ['stamp-h']} + +# Instead of having targets depending directly on the generated headers, have +# them depend on a stamp files for all of them. Dependencies on headers are +# implemented as order-only dependencies in meson (later using compiler +# generated dependencies). The benefit of using a stamp file is that it makes +# ninja.build smaller and meson setup faster. +generated_headers_stamp = custom_target('generated-headers-stamp.h', + output: 'generated-headers-stamp.h', + input: generated_headers, + command: stamp_cmd, +) + +generated_backend_headers_stamp = custom_target('generated-backend-headers-stamp.h', + output: 'generated-backend-headers-stamp.h', + input: generated_backend_headers, + depends: generated_headers_stamp, + command: stamp_cmd, +) diff --git a/src/backend/meson.build b/src/backend/meson.build index 2b0db214804..7fc649c3ebd 100644 --- a/src/backend/meson.build +++ b/src/backend/meson.build @@ -169,7 +169,7 @@ backend_mod_code = declare_dependency( compile_args: pg_mod_c_args, include_directories: postgres_inc, link_args: pg_mod_link_args, - sources: generated_headers + generated_backend_headers, + sources: [generated_backend_headers_stamp], dependencies: backend_mod_deps, ) diff --git a/src/fe_utils/meson.build b/src/fe_utils/meson.build index a18cbc939e4..5a9ddb73463 100644 --- a/src/fe_utils/meson.build +++ b/src/fe_utils/meson.build @@ -29,7 +29,7 @@ generated_sources += psqlscan fe_utils_sources += psqlscan fe_utils = static_library('libpgfeutils', - fe_utils_sources + generated_headers, + fe_utils_sources, c_pch: pch_postgres_fe_h, include_directories: [postgres_inc, libpq_inc], c_args: host_system == 'windows' ? ['-DFD_SETSIZE=1024'] : [], diff --git a/meson.build b/meson.build index 13c13748e5d..9bfa96ad255 100644 --- a/meson.build +++ b/meson.build @@ -2973,6 +2973,8 @@ gen_export_kwargs = { 'install': false, } +# command to create stamp files on all OSs +stamp_cmd = [python, '-c', 'import sys; open(sys.argv[1], "w")', '@OUTPUT0@'] ### @@ -3090,14 +3092,14 @@ subdir('src/port') frontend_common_code = declare_dependency( compile_args: ['-DFRONTEND'], include_directories: [postgres_inc], - sources: generated_headers, + sources: generated_headers_stamp, dependencies: [os_deps, zlib, zstd, lz4], ) backend_common_code = declare_dependency( compile_args: ['-DBUILDING_DLL'], include_directories: [postgres_inc], - sources: generated_headers, + sources: generated_headers_stamp, dependencies: [os_deps, zlib, zstd], ) @@ -3112,7 +3114,7 @@ shlib_code = declare_dependency( frontend_stlib_code = declare_dependency( include_directories: [postgres_inc], link_with: [common_static, pgport_static], - sources: generated_headers, + sources: generated_headers_stamp, dependencies: [os_deps, libintl], ) @@ -3120,7 +3122,7 @@ frontend_stlib_code = declare_dependency( frontend_shlib_code = declare_dependency( include_directories: [postgres_inc], link_with: [common_shlib, pgport_shlib], - sources: generated_headers, + sources: generated_headers_stamp, dependencies: [shlib_code, os_deps, libintl], ) @@ -3130,7 +3132,7 @@ frontend_shlib_code = declare_dependency( frontend_no_fe_utils_code = declare_dependency( include_directories: [postgres_inc], link_with: [common_static, pgport_static], - sources: generated_headers, + sources: generated_headers_stamp, dependencies: [os_deps, libintl], ) @@ -3156,7 +3158,7 @@ subdir('src/fe_utils') frontend_code = declare_dependency( include_directories: [postgres_inc], link_with: [fe_utils, common_static, pgport_static], - sources: generated_headers, + sources: generated_headers_stamp, dependencies: [os_deps, libintl], ) @@ -3184,7 +3186,7 @@ backend_code = declare_dependency( include_directories: [postgres_inc], link_args: ldflags_be, link_with: [], - sources: generated_headers + generated_backend_headers, + sources: [generated_backend_headers_stamp], dependencies: os_deps + backend_both_deps + backend_deps, ) -- 2.47.2 [text/x-patch] v3-0002-meson-Add-postgresql-extension.pc-for-building-ex.patch (4.5K, 3-v3-0002-meson-Add-postgresql-extension.pc-for-building-ex.patch) download | inline diff: From bcca7523a9e8eec124b9422978f66c903cfb5fe6 Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Sat, 27 Aug 2022 09:52:03 -0700 Subject: [PATCH v3 2/7] meson: Add postgresql-extension.pc for building extension libraries This should work with several other buildsystems. TODO: Docs Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/backend/meson.build | 110 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 110 insertions(+) diff --git a/src/backend/meson.build b/src/backend/meson.build index 7fc649c3ebd..9d79d4d058c 100644 --- a/src/backend/meson.build +++ b/src/backend/meson.build @@ -188,6 +188,116 @@ pg_test_mod_args = pg_mod_args + { +############################################################### +# Define a .pc file that can be used to build server extensions +############################################################### + +pg_ext_vars = [] +pg_ext_vars_inst = [] +pg_ext_vars_uninst = [] + +pg_ext_cflags = pg_mod_c_args + cppflags +pg_ext_libs = [backend_mod_deps, thread_dep, ldflags, ldflags_mod] +pg_ext_subdirs = [''] + +# Compute directories to add include directories to the .pc files for. +# This is a bit more complicated due to port/win32 etc. +i = 0 +foreach incdir : postgres_inc_d + if fs.is_absolute(incdir) + # an absolute path from -Dextra_include_dirs + pg_ext_cflags += '-I@0@'.format(incdir) + continue + elif incdir.startswith('src/include') + subincdir = dir_include_pkg_rel / 'server' / incdir.split('src/include/').get(1, '') + else + subincdir = '' + endif + pg_ext_subdirs += subincdir + + # Add directories in source / build dir containing headers to cflags for the + # -uninstalled.pc. Older versions of pkg-config complain if a referenced + # variable is not defined, so we emit an empty one for the installed .pc + # file. + pg_ext_vars += [ + 'build_inc@0@=""'.format(i), + 'src_inc@0@=""'.format(i), + ] + pg_ext_vars_uninst += [ + 'build_inc@0@=-I${prefix}/@1@'.format(i, incdir), + 'src_inc@0@=-I${srcdir}/@1@'.format(i, incdir), + ] + pg_ext_cflags += [ + '${build_inc@0@}'.format(i), + '${src_inc@0@}'.format(i) + ] + + i += 1 +endforeach + + +# Extension modules should likely also use -fwrapv etc. But it it's a bit odd +# to expose it to a .pc file? +pg_ext_cflags_warn = pg_ext_cflags + cflags_warn +pg_ext_cflags += cflags + +# Directories for extensions to install into +# XXX: more needed +pg_ext_vars += 'pkglibdir=${prefix}/@0@'.format(dir_lib_pkg) +pg_ext_vars += 'dir_mod=${pkglibdir}' +pg_ext_vars += 'dir_data=${prefix}/@0@'.format(dir_data_extension) +pg_ext_vars += 'dir_include=${prefix}/@0@'.format(dir_include_extension) +pg_ext_vars += 'dir_doc=${prefix}/@0@'.format(dir_doc_extension) +pg_ext_vars += 'dir_bitcode=${prefix}/@0@'.format(dir_bitcode) +# referenced on some platforms, via mod_link_with_dir +pg_ext_vars += 'bindir=${prefix}/@0@'.format(dir_bin) + +# XXX: Define variables making it easy to define tests, too + +# Some platforms need linker flags to link with binary, they are the same +# between building with meson and .pc file, except that we have have to +# reference a variable to make it work for both normal and -uninstalled .pc +# files. +if mod_link_args_fmt.length() != 0 + assert(link_with_inst != '') + assert(link_with_uninst != '') + + pg_ext_vars_inst += 'mod_link_with=@0@'.format(link_with_inst) + pg_ext_vars_uninst += 'mod_link_with=@0@'.format(link_with_uninst) + + foreach el : mod_link_args_fmt + pg_ext_libs += el.format('${mod_link_with}') + endforeach +endif + +# main .pc to build extensions +pkgconfig.generate( + name: 'postgresql-extension', + description: 'PostgreSQL Extension Support', + url: pg_url, + + subdirs: pg_ext_subdirs, + libraries: pg_ext_libs, + extra_cflags: pg_ext_cflags, + + variables: pg_ext_vars + pg_ext_vars_inst, + uninstalled_variables: pg_ext_vars + pg_ext_vars_uninst, +) + +# a .pc depending on the above, but with all our warnings enabled +pkgconfig.generate( + name: 'postgresql-extension-warnings', + description: 'PostgreSQL Extension Support - Compiler Warnings', + requires: 'postgresql-extension', + url: pg_url, + extra_cflags: pg_ext_cflags_warn, + + variables: pg_ext_vars + pg_ext_vars_inst, + uninstalled_variables: pg_ext_vars + pg_ext_vars_uninst, +) + + + # Shared modules that, on some system, link against the server binary. Only # enter these after we defined the server build. -- 2.47.2 [text/x-patch] v3-0003-meson-Test-building-extensions-by-using-postgresq.patch (10.7K, 4-v3-0003-meson-Test-building-extensions-by-using-postgresq.patch) download | inline diff: From d19d35d7106b6b9ec5038ab4ef2c1d8fc967cecc Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Thu, 27 Feb 2025 17:45:31 +0300 Subject: [PATCH v3 3/7] meson: Test building extensions by using postgresql-extension.pc The 'test_meson_extensions' pyton wrapper is added to run these tests. It compiles and builds extensions at ${build}/testrun/meson_extensions/${extension_name} path. The tests for building amcheck, auth_delay and postgres_fdw extensions are added. These are also examples of how to build extensions by using postgresql-extension.pc. Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/test/modules/meson.build | 1 + .../modules/test_meson_extensions/meson.build | 3 + .../amcheck/meson.build | 28 ++++++++ .../auth_delay/meson.build | 17 +++++ .../test_pkg_config_extensions/meson.build | 24 +++++++ .../postgres_fdw/meson.build | 31 ++++++++ meson.build | 32 +++++++++ src/tools/ci/test_meson_extensions | 70 +++++++++++++++++++ 8 files changed, 206 insertions(+) create mode 100644 src/test/modules/test_meson_extensions/meson.build create mode 100644 src/test/modules/test_meson_extensions/test_pkg_config_extensions/amcheck/meson.build create mode 100644 src/test/modules/test_meson_extensions/test_pkg_config_extensions/auth_delay/meson.build create mode 100644 src/test/modules/test_meson_extensions/test_pkg_config_extensions/meson.build create mode 100644 src/test/modules/test_meson_extensions/test_pkg_config_extensions/postgres_fdw/meson.build create mode 100644 src/tools/ci/test_meson_extensions diff --git a/src/test/modules/meson.build b/src/test/modules/meson.build index 2b057451473..22ec5e969ae 100644 --- a/src/test/modules/meson.build +++ b/src/test/modules/meson.build @@ -25,6 +25,7 @@ subdir('test_ginpostinglist') subdir('test_integerset') subdir('test_json_parser') subdir('test_lfind') +subdir('test_meson_extensions') subdir('test_misc') subdir('test_oat_hooks') subdir('test_parser') diff --git a/src/test/modules/test_meson_extensions/meson.build b/src/test/modules/test_meson_extensions/meson.build new file mode 100644 index 00000000000..06cf5d555df --- /dev/null +++ b/src/test/modules/test_meson_extensions/meson.build @@ -0,0 +1,3 @@ +# Copyright (c) 2022-2025, PostgreSQL Global Development Group + +subdir('test_pkg_config_extensions') diff --git a/src/test/modules/test_meson_extensions/test_pkg_config_extensions/amcheck/meson.build b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/amcheck/meson.build new file mode 100644 index 00000000000..482a543eb86 --- /dev/null +++ b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/amcheck/meson.build @@ -0,0 +1,28 @@ +# Copyright (c) 2022-2025, PostgreSQL Global Development Group + +project('amcheck', 'c') + +amcheck_path = '../../../../../../contrib/amcheck/' + +amcheck_sources = files( + amcheck_path / 'verify_heapam.c', + amcheck_path / 'verify_nbtree.c', +) + +pg_ext = dependency('postgresql-extension-warnings') + +amcheck = shared_module('amcheck', + amcheck_sources, + dependencies: pg_ext, + install_dir: pg_ext.get_variable(pkgconfig: 'dir_mod'), +) + +install_data( + amcheck_path / 'amcheck.control', + amcheck_path / 'amcheck--1.0.sql', + amcheck_path / 'amcheck--1.0--1.1.sql', + amcheck_path / 'amcheck--1.1--1.2.sql', + amcheck_path / 'amcheck--1.2--1.3.sql', + amcheck_path / 'amcheck--1.3--1.4.sql', + install_dir: pg_ext.get_variable(pkgconfig: 'dir_data'), +) diff --git a/src/test/modules/test_meson_extensions/test_pkg_config_extensions/auth_delay/meson.build b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/auth_delay/meson.build new file mode 100644 index 00000000000..98ad24cc183 --- /dev/null +++ b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/auth_delay/meson.build @@ -0,0 +1,17 @@ +# Copyright (c) 2022-2025, PostgreSQL Global Development Group + +project('auth_delay', 'c') + +auth_delay_path = '../../../../../../contrib/auth_delay/' + +auth_delay_sources = files( + auth_delay_path / 'auth_delay.c', +) + +pg_ext = dependency('postgresql-extension-warnings') + +auth_delay = shared_module('auth_delay', + auth_delay_sources, + dependencies: pg_ext, + install_dir: pg_ext.get_variable(pkgconfig: 'dir_mod'), +) diff --git a/src/test/modules/test_meson_extensions/test_pkg_config_extensions/meson.build b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/meson.build new file mode 100644 index 00000000000..dae94a384a1 --- /dev/null +++ b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/meson.build @@ -0,0 +1,24 @@ +# Copyright (c) 2022-2025, PostgreSQL Global Development Group + +# pkgconfig is not available on Windows, so skip it. +if host_machine.system() == 'windows' + subdir_done() +endif + +meson_extension_tests += { + 'name': 'amcheck', + 'kind': 'pkg_config', + 'sd': meson.current_source_dir() / 'amcheck', +} + +meson_extension_tests += { + 'name': 'auth_delay', + 'kind': 'pkg_config', + 'sd': meson.current_source_dir() / 'auth_delay', +} + +meson_extension_tests += { + 'name': 'postgres_fdw', + 'kind': 'pkg_config', + 'sd': meson.current_source_dir() / 'postgres_fdw', +} diff --git a/src/test/modules/test_meson_extensions/test_pkg_config_extensions/postgres_fdw/meson.build b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/postgres_fdw/meson.build new file mode 100644 index 00000000000..d49625e0611 --- /dev/null +++ b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/postgres_fdw/meson.build @@ -0,0 +1,31 @@ +# Copyright (c) 2022-2025, PostgreSQL Global Development Group + +project('auth_delay', 'c') + +postgres_fdw_path = '../../../../../../contrib/postgres_fdw/' + +postgres_fdw_sources = files( + postgres_fdw_path / 'connection.c', + postgres_fdw_path / 'deparse.c', + postgres_fdw_path / 'option.c', + postgres_fdw_path / 'postgres_fdw.c', + postgres_fdw_path / 'shippable.c', +) + +pg_ext = dependency('postgresql-extension-warnings') +libpq = dependency('libpq') + +postgres_fdw = shared_module('postgres_fdw', + postgres_fdw_sources, + dependencies: [pg_ext, libpq], + install_dir: pg_ext.get_variable(pkgconfig: 'dir_mod'), +) + +install_data( + postgres_fdw_path / 'postgres_fdw.control', + postgres_fdw_path / 'postgres_fdw--1.0.sql', + postgres_fdw_path / 'postgres_fdw--1.0--1.1.sql', + postgres_fdw_path / 'postgres_fdw--1.1--1.2.sql', + install_dir: pg_ext.get_variable(pkgconfig: 'dir_data'), +) + diff --git a/meson.build b/meson.build index 9bfa96ad255..765b7f6a207 100644 --- a/meson.build +++ b/meson.build @@ -2891,6 +2891,7 @@ nls_targets = [] # Define the tests to distribute them to the correct test styles later test_deps = [] tests = [] +meson_extension_tests = [] # Default options for targets @@ -3443,6 +3444,37 @@ sys.exit(sp.returncode) suite: ['setup']) +# it seems freebsd doesn't use libdir for pkgconfig path +if host_system == 'freebsd' + pkgconf_installdir = dir_prefix / 'libdata' / 'pkgconfig' +else + pkgconf_installdir = dir_prefix / dir_lib / 'pkgconfig' +endif +test_pkg_conf_file = files('src/tools/ci/test_meson_extensions') + +foreach test : meson_extension_tests + if test['kind'] not in ['pkg_config'] + error('unknown kind @0@ of test in @1@'.format(test['kind'], test['sd'])) + endif + + test_group = 'meson_@0@_extensions'.format(test['kind']) + + test(test_group / test['name'], + test_pkg_conf_file, + args: [ + '--meson', meson_bin.path(), + '--meson_args', meson_args, + '--test_dir', test['sd'], + '--test_out_dir', test_result_dir / 'meson_extensions' / test['name'], + '--builddir', meson.build_root(), + '--pkg_conf_path', get_option('pkg_config_path'), + '--', + cc.cmd_array(), + ], + suite: test_group, + ) + +endforeach ############################################################### # Test Generation diff --git a/src/tools/ci/test_meson_extensions b/src/tools/ci/test_meson_extensions new file mode 100644 index 00000000000..50358121f49 --- /dev/null +++ b/src/tools/ci/test_meson_extensions @@ -0,0 +1,70 @@ +#!/usr/bin/env python3 + +import argparse +import os +import shutil +import subprocess + +parser = argparse.ArgumentParser() + +parser.add_argument('--meson', help='path to meson binary', + type=str, required=True) +parser.add_argument('--meson_args', help='args of meson binary', + type=str, nargs='*', required=False) +parser.add_argument('--test_dir', help='test source directory', + type=str, required=True) +parser.add_argument('--test_out_dir', help='test output directory', + type=str, required=True) +parser.add_argument('--builddir', help='meson build directory', + type=str, required=True) +parser.add_argument('--pkg_conf_path', + help='PKG_CONF_PATH from surrounding meson build', + type=str, nargs='?', const='', required=False) +parser.add_argument('c_args', help='c_args from surrounding meson build', + nargs='*') + +args = parser.parse_args() + +meson_bin = args.meson +meson_args = ' '.join(args.meson_args) +test_source_dir = args.test_dir +test_out_dir = args.test_out_dir +build_dir = args.builddir +pkg_conf_path = args.pkg_conf_path +c_args = ' '.join(args.c_args) + +exit_code = 0 + +def remove_duplicates(duplicate_str): + words = duplicate_str.split() + return ' '.join(sorted(set(words), key=words.index)) + + +def run_tests(pkg_conf_path_local, message): + print('\n{}\n{}\n'.format('#' * 60, message), flush=True) + + env = {**os.environ, } + env['PKG_CONFIG_PATH'] = '{}:{}:{}'.format( + pkg_conf_path_local, pkg_conf_path, env.get('PKG_CONFIG_PATH', ''), + ).strip(': ') + env['CC'] = '{} {}'.format( + c_args, env.get('CC', ''), + ) + env['CC'] = remove_duplicates(env['CC']) + + # Clear the build directory beforehand. + if os.path.exists(test_out_dir): + shutil.rmtree(test_out_dir) + + if meson_args: + meson_setup_command = [meson_bin, meson_args, 'setup', test_out_dir] + else: + meson_setup_command = [meson_bin, 'setup', test_out_dir] + + meson_compile_command = ['meson', 'compile', '-C', test_out_dir, '-v'] + + subprocess.run(meson_setup_command, env=env, cwd=test_source_dir, check=True) + subprocess.run(meson_compile_command, cwd=test_source_dir, check=True) + +run_tests(os.path.join(build_dir, 'meson-uninstalled'), + message='Testing postgresql-extension-warnings-uninstalled') -- 2.47.2 [text/x-patch] v3-0004-meson-WIP-Add-docs-for-postgresql-extension.pc.patch (12.3K, 5-v3-0004-meson-WIP-Add-docs-for-postgresql-extension.pc.patch) download | inline diff: From 4efd02d85ecb67ac9a66ca34b9b501931e48c576 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Thu, 6 Mar 2025 17:46:57 +0300 Subject: [PATCH v3 4/7] meson: [WIP] Add docs for postgresql-extension.pc Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- doc/src/sgml/acronyms.sgml | 2 +- doc/src/sgml/extend.sgml | 101 +++++++++++++++++++++++-------------- doc/src/sgml/jit.sgml | 2 +- 3 files changed, 66 insertions(+), 39 deletions(-) diff --git a/doc/src/sgml/acronyms.sgml b/doc/src/sgml/acronyms.sgml index 58d0d90fece..240db5cb827 100644 --- a/doc/src/sgml/acronyms.sgml +++ b/doc/src/sgml/acronyms.sgml @@ -561,7 +561,7 @@ <term><acronym>PGXS</acronym></term> <listitem> <para> - <link linkend="extend-pgxs"><productname>PostgreSQL</productname> Extension System</link> + <link linkend="extend-postgres-pgxs"><productname>PostgreSQL</productname> Extension System</link> </para> </listitem> </varlistentry> diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml index ba492ca27c0..4a8c19b3f53 100644 --- a/doc/src/sgml/extend.sgml +++ b/doc/src/sgml/extend.sgml @@ -1421,7 +1421,7 @@ include $(PGXS) </programlisting> This makefile relies on <acronym>PGXS</acronym>, which is described - in <xref linkend="extend-pgxs"/>. The command <literal>make install</literal> + in <xref linkend="extend-postgres-pgxs"/>. The command <literal>make install</literal> will install the control and script files into the correct directory as reported by <application>pg_config</application>. </para> @@ -1434,21 +1434,26 @@ include $(PGXS) </sect2> </sect1> - <sect1 id="extend-pgxs"> + <sect1 id="extend-postgres"> <title>Extension Building Infrastructure</title> - <indexterm zone="extend-pgxs"> - <primary>pgxs</primary> - </indexterm> - <para> If you are thinking about distributing your <productname>PostgreSQL</productname> extension modules, setting up a portable build system for them can be fairly difficult. Therefore the <productname>PostgreSQL</productname> installation provides a build - infrastructure for extensions, called <acronym>PGXS</acronym>, so - that simple extension modules can be built simply against an - already installed server. <acronym>PGXS</acronym> is mainly intended + infrastructure for extensions, called <literal>PGXS</literal> + (<xref linkend="extend-postgres-pgxs"/>) and + its meson counterpart <literal>postgresql-extension.pc</literal> + (<xref linkend="extend-postgres-meson"/>). + </para> + + </sect1> + + <sect1 id="extend-postgres-pgxs"> + <title>PGXS</title> + + <para> <acronym>PGXS</acronym> is mainly intended for extensions that include C code, although it can be used for pure-SQL extensions too. Note that <acronym>PGXS</acronym> is not intended to be a universal build system framework that can be used @@ -1488,7 +1493,7 @@ include $(PGXS) Set one of these three variables to specify what is built: <variablelist> - <varlistentry id="extend-pgxs-modules"> + <varlistentry id="extend-postgres-pgxs-modules"> <term><varname>MODULES</varname></term> <listitem> <para> @@ -1498,7 +1503,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-module-big"> + <varlistentry id="extend-postgres-pgxs-module-big"> <term><varname>MODULE_big</varname></term> <listitem> <para> @@ -1508,7 +1513,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-program"> + <varlistentry id="extend-postgres-pgxs-program"> <term><varname>PROGRAM</varname></term> <listitem> <para> @@ -1522,7 +1527,7 @@ include $(PGXS) The following variables can also be set: <variablelist> - <varlistentry id="extend-pgxs-extension"> + <varlistentry id="extend-postgres-pgxs-extension"> <term><varname>EXTENSION</varname></term> <listitem> <para> @@ -1534,7 +1539,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-moduledir"> + <varlistentry id="extend-postgres-pgxs-moduledir"> <term><varname>MODULEDIR</varname></term> <listitem> <para> @@ -1547,7 +1552,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-data"> + <varlistentry id="extend-postgres-pgxs-data"> <term><varname>DATA</varname></term> <listitem> <para> @@ -1556,7 +1561,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-data-built"> + <varlistentry id="extend-postgres-pgxs-data-built"> <term><varname>DATA_built</varname></term> <listitem> <para> @@ -1567,7 +1572,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-data-tsearch"> + <varlistentry id="extend-postgres-pgxs-data-tsearch"> <term><varname>DATA_TSEARCH</varname></term> <listitem> <para> @@ -1577,7 +1582,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-docs"> + <varlistentry id="extend-postgres-pgxs-docs"> <term><varname>DOCS</varname></term> <listitem> <para> @@ -1587,7 +1592,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-headers"> + <varlistentry id="extend-postgres-pgxs-headers"> <term><varname>HEADERS</varname></term> <term><varname>HEADERS_built</varname></term> <listitem> @@ -1603,7 +1608,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-headers-module"> + <varlistentry id="extend-postgres-pgxs-headers-module"> <term><varname>HEADERS_$MODULE</varname></term> <term><varname>HEADERS_built_$MODULE</varname></term> <listitem> @@ -1629,7 +1634,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-scripts"> + <varlistentry id="extend-postgres-pgxs-scripts"> <term><varname>SCRIPTS</varname></term> <listitem> <para> @@ -1639,7 +1644,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-scripts-built"> + <varlistentry id="extend-postgres-pgxs-scripts-built"> <term><varname>SCRIPTS_built</varname></term> <listitem> <para> @@ -1650,7 +1655,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-regress"> + <varlistentry id="extend-postgres-pgxs-regress"> <term><varname>REGRESS</varname></term> <listitem> <para> @@ -1659,7 +1664,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-regress-opts"> + <varlistentry id="extend-postgres-pgxs-regress-opts"> <term><varname>REGRESS_OPTS</varname></term> <listitem> <para> @@ -1668,7 +1673,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-isolation"> + <varlistentry id="extend-postgres-pgxs-isolation"> <term><varname>ISOLATION</varname></term> <listitem> <para> @@ -1677,7 +1682,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-isolation-opts"> + <varlistentry id="extend-postgres-pgxs-isolation-opts"> <term><varname>ISOLATION_OPTS</varname></term> <listitem> <para> @@ -1687,7 +1692,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-tap-tests"> + <varlistentry id="extend-postgres-pgxs-tap-tests"> <term><varname>TAP_TESTS</varname></term> <listitem> <para> @@ -1696,7 +1701,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-no-install"> + <varlistentry id="extend-postgres-pgxs-no-install"> <term><varname>NO_INSTALL</varname></term> <listitem> <para> @@ -1706,7 +1711,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-no-installcheck"> + <varlistentry id="extend-postgres-pgxs-no-installcheck"> <term><varname>NO_INSTALLCHECK</varname></term> <listitem> <para> @@ -1715,7 +1720,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-extra-clean"> + <varlistentry id="extend-postgres-pgxs-extra-clean"> <term><varname>EXTRA_CLEAN</varname></term> <listitem> <para> @@ -1724,7 +1729,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-cppflags"> + <varlistentry id="extend-postgres-pgxs-pg-cppflags"> <term><varname>PG_CPPFLAGS</varname></term> <listitem> <para> @@ -1733,7 +1738,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-cflags"> + <varlistentry id="extend-postgres-pgxs-pg-cflags"> <term><varname>PG_CFLAGS</varname></term> <listitem> <para> @@ -1742,7 +1747,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-cxxflags"> + <varlistentry id="extend-postgres-pgxs-pg-cxxflags"> <term><varname>PG_CXXFLAGS</varname></term> <listitem> <para> @@ -1751,7 +1756,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-ldflags"> + <varlistentry id="extend-postgres-pgxs-pg-ldflags"> <term><varname>PG_LDFLAGS</varname></term> <listitem> <para> @@ -1760,7 +1765,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-libs"> + <varlistentry id="extend-postgres-pgxs-pg-libs"> <term><varname>PG_LIBS</varname></term> <listitem> <para> @@ -1769,7 +1774,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-shlib-link"> + <varlistentry id="extend-postgres-pgxs-shlib-link"> <term><varname>SHLIB_LINK</varname></term> <listitem> <para> @@ -1778,7 +1783,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-config"> + <varlistentry id="extend-postgres-pgxs-pg-config"> <term><varname>PG_CONFIG</varname></term> <listitem> <para> @@ -1894,4 +1899,26 @@ make VPATH=/path/to/extension/source/tree install </tip> </sect1> + <sect1 id="extend-postgres-meson"> + <title>postgresql-extension.pc</title> + + <para> + When Postgres is built by using meson, it generates + <literal>postgresql-extension.pc</literal> pkg-config file. Extension + libraries can use this file like <literal>PGXS</literal> + (<xref linkend="extend-postgres-pgxs"/>). + + To use the <literal>postgresql-extension.pc</literal> infrastructure for + your extension, you must write a simple meson.build file. In the + meson.build file, you need to include the + <literal>postgresql-extension.pc</literal> pkg-config file. Here is an + example that builds an extension module named isbn_issn, consisting of a + shared library containing some C code, an extension control file, an SQL + script, an include file (only needed if other modules might need to access + the extension functions without going via SQL), and a documentation text + file: + </para> + + </sect1> + </chapter> diff --git a/doc/src/sgml/jit.sgml b/doc/src/sgml/jit.sgml index 44e18bf1a6f..81a4644a97d 100644 --- a/doc/src/sgml/jit.sgml +++ b/doc/src/sgml/jit.sgml @@ -223,7 +223,7 @@ SET of types <literal>C</literal> and <literal>internal</literal>, as well as operators based on such functions. To do so for functions in extensions, the definitions of those functions need to be made available. - When using <link linkend="extend-pgxs">PGXS</link> to build an extension + When using <link linkend="extend-postgres-pgxs">PGXS</link> to build an extension against a server that has been compiled with LLVM JIT support, the relevant files will be built and installed automatically. </para> -- 2.47.2 [text/x-patch] v3-0005-meson-Add-architecture-for-LLVM-bitcode-emission.patch (9.1K, 6-v3-0005-meson-Add-architecture-for-LLVM-bitcode-emission.patch) download | inline diff: From b9f4e945c0bd8381e0928106796707450106ec4e Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Fri, 7 Mar 2025 12:10:58 +0300 Subject: [PATCH v3 5/7] meson: Add architecture for LLVM bitcode emission This commit adds suport for bitcode emission for both normal and generated source files (processed by bison, flex, etc). These bitcode files are installed into $pkglibdir/bitcode/ directory if the LLVM is found. New variable `bitcode_modules` is introduced to generate bitcode files. All required information is gathered in this variable. Then, this variable is processed by the main meson LLVM bitcode emission scripts: src/backend/jit/llvm/bitcode/meson.build -> src/tools/irlink. An example of a possible structure of bitcode_modules is: ``` bitcode_modules = [ { 'name': '...', 'target': ..., 'srcfiles': [ '...', '...', ], 'additional_flags': [ '-I...', '-I...', ], 'gen_srcfiles': [ { 'srcfiles': [ <custom_target for ...>, <custom_target for ...>, ], 'additional_flags': [ '-I...', '-I...', ] } ] } ] ``` Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Author: Diego Fronza <[email protected]> Reviewed-by: Diego Fronza <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/backend/jit/llvm/bitcode/meson.build | 69 ++++++++++++++++++++++++ src/backend/jit/llvm/meson.build | 31 +++++++---- src/backend/meson.build | 6 +++ meson.build | 21 ++++++++ src/tools/irlink | 25 +++++++++ 5 files changed, 141 insertions(+), 11 deletions(-) create mode 100644 src/backend/jit/llvm/bitcode/meson.build create mode 100644 src/tools/irlink diff --git a/src/backend/jit/llvm/bitcode/meson.build b/src/backend/jit/llvm/bitcode/meson.build new file mode 100644 index 00000000000..f68c39975e6 --- /dev/null +++ b/src/backend/jit/llvm/bitcode/meson.build @@ -0,0 +1,69 @@ +# Copyright (c) 2022-2024, PostgreSQL Global Development Group +# +# emit LLVM bitcode for JIT inlining + +assert(llvm.found()) + +foreach bitcode_module : bitcode_modules + bitcode_targets = [] + bitcode_obj = bitcode_module['target'] + bitcode_cflags_local = bitcode_cflags + bitcode_module.get('additional_flags', []) + bitcode_name = bitcode_module.get('name', bitcode_obj.name()) + + foreach srcfile : bitcode_module['srcfiles'] + if meson.version().version_compare('>=0.59') + srcfilename = fs.parent(srcfile) / fs.name(srcfile) + else + srcfilename = '@0@'.format(srcfile) + endif + + targetname = '@0@_@[email protected]'.format( + bitcode_name, + srcfilename.underscorify(), + ) + bitcode_targets += custom_target( + targetname, + depends: [bitcode_obj], + input: [srcfile], + output: targetname, + command: [llvm_irgen_command, bitcode_cflags_local], + install: true, + install_dir: dir_bitcode, + ) + endforeach + + # Process generated sources, which may include custom compilation flags. + foreach gen_srcfiles: bitcode_module.get('gen_srcfiles', []) + bitcode_cflags_gen_local = bitcode_cflags_local + gen_srcfiles.get('additional_flags', []) + + foreach srcfile: gen_srcfiles['srcfiles'] + # Generated sources are stored in some folder under meson.build_root()/**, + # remove the build prefix from the string. + srcfilename = srcfile.full_path().split(meson.build_root() + '/')[1] + + targetname = '@0@_@[email protected]'.format( + bitcode_name, + srcfilename.underscorify(), + ) + bitcode_targets += custom_target( + targetname, + depends: [bitcode_obj], + input: [srcfile], + output: targetname, + command: [llvm_irgen_command, bitcode_cflags_gen_local], + install: true, + install_dir: dir_bitcode, + ) + endforeach + endforeach + + index_name = '@[email protected]'.format(bitcode_name) + bitcode_index = custom_target('@0@'.format(bitcode_name), + output: index_name, + input: bitcode_targets, + command: [irlink, '--lto', llvm_lto, '--outdir', '@OUTDIR@', '--index', index_name, '@INPUT@'], + install: true, + install_dir: dir_bitcode, + ) + backend_targets += bitcode_index +endforeach diff --git a/src/backend/jit/llvm/meson.build b/src/backend/jit/llvm/meson.build index c8e06dfbe35..2dd922e573b 100644 --- a/src/backend/jit/llvm/meson.build +++ b/src/backend/jit/llvm/meson.build @@ -42,21 +42,22 @@ backend_targets += llvmjit # Define a few bits and pieces used here and elsewhere to generate bitcode -llvm_irgen_args = [ - '-c', '-o', '@OUTPUT@', '@INPUT@', +llvm_irgen_command = [] +if ccache.found() + llvm_irgen_command += ccache +endif + +llvm_irgen_command += [ + clang, + '-c', '-o', '@OUTPUT0@', '@INPUT0@', '-flto=thin', '-emit-llvm', - '-MD', '-MQ', '@OUTPUT@', '-MF', '@DEPFILE@', '-O2', '-Wno-ignored-attributes', '-Wno-empty-body', + '-Wno-unknown-warning-option', + '-Wno-compound-token-split-by-macro', ] - -if ccache.found() - llvm_irgen_command = ccache - llvm_irgen_args = [clang.path()] + llvm_irgen_args -else - llvm_irgen_command = clang -endif +llvm_irgen_dep_args = ['-MD', '-MQ', '@OUTPUT0@', '-MF', '@DEPFILE@'] # XXX: Need to determine proper version of the function cflags for clang @@ -73,7 +74,7 @@ bitcode_cflags += '-I@SOURCE_ROOT@/src/include' # Note this is intentionally not installed to bitcodedir, as it's not for # inlining llvmjit_types = custom_target('llvmjit_types.bc', - command: [llvm_irgen_command] + llvm_irgen_args + bitcode_cflags, + command: llvm_irgen_command + llvm_irgen_dep_args + bitcode_cflags, input: 'llvmjit_types.c', output: 'llvmjit_types.bc', depends: [postgres], @@ -82,3 +83,11 @@ llvmjit_types = custom_target('llvmjit_types.bc', depfile: '@[email protected]', ) backend_targets += llvmjit_types + +# Figure out -I's needed to build all postgres code, including all its +# dependencies +pkg_config = find_program(['pkg-config', 'pkgconf'], required: true) +r = run_command(pkg_config, + ['--cflags-only-I', meson.build_root() / 'meson-uninstalled/postgresql-extension-uninstalled.pc'], + check: true) +bitcode_cflags += r.stdout().split() diff --git a/src/backend/meson.build b/src/backend/meson.build index 9d79d4d058c..5fb33660d3d 100644 --- a/src/backend/meson.build +++ b/src/backend/meson.build @@ -140,6 +140,12 @@ postgres = executable('postgres', backend_targets += postgres +bitcode_modules += { + 'name': 'postgres', + 'target': postgres_lib, + 'srcfiles': backend_sources, +} + pg_mod_c_args = cflags_mod pg_mod_cpp_args = cxxflags_mod pg_mod_link_args = ldflags_sl + ldflags_mod diff --git a/meson.build b/meson.build index 765b7f6a207..a01dbca66d1 100644 --- a/meson.build +++ b/meson.build @@ -814,6 +814,8 @@ if add_languages('cpp', required: llvmopt, native: false) # Some distros put LLVM and clang in different paths, so fallback to # find via PATH, too. clang = find_program(llvm_binpath / 'clang', 'clang', required: true) + llvm_lto = find_program(llvm_binpath / 'llvm-lto', required: true) + irlink = find_program('src/tools/irlink', native: true) endif elif llvmopt.auto() message('llvm requires a C++ compiler') @@ -2893,6 +2895,11 @@ test_deps = [] tests = [] meson_extension_tests = [] +# List of object files + source files to generated LLVM IR for inlining. +# Each element is a hash of: +# {'target': target, 'srcfiles': ..., 'additional_flags': ...}. +bitcode_modules = [] + # Default options for targets @@ -3212,6 +3219,11 @@ subdir('src/interfaces/ecpg/test') subdir('doc/src/sgml') +# generate bitcode for JIT inlining after giving contrib modules etc a chance +# to add themselves to bitcode_modules[] +subdir('src/backend/jit/llvm/bitcode', if_found: llvm) + + generated_sources_ac += {'': ['GNUmakefile']} # After processing src/test, add test_install_libs to the testprep_targets @@ -3847,6 +3859,15 @@ if meson.version().version_compare('>=0.57') section: 'Programs', ) + if llvm.found() + summary( + { + 'clang': clang, + }, + section: 'Programs', + ) + endif + summary( { 'bonjour': bonjour, diff --git a/src/tools/irlink b/src/tools/irlink new file mode 100644 index 00000000000..793c0abf91a --- /dev/null +++ b/src/tools/irlink @@ -0,0 +1,25 @@ +#!/usr/bin/env python3 + +import argparse +import os +import shutil +import subprocess +import sys + +parser = argparse.ArgumentParser( + description='generate PostgreSQL JIT IR module') + +parser.add_argument('--index', type=str, required=True) +parser.add_argument('--lto', type=str, required=True) +parser.add_argument('--outdir', type=str, required=True) +parser.add_argument('INPUT', type=str, nargs='+') + +args = parser.parse_args() + +file_names = [os.path.basename(f) for f in args.INPUT] +command = [args.lto, + '-thinlto', '-thinlto-action=thinlink', + '-o', args.index] + file_names +res = subprocess.run(command, cwd=args.outdir) + +exit(res.returncode) -- 2.47.2 [text/x-patch] v3-0006-meson-Add-LLVM-bitcode-emissions-for-contrib-libr.patch (22.6K, 7-v3-0006-meson-Add-LLVM-bitcode-emissions-for-contrib-libr.patch) download | inline diff: From 6152ccc6d0ca2aea0919479b7ca49f5d00acdb9e Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Wed, 12 Mar 2025 10:49:18 +0300 Subject: [PATCH v3 6/7] meson: Add LLVM bitcode emissions for contrib libraries The libraries which the bitcode files will be generated in are selected manually. Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Author: Diego Fronza <[email protected]> Reviewed-by: Diego Fronza <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/interfaces/libpq/meson.build | 3 +++ contrib/bloom/meson.build | 5 +++++ contrib/bool_plperl/meson.build | 9 +++++++++ contrib/btree_gin/meson.build | 5 +++++ contrib/btree_gist/meson.build | 5 +++++ contrib/citext/meson.build | 5 +++++ contrib/cube/meson.build | 13 +++++++++++++ contrib/dict_int/meson.build | 5 +++++ contrib/dict_xsyn/meson.build | 5 +++++ contrib/earthdistance/meson.build | 6 ++++++ contrib/fuzzystrmatch/meson.build | 9 +++++++++ contrib/hstore/meson.build | 7 +++++++ contrib/hstore_plperl/meson.build | 10 ++++++++++ contrib/hstore_plpython/meson.build | 12 ++++++++++++ contrib/intarray/meson.build | 5 +++++ contrib/isn/meson.build | 5 +++++ contrib/jsonb_plperl/meson.build | 8 ++++++++ contrib/jsonb_plpython/meson.build | 10 ++++++++++ contrib/lo/meson.build | 5 +++++ contrib/ltree/meson.build | 10 ++++++++++ contrib/ltree_plpython/meson.build | 11 +++++++++++ contrib/pg_buffercache/meson.build | 5 +++++ contrib/pg_freespacemap/meson.build | 5 +++++ contrib/pg_logicalinspect/meson.build | 5 +++++ contrib/pg_surgery/meson.build | 4 ++++ contrib/pg_trgm/meson.build | 5 +++++ contrib/pgcrypto/meson.build | 4 ++++ contrib/seg/meson.build | 12 ++++++++++++ contrib/spi/meson.build | 20 ++++++++++++++++++++ contrib/sslinfo/meson.build | 5 +++++ contrib/tablefunc/meson.build | 5 +++++ contrib/tcn/meson.build | 5 +++++ contrib/tsm_system_rows/meson.build | 5 +++++ contrib/tsm_system_time/meson.build | 5 +++++ contrib/unaccent/meson.build | 5 +++++ contrib/uuid-ossp/meson.build | 5 +++++ contrib/xml2/meson.build | 5 +++++ src/pl/plperl/meson.build | 1 + src/pl/plpython/meson.build | 1 + meson.build | 1 + 40 files changed, 256 insertions(+) diff --git a/src/interfaces/libpq/meson.build b/src/interfaces/libpq/meson.build index 19f4a52a97a..4742489e0ce 100644 --- a/src/interfaces/libpq/meson.build +++ b/src/interfaces/libpq/meson.build @@ -50,6 +50,9 @@ export_file = custom_target('libpq.exports', libpq_inc = include_directories('.', '../../port') libpq_c_args = ['-DSO_MAJOR_VERSION=5'] +# libpq-fe.h is needed to generate bitcode for some extensions +libpq_dir = meson.current_source_dir() + # Not using both_libraries() here as # 1) resource files should only be in the shared library # 2) we want the .pc file to include a dependency to {pgport,common}_static for diff --git a/contrib/bloom/meson.build b/contrib/bloom/meson.build index 695712a455e..1090c281532 100644 --- a/contrib/bloom/meson.build +++ b/contrib/bloom/meson.build @@ -28,6 +28,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': bloom, + 'srcfiles': bloom_sources, +} + tests += { 'name': 'bloom', 'sd': meson.current_source_dir(), diff --git a/contrib/bool_plperl/meson.build b/contrib/bool_plperl/meson.build index f489d99044c..1758adb7489 100644 --- a/contrib/bool_plperl/meson.build +++ b/contrib/bool_plperl/meson.build @@ -37,6 +37,15 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': bool_plperl, + 'srcfiles': bool_plperl_sources, + 'additional_flags': [ + '-I@0@'.format(plperl_dir), + perl_ccflags + ] +} + tests += { 'name': 'bool_plperl', 'sd': meson.current_source_dir(), diff --git a/contrib/btree_gin/meson.build b/contrib/btree_gin/meson.build index b2749f6e669..26d49d005ab 100644 --- a/contrib/btree_gin/meson.build +++ b/contrib/btree_gin/meson.build @@ -25,6 +25,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': btree_gin, + 'srcfiles': btree_gin_sources, +} + tests += { 'name': 'btree_gin', 'sd': meson.current_source_dir(), diff --git a/contrib/btree_gist/meson.build b/contrib/btree_gist/meson.build index f4fa9574f1f..2649d5a7ffb 100644 --- a/contrib/btree_gist/meson.build +++ b/contrib/btree_gist/meson.build @@ -54,6 +54,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': btree_gist, + 'srcfiles': btree_gist_sources, +} + tests += { 'name': 'btree_gist', 'sd': meson.current_source_dir(), diff --git a/contrib/citext/meson.build b/contrib/citext/meson.build index 7fff34f2368..fa07ff72be4 100644 --- a/contrib/citext/meson.build +++ b/contrib/citext/meson.build @@ -30,6 +30,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': citext, + 'srcfiles': citext_sources, +} + tests += { 'name': 'citext', 'sd': meson.current_source_dir(), diff --git a/contrib/cube/meson.build b/contrib/cube/meson.build index fd3c057f469..99cff1addce 100644 --- a/contrib/cube/meson.build +++ b/contrib/cube/meson.build @@ -3,6 +3,7 @@ cube_sources = files( 'cube.c', ) +bc_cube_sources = cube_sources cube_scan = custom_target('cubescan', input: 'cubescan.l', @@ -11,6 +12,7 @@ cube_scan = custom_target('cubescan', ) generated_sources += cube_scan cube_sources += cube_scan +bc_cube_gen_sources = [{'srcfiles': [cube_scan]}] cube_parse = custom_target('cubeparse', input: 'cubeparse.y', @@ -18,6 +20,7 @@ cube_parse = custom_target('cubeparse', ) generated_sources += cube_parse.to_list() cube_sources += cube_parse +bc_cube_gen_sources += {'srcfiles': [cube_parse[0]]} if host_system == 'windows' cube_sources += rc_lib_gen.process(win32ver_rc, extra_args: [ @@ -48,6 +51,16 @@ install_headers( install_dir: dir_include_extension / 'cube', ) +cube_dir = meson.current_source_dir() +bitcode_modules += { + 'target': cube, + 'srcfiles': bc_cube_sources, + 'gen_srcfiles': bc_cube_gen_sources, + 'additional_flags': [ + '-I@0@'.format(cube_dir), + ] +} + tests += { 'name': 'cube', 'sd': meson.current_source_dir(), diff --git a/contrib/dict_int/meson.build b/contrib/dict_int/meson.build index ab41588547b..894c8a9c94d 100644 --- a/contrib/dict_int/meson.build +++ b/contrib/dict_int/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': dict_int, + 'srcfiles': dict_int_sources, +} + tests += { 'name': 'dict_int', 'sd': meson.current_source_dir(), diff --git a/contrib/dict_xsyn/meson.build b/contrib/dict_xsyn/meson.build index 93f41b1f963..d4bd4a6ef5a 100644 --- a/contrib/dict_xsyn/meson.build +++ b/contrib/dict_xsyn/meson.build @@ -29,6 +29,11 @@ install_data( } ) +bitcode_modules += { + 'target': dict_xsyn, + 'srcfiles': dict_xsyn_sources, +} + tests += { 'name': 'dict_xsyn', 'sd': meson.current_source_dir(), diff --git a/contrib/earthdistance/meson.build b/contrib/earthdistance/meson.build index a5f8d1a3ffa..c101359b118 100644 --- a/contrib/earthdistance/meson.build +++ b/contrib/earthdistance/meson.build @@ -24,6 +24,12 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': earthdistance, + 'srcfiles': earthdistance_sources, +} + + tests += { 'name': 'earthdistance', 'sd': meson.current_source_dir(), diff --git a/contrib/fuzzystrmatch/meson.build b/contrib/fuzzystrmatch/meson.build index f52daa4ea1c..fc5e88d0c92 100644 --- a/contrib/fuzzystrmatch/meson.build +++ b/contrib/fuzzystrmatch/meson.build @@ -5,6 +5,7 @@ fuzzystrmatch_sources = files( 'dmetaphone.c', 'fuzzystrmatch.c', ) +bc_fuzzystrmatch_sources = fuzzystrmatch_sources daitch_mokotoff_h = custom_target('daitch_mokotoff', input: 'daitch_mokotoff_header.pl', @@ -35,6 +36,14 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': fuzzystrmatch, + 'srcfiles': bc_fuzzystrmatch_sources, + 'additional_flags': [ + '-I@0@'.format(meson.current_build_dir()) + ] +} + tests += { 'name': 'fuzzystrmatch', 'sd': meson.current_source_dir(), diff --git a/contrib/hstore/meson.build b/contrib/hstore/meson.build index 39622d1024d..344a620273f 100644 --- a/contrib/hstore/meson.build +++ b/contrib/hstore/meson.build @@ -43,6 +43,13 @@ install_headers( install_dir: dir_include_extension / 'hstore', ) +# some libraries include "hstore/hstore.h" instead of "hstore.h" +hstore_dir_up = join_paths(meson.current_source_dir(), '..') +bitcode_modules += { + 'target': hstore, + 'srcfiles': hstore_sources, +} + tests += { 'name': 'hstore', 'sd': meson.current_source_dir(), diff --git a/contrib/hstore_plperl/meson.build b/contrib/hstore_plperl/meson.build index 60b8ad23569..b689b55e1c9 100644 --- a/contrib/hstore_plperl/meson.build +++ b/contrib/hstore_plperl/meson.build @@ -37,6 +37,16 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': hstore_plperl, + 'srcfiles': hstore_plperl_sources, + 'additional_flags': [ + '-I@0@'.format(hstore_dir_up), + '-I@0@'.format(plperl_dir), + perl_ccflags, + ] +} + tests += { 'name': 'hstore_plperl', 'sd': meson.current_source_dir(), diff --git a/contrib/hstore_plpython/meson.build b/contrib/hstore_plpython/meson.build index ea8e20a377b..d68de3dc46b 100644 --- a/contrib/hstore_plpython/meson.build +++ b/contrib/hstore_plpython/meson.build @@ -30,6 +30,18 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': hstore_plpython, + 'srcfiles': hstore_plpython_sources, + 'additional_flags': [ + '-I@0@'.format(hstore_dir_up), + '-DPLPYTHON_LIBNAME="plpython3"', + '-I@0@'.format(python3_inc_dir), + '-I@0@'.format(plpython_dir), + perl_ccflags, + ] +} + hstore_plpython_regress = [ 'hstore_plpython' ] diff --git a/contrib/intarray/meson.build b/contrib/intarray/meson.build index fae9add981d..3d7da0c9c9a 100644 --- a/contrib/intarray/meson.build +++ b/contrib/intarray/meson.build @@ -33,6 +33,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': intarray, + 'srcfiles': intarray_sources, +} + tests += { 'name': 'intarray', 'sd': meson.current_source_dir(), diff --git a/contrib/isn/meson.build b/contrib/isn/meson.build index fbbeeff01bb..d42473134ea 100644 --- a/contrib/isn/meson.build +++ b/contrib/isn/meson.build @@ -29,6 +29,11 @@ install_headers( install_dir: dir_include_extension / 'isn', ) +bitcode_modules += { + 'target': isn, + 'srcfiles': isn_sources, +} + tests += { 'name': 'isn', 'sd': meson.current_source_dir(), diff --git a/contrib/jsonb_plperl/meson.build b/contrib/jsonb_plperl/meson.build index 95a9a7bc082..febafc73a99 100644 --- a/contrib/jsonb_plperl/meson.build +++ b/contrib/jsonb_plperl/meson.build @@ -37,6 +37,14 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': jsonb_plperl, + 'srcfiles': jsonb_plperl_sources, + 'additional_flags': [ + '-I@0@'.format(plperl_dir), + perl_ccflags, + ] +} tests += { 'name': 'jsonb_plperl', diff --git a/contrib/jsonb_plpython/meson.build b/contrib/jsonb_plpython/meson.build index 5fe80483e58..6be92221272 100644 --- a/contrib/jsonb_plpython/meson.build +++ b/contrib/jsonb_plpython/meson.build @@ -30,6 +30,16 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': jsonb_plpython, + 'srcfiles': jsonb_plpython_sources, + 'additional_flags': [ + '-DPLPYTHON_LIBNAME="plpython3"', + '-I@0@'.format(python3_inc_dir), + '-I@0@'.format(plpython_dir), + ] +} + jsonb_plpython_regress = [ 'jsonb_plpython' ] diff --git a/contrib/lo/meson.build b/contrib/lo/meson.build index 2d78907ba12..1222faa9169 100644 --- a/contrib/lo/meson.build +++ b/contrib/lo/meson.build @@ -24,6 +24,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': lo, + 'srcfiles': lo_sources, +} + tests += { 'name': 'lo', 'sd': meson.current_source_dir(), diff --git a/contrib/ltree/meson.build b/contrib/ltree/meson.build index f9b06302839..cda86935544 100644 --- a/contrib/ltree/meson.build +++ b/contrib/ltree/meson.build @@ -41,6 +41,16 @@ install_headers( install_dir: dir_include_extension / 'ltree', ) +ltree_dir = meson.current_source_dir() +ltree_dir_up = join_paths(ltree_dir, '..') +bitcode_modules += { + 'target': ltree, + 'srcfiles': ltree_sources, + 'additional_flags': [ + '-I@0@'.format(ltree_dir) + ] +} + tests += { 'name': 'ltree', 'sd': meson.current_source_dir(), diff --git a/contrib/ltree_plpython/meson.build b/contrib/ltree_plpython/meson.build index a37732c486b..74b285da05c 100644 --- a/contrib/ltree_plpython/meson.build +++ b/contrib/ltree_plpython/meson.build @@ -30,6 +30,17 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': ltree_plpython, + 'srcfiles': ltree_plpython_sources, + 'additional_flags': [ + '-I@0@'.format(ltree_dir_up), + '-DPLPYTHON_LIBNAME="plpython3"', + '-I@0@'.format(python3_inc_dir), + '-I@0@'.format(plpython_dir), + ] +} + ltree_plpython_regress = [ 'ltree_plpython' ] diff --git a/contrib/pg_buffercache/meson.build b/contrib/pg_buffercache/meson.build index 12d1fe48717..5385318fdc9 100644 --- a/contrib/pg_buffercache/meson.build +++ b/contrib/pg_buffercache/meson.build @@ -27,6 +27,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pg_buffercache, + 'srcfiles': pg_buffercache_sources, +} + tests += { 'name': 'pg_buffercache', 'sd': meson.current_source_dir(), diff --git a/contrib/pg_freespacemap/meson.build b/contrib/pg_freespacemap/meson.build index ff8eda3580e..120def1ad2e 100644 --- a/contrib/pg_freespacemap/meson.build +++ b/contrib/pg_freespacemap/meson.build @@ -25,6 +25,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pg_freespacemap, + 'srcfiles': pg_freespacemap_sources, +} + tests += { 'name': 'pg_freespacemap', 'sd': meson.current_source_dir(), diff --git a/contrib/pg_logicalinspect/meson.build b/contrib/pg_logicalinspect/meson.build index 5c0528a8c63..1b4f46277ed 100644 --- a/contrib/pg_logicalinspect/meson.build +++ b/contrib/pg_logicalinspect/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pg_logicalinspect, + 'srcfiles': pg_logicalinspect_sources, +} + tests += { 'name': 'pg_logicalinspect', 'sd': meson.current_source_dir(), diff --git a/contrib/pg_surgery/meson.build b/contrib/pg_surgery/meson.build index c6cfa9c4694..9d7199392c7 100644 --- a/contrib/pg_surgery/meson.build +++ b/contrib/pg_surgery/meson.build @@ -22,6 +22,10 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pg_surgery, + 'srcfiles': pg_surgery_sources, +} tests += { 'name': 'pg_surgery', diff --git a/contrib/pg_trgm/meson.build b/contrib/pg_trgm/meson.build index a31aa5c574d..408e287172e 100644 --- a/contrib/pg_trgm/meson.build +++ b/contrib/pg_trgm/meson.build @@ -32,6 +32,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pg_trgm, + 'srcfiles': pg_trgm_sources, +} + tests += { 'name': 'pg_trgm', 'sd': meson.current_source_dir(), diff --git a/contrib/pgcrypto/meson.build b/contrib/pgcrypto/meson.build index 7a4e8e76d64..d6b0559f232 100644 --- a/contrib/pgcrypto/meson.build +++ b/contrib/pgcrypto/meson.build @@ -98,6 +98,10 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pgcrypto, + 'srcfiles': pgcrypto_sources, +} tests += { 'name': 'pgcrypto', diff --git a/contrib/seg/meson.build b/contrib/seg/meson.build index e331e097230..61ace636e43 100644 --- a/contrib/seg/meson.build +++ b/contrib/seg/meson.build @@ -3,6 +3,7 @@ seg_sources = files( 'seg.c', ) +bc_seg_sources = seg_sources seg_scan = custom_target('segscan', input: 'segscan.l', @@ -11,6 +12,7 @@ seg_scan = custom_target('segscan', ) generated_sources += seg_scan seg_sources += seg_scan +bc_seg_gen_sources = [{'srcfiles': [seg_scan]}] seg_parse = custom_target('segparse', input: 'segparse.y', @@ -18,6 +20,7 @@ seg_parse = custom_target('segparse', ) generated_sources += seg_parse.to_list() seg_sources += seg_parse +bc_seg_gen_sources += {'srcfiles': [seg_parse[0]]} if host_system == 'windows' seg_sources += rc_lib_gen.process(win32ver_rc, extra_args: [ @@ -47,6 +50,15 @@ install_headers( install_dir: dir_include_extension / 'seg', ) +bitcode_modules += { + 'target': seg, + 'srcfiles': bc_seg_sources, + 'gen_srcfiles': bc_seg_gen_sources, + 'additional_flags': [ + '-I@0@'.format(meson.current_source_dir()), + ] +} + tests += { 'name': 'seg', 'sd': meson.current_source_dir(), diff --git a/contrib/spi/meson.build b/contrib/spi/meson.build index eeab1ab210b..b16c4384bb0 100644 --- a/contrib/spi/meson.build +++ b/contrib/spi/meson.build @@ -24,6 +24,11 @@ install_data('autoinc.example', kwargs: contrib_doc_args, ) +bitcode_modules += { + 'target': autoinc, + 'srcfiles': autoinc_sources, +} + insert_username_sources = files( 'insert_username.c', @@ -51,6 +56,11 @@ install_data('insert_username.example', kwargs: contrib_doc_args, ) +bitcode_modules += { + 'target': insert_username, + 'srcfiles': insert_username_sources, +} + moddatetime_sources = files( 'moddatetime.c', @@ -78,6 +88,11 @@ install_data('moddatetime.example', kwargs: contrib_doc_args, ) +bitcode_modules += { + 'target': moddatetime, + 'srcfiles': moddatetime_sources, +} + # this is needed for the regression tests; # comment out if you want a quieter refint package for other uses @@ -107,3 +122,8 @@ install_data('refint.control', 'refint--1.0.sql', install_data('refint.example', kwargs: contrib_doc_args, ) + +bitcode_modules += { + 'target': refint, + 'srcfiles': refint_sources, +} diff --git a/contrib/sslinfo/meson.build b/contrib/sslinfo/meson.build index 4c513759200..5750d783b92 100644 --- a/contrib/sslinfo/meson.build +++ b/contrib/sslinfo/meson.build @@ -29,3 +29,8 @@ install_data( 'sslinfo.control', kwargs: contrib_data_args, ) + +bitcode_modules += { + 'target': sslinfo, + 'srcfiles': sslinfo_sources, +} diff --git a/contrib/tablefunc/meson.build b/contrib/tablefunc/meson.build index ee67272ec0a..6c26ffafddf 100644 --- a/contrib/tablefunc/meson.build +++ b/contrib/tablefunc/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': tablefunc, + 'srcfiles': tablefunc_sources, +} + tests += { 'name': 'tablefunc', 'sd': meson.current_source_dir(), diff --git a/contrib/tcn/meson.build b/contrib/tcn/meson.build index 6ffb136af90..df71e8a91f9 100644 --- a/contrib/tcn/meson.build +++ b/contrib/tcn/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': tcn, + 'srcfiles': tcn_sources, +} + tests += { 'name': 'tcn', 'sd': meson.current_source_dir(), diff --git a/contrib/tsm_system_rows/meson.build b/contrib/tsm_system_rows/meson.build index b8cece3d80f..1ea74cce949 100644 --- a/contrib/tsm_system_rows/meson.build +++ b/contrib/tsm_system_rows/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': tsm_system_rows, + 'srcfiles': tsm_system_rows_sources, +} + tests += { 'name': 'tsm_system_rows', 'sd': meson.current_source_dir(), diff --git a/contrib/tsm_system_time/meson.build b/contrib/tsm_system_time/meson.build index 8a143a8f8e6..1f6b98d3ec4 100644 --- a/contrib/tsm_system_time/meson.build +++ b/contrib/tsm_system_time/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': tsm_system_time, + 'srcfiles': tsm_system_time_sources, +} + tests += { 'name': 'tsm_system_time', 'sd': meson.current_source_dir(), diff --git a/contrib/unaccent/meson.build b/contrib/unaccent/meson.build index 33d4649bae1..1f5d120e3d8 100644 --- a/contrib/unaccent/meson.build +++ b/contrib/unaccent/meson.build @@ -28,6 +28,11 @@ install_data( install_dir: dir_data / 'tsearch_data' ) +bitcode_modules += { + 'target': unaccent, + 'srcfiles': unaccent_sources, +} + # XXX: Implement downlo tests += { 'name': 'unaccent', diff --git a/contrib/uuid-ossp/meson.build b/contrib/uuid-ossp/meson.build index 982f27c085f..8d21bb0a996 100644 --- a/contrib/uuid-ossp/meson.build +++ b/contrib/uuid-ossp/meson.build @@ -29,6 +29,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': uuid_ossp, + 'srcfiles': uuid_ossp_sources, +} + tests += { 'name': 'uuid-ossp', 'sd': meson.current_source_dir(), diff --git a/contrib/xml2/meson.build b/contrib/xml2/meson.build index 08d3c3b8e30..b23a8fe0cd7 100644 --- a/contrib/xml2/meson.build +++ b/contrib/xml2/meson.build @@ -32,6 +32,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': xml2, + 'srcfiles': xml2_sources, +} + tests += { 'name': 'xml2', 'sd': meson.current_source_dir(), diff --git a/src/pl/plperl/meson.build b/src/pl/plperl/meson.build index b463d4d56c5..279277dee73 100644 --- a/src/pl/plperl/meson.build +++ b/src/pl/plperl/meson.build @@ -37,6 +37,7 @@ foreach n : ['SPI', 'Util'] plperl_sources += xs_c endforeach +plperl_dir = meson.current_source_dir() plperl_inc = include_directories('.') if host_system == 'windows' diff --git a/src/pl/plpython/meson.build b/src/pl/plpython/meson.build index 709e5932a93..e2c19771d85 100644 --- a/src/pl/plpython/meson.build +++ b/src/pl/plpython/meson.build @@ -29,6 +29,7 @@ plpython_sources += custom_target('spiexceptions.h', # FIXME: need to duplicate import library ugliness? plpython_inc = include_directories('.') +plpython_dir = meson.current_source_dir() if host_system == 'windows' plpython_sources += rc_lib_gen.process(win32ver_rc, extra_args: [ diff --git a/meson.build b/meson.build index a01dbca66d1..9e20be4590d 100644 --- a/meson.build +++ b/meson.build @@ -1244,6 +1244,7 @@ if not pyopt.disabled() python3_inst = pm.find_installation(python.path(), required: pyopt) if python3_inst.found() python3_dep = python3_inst.dependency(embed: true, required: pyopt) + python3_inc_dir = python3_inst.get_variable('INCLUDEPY') # Remove this check after we depend on Meson >= 1.1.0 if not cc.check_header('Python.h', dependencies: python3_dep, required: pyopt, include_directories: postgres_inc) python3_dep = not_found_dep -- 2.47.2 [text/x-patch] v3-0007-meson-Add-LLVM-bitcode-emission-for-backend-sourc.patch (5.2K, 8-v3-0007-meson-Add-LLVM-bitcode-emission-for-backend-sourc.patch) download | inline diff: From d87417348a3e2caedbdf6955258c7f1fbf426853 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Wed, 12 Mar 2025 10:44:46 +0300 Subject: [PATCH v3 7/7] meson: Add LLVM bitcode emission for backend sources Since generated backend sources may have their own compilation flags and must also be included in the postgres.index.bc, the way to make it work with current code was to create a new variable, called `bc_generated_backend_sources`, which is a list of dictionaries, each one having an optional 'additional_flags' and a `srclist` pointing to the list of custom_target generated sources. An example of a possible structure of bitcode_modules which is processed by the main meson llvm bitcode emission file src/backend/jit/llvm/bitcode/meson.build: ``` bitcode_modules = [ { 'name': 'postgres', 'target': postgres_lib, 'src_file': backend_sources, 'gen_srcfiles': [ { 'additional_flags': [ '-I/path/postgresl/src/backend/parser', '-I/path/postgresl/build/src/backend/parser', ], 'srcfiles': [ <custom_target for scan.c>, <custom_target for gram.c> ] } ] } ] ``` Author: Diego Fronza <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/backend/bootstrap/meson.build | 2 ++ src/backend/meson.build | 2 ++ src/backend/parser/meson.build | 8 ++++++++ src/backend/replication/meson.build | 4 ++++ src/backend/utils/fmgr/meson.build | 1 + 5 files changed, 17 insertions(+) diff --git a/src/backend/bootstrap/meson.build b/src/backend/bootstrap/meson.build index 29726c1ab4f..389c75f8081 100644 --- a/src/backend/bootstrap/meson.build +++ b/src/backend/bootstrap/meson.build @@ -13,6 +13,7 @@ bootscanner = custom_target('bootscanner', ) generated_sources += bootscanner boot_parser_sources += bootscanner +bc_generated_backend_sources += {'srcfiles': [bootscanner]} bootparse = custom_target('bootparse', input: 'bootparse.y', @@ -20,6 +21,7 @@ bootparse = custom_target('bootparse', ) generated_sources += bootparse.to_list() boot_parser_sources += bootparse +bc_generated_backend_sources += {'srcfiles': [bootparse[0]]} boot_parser = static_library('boot_parser', boot_parser_sources, diff --git a/src/backend/meson.build b/src/backend/meson.build index 5fb33660d3d..76e5939feca 100644 --- a/src/backend/meson.build +++ b/src/backend/meson.build @@ -5,6 +5,7 @@ backend_sources = [] backend_link_with = [pgport_srv, common_srv] generated_backend_sources = [] +bc_generated_backend_sources = [] post_export_backend_sources = [] subdir('access') @@ -144,6 +145,7 @@ bitcode_modules += { 'name': 'postgres', 'target': postgres_lib, 'srcfiles': backend_sources, + 'gen_srcfiles': bc_generated_backend_sources, } pg_mod_c_args = cflags_mod diff --git a/src/backend/parser/meson.build b/src/backend/parser/meson.build index 874aa749aa6..add472a0cd8 100644 --- a/src/backend/parser/meson.build +++ b/src/backend/parser/meson.build @@ -42,6 +42,14 @@ backend_parser = custom_target('gram', generated_sources += backend_parser.to_list() parser_sources += backend_parser +bc_generated_backend_sources += { + 'additional_flags': [ + '-I@0@'.format(meson.current_build_dir()), + '-I@0@'.format(meson.current_source_dir()), + ], + 'srcfiles': [backend_scanner, backend_parser[0]], +} + parser = static_library('parser', parser_sources, dependencies: [backend_code], diff --git a/src/backend/replication/meson.build b/src/backend/replication/meson.build index b0601498865..71ab1164960 100644 --- a/src/backend/replication/meson.build +++ b/src/backend/replication/meson.build @@ -19,6 +19,7 @@ repl_scanner = custom_target('repl_scanner', ) generated_sources += repl_scanner repl_parser_sources += repl_scanner +bc_generated_backend_sources += {'srcfiles': [repl_scanner]} repl_gram = custom_target('repl_gram', input: 'repl_gram.y', @@ -26,6 +27,7 @@ repl_gram = custom_target('repl_gram', ) generated_sources += repl_gram.to_list() repl_parser_sources += repl_gram +bc_generated_backend_sources += {'srcfiles': [repl_gram[0]]} syncrep_scanner = custom_target('syncrep_scanner', input: 'syncrep_scanner.l', @@ -34,6 +36,7 @@ syncrep_scanner = custom_target('syncrep_scanner', ) generated_sources += syncrep_scanner repl_parser_sources += syncrep_scanner +bc_generated_backend_sources += {'srcfiles': [syncrep_scanner]} syncrep_gram = custom_target('syncrep_gram', input: 'syncrep_gram.y', @@ -41,6 +44,7 @@ syncrep_gram = custom_target('syncrep_gram', ) generated_sources += syncrep_gram.to_list() repl_parser_sources += syncrep_gram +bc_generated_backend_sources += {'srcfiles': [syncrep_gram[0]]} repl_parser = static_library('repl_parser', repl_parser_sources, diff --git a/src/backend/utils/fmgr/meson.build b/src/backend/utils/fmgr/meson.build index b1dcab93e70..080f59988e9 100644 --- a/src/backend/utils/fmgr/meson.build +++ b/src/backend/utils/fmgr/meson.build @@ -8,3 +8,4 @@ backend_sources += files( # fmgrtab.c generated_backend_sources += fmgrtab_target[2] +bc_generated_backend_sources += {'srcfiles': [fmgrtab_target[2]]} -- 2.47.2 ^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: meson vs. llvm bitcode files @ 2025-03-12 13:39 Diego Fronza <[email protected]> parent: Nazir Bilal Yavuz <[email protected]> 0 siblings, 1 reply; 10+ messages in thread From: Diego Fronza @ 2025-03-12 13:39 UTC (permalink / raw) To: Nazir Bilal Yavuz <[email protected]>; +Cc: Peter Eisentraut <[email protected]>; pgsql-hackers; Andres Freund <[email protected]> Hi, The v7 patch looks good to me, handling the bitcode modules in a uniform way and also avoiding the hacky code and warnings, much better now. A small note about the bitcode emission for generated sources in contrib, using cube as example, currently it creates two dict entries in a list: bc_seg_gen_sources = [{'srcfiles': [seg_scan]}] bc_seg_gen_sources += {'srcfiles': [seg_parse[0]]} Then pass it to the bitcode_modules: bitcode_modules += { ... 'gen_srcfiles': bc_seg_gen_sources, } It could be passed as a list with a single dict, since both generated sources share the same compilation flags: bitcode_modules += { ... 'gen_srcfiles': [ { 'srcfiles': [cube_scan, cube_parse[0]] }. ] } Both approaches work, the first one has the advantage of being able to pass separate additional_flags per generated source. Thanks for your reply Nazir, also waiting for more opinions on this. Regards, Diego On Wed, Mar 12, 2025 at 7:27 AM Nazir Bilal Yavuz <[email protected]> wrote: > Hi, > > On Tue, 11 Mar 2025 at 01:04, Diego Fronza <[email protected]> > wrote: > > I did a full review on the provided patches plus some tests, I was able > to validate that the loading of bitcode modules is working also JIT works > for both backend and contrib modules. > > Thank you! > > > To test JIT on contrib modules I just lowered the costs for all jit > settings and used the intarray extension, using the data/test__int.data: > > CREATE EXTENSION intarray; > > CREATE TABLE test__int( a int[] );1 > > \copy test__int from 'data/test__int.data' > > > > For queries any from line 98+ on contrib/intarray/sql/_int.sql will work. > > > > Then I added extra debug messages to llvmjit_inline.cpp on > add_module_to_inline_search_path() function, also on > llvm_build_inline_plan(), I was able to see many functions in this module > being successfully inlined. > > > > I'm attaching a new patch based on your original work which add further > support for generating bitcode from: > > Thanks for doing that! > > > - Generated backend sources: processed by flex, bison, etc. > > - Generated contrib module sources, > > I think we do not need to separate these two. > > foreach srcfile : bitcode_module['srcfiles'] > - if meson.version().version_compare('>=0.59') > + srcfilename = '@0@'.format(srcfile) > + if srcfilename.startswith('<CustomTarget') > + srcfilename = srcfile.full_path().split(meson.build_root() + '/')[1] > + elif meson.version().version_compare('>=0.59') > > Also, checking if the string starts with '<CustomTarget' is a bit > hacky, and 'srcfilename = '@0@'.format(srcfile)' causes a deprecation > warning. So, instead of this we can process all generated sources like > how generated backend sources are processed. I updated the patch with > that. > > > On this patch I just included fmgrtab.c and src/backend/parser for the > backend generated code. > > For contrib generated sources I added contrib/cube as an example. > > I applied your contrib/cube example and did the same thing for the > contrib/seg. > > > All relevant details about the changes are included in the patch itself. > > > > As you may know already I also created a PR focused on llvm bitcode > emission on meson, it generates bitcode for all backend and contribution > modules, currently under review by some colleagues at Percona: > https://github.com/percona/postgres/pull/103 > > I'm curious if we should get all or some of the generated backend > sources compiled to bitcode, similar to contrib modules. > > I think we can do this. I added other backend sources like you did in > the PR but attached it as another patch (0007) because I wanted to > hear other people's opinions on that first. > > v3 is attached. > > -- > Regards, > Nazir Bilal Yavuz > Microsoft > ^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: meson vs. llvm bitcode files @ 2025-03-13 10:11 Nazir Bilal Yavuz <[email protected]> parent: Diego Fronza <[email protected]> 0 siblings, 1 reply; 10+ messages in thread From: Nazir Bilal Yavuz @ 2025-03-13 10:11 UTC (permalink / raw) To: Diego Fronza <[email protected]>; +Cc: Peter Eisentraut <[email protected]>; pgsql-hackers; Andres Freund <[email protected]> Hi, On Wed, 12 Mar 2025 at 16:39, Diego Fronza <[email protected]> wrote: > > Hi, > > The v7 patch looks good to me, handling the bitcode modules in a uniform way and also avoiding the hacky code and warnings, much better now. > > A small note about the bitcode emission for generated sources in contrib, using cube as example, currently it creates two dict entries in a list: > bc_seg_gen_sources = [{'srcfiles': [seg_scan]}] > bc_seg_gen_sources += {'srcfiles': [seg_parse[0]]} > > Then pass it to the bitcode_modules: > bitcode_modules += { > ... > 'gen_srcfiles': bc_seg_gen_sources, > } > > It could be passed as a list with a single dict, since both generated sources share the same compilation flags: > bitcode_modules += { > ... > 'gen_srcfiles': [ > { 'srcfiles': [cube_scan, cube_parse[0]] }. > ] > } > > Both approaches work, the first one has the advantage of being able to pass separate additional_flags per generated source. I liked the current approach as it makes bitcode_modules easier to understand but both approaches work for me as well. One thing I noticed is that gen_srcfiles['srcfiles'] seems wrong. gen_sources is a better name compared to gen_srcfiles. So, I changed it to gen_sources in v4. -- Regards, Nazir Bilal Yavuz Microsoft Attachments: [text/x-patch] v4-0001-meson-Add-generated-header-stamps.patch (5.0K, 2-v4-0001-meson-Add-generated-header-stamps.patch) download | inline diff: From 578fbb407ce89293c8c955179e61b10aadc1709f Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Thu, 24 Oct 2024 07:23:05 -0400 Subject: [PATCH v4 1/7] meson: Add generated header stamps Otherwise build commands become too long and this has visible effect on creation time of meson build files. Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/include/meson.build | 18 ++++++++++++++++++ src/backend/meson.build | 2 +- src/fe_utils/meson.build | 2 +- meson.build | 16 +++++++++------- 4 files changed, 29 insertions(+), 9 deletions(-) diff --git a/src/include/meson.build b/src/include/meson.build index 2e4b7aa529e..c488a5dc4c9 100644 --- a/src/include/meson.build +++ b/src/include/meson.build @@ -177,3 +177,21 @@ install_subdir('catalog', # autoconf generates the file there, ensure we get a conflict generated_sources_ac += {'src/include': ['stamp-h']} + +# Instead of having targets depending directly on the generated headers, have +# them depend on a stamp files for all of them. Dependencies on headers are +# implemented as order-only dependencies in meson (later using compiler +# generated dependencies). The benefit of using a stamp file is that it makes +# ninja.build smaller and meson setup faster. +generated_headers_stamp = custom_target('generated-headers-stamp.h', + output: 'generated-headers-stamp.h', + input: generated_headers, + command: stamp_cmd, +) + +generated_backend_headers_stamp = custom_target('generated-backend-headers-stamp.h', + output: 'generated-backend-headers-stamp.h', + input: generated_backend_headers, + depends: generated_headers_stamp, + command: stamp_cmd, +) diff --git a/src/backend/meson.build b/src/backend/meson.build index 2b0db214804..7fc649c3ebd 100644 --- a/src/backend/meson.build +++ b/src/backend/meson.build @@ -169,7 +169,7 @@ backend_mod_code = declare_dependency( compile_args: pg_mod_c_args, include_directories: postgres_inc, link_args: pg_mod_link_args, - sources: generated_headers + generated_backend_headers, + sources: [generated_backend_headers_stamp], dependencies: backend_mod_deps, ) diff --git a/src/fe_utils/meson.build b/src/fe_utils/meson.build index a18cbc939e4..5a9ddb73463 100644 --- a/src/fe_utils/meson.build +++ b/src/fe_utils/meson.build @@ -29,7 +29,7 @@ generated_sources += psqlscan fe_utils_sources += psqlscan fe_utils = static_library('libpgfeutils', - fe_utils_sources + generated_headers, + fe_utils_sources, c_pch: pch_postgres_fe_h, include_directories: [postgres_inc, libpq_inc], c_args: host_system == 'windows' ? ['-DFD_SETSIZE=1024'] : [], diff --git a/meson.build b/meson.build index 13c13748e5d..9bfa96ad255 100644 --- a/meson.build +++ b/meson.build @@ -2973,6 +2973,8 @@ gen_export_kwargs = { 'install': false, } +# command to create stamp files on all OSs +stamp_cmd = [python, '-c', 'import sys; open(sys.argv[1], "w")', '@OUTPUT0@'] ### @@ -3090,14 +3092,14 @@ subdir('src/port') frontend_common_code = declare_dependency( compile_args: ['-DFRONTEND'], include_directories: [postgres_inc], - sources: generated_headers, + sources: generated_headers_stamp, dependencies: [os_deps, zlib, zstd, lz4], ) backend_common_code = declare_dependency( compile_args: ['-DBUILDING_DLL'], include_directories: [postgres_inc], - sources: generated_headers, + sources: generated_headers_stamp, dependencies: [os_deps, zlib, zstd], ) @@ -3112,7 +3114,7 @@ shlib_code = declare_dependency( frontend_stlib_code = declare_dependency( include_directories: [postgres_inc], link_with: [common_static, pgport_static], - sources: generated_headers, + sources: generated_headers_stamp, dependencies: [os_deps, libintl], ) @@ -3120,7 +3122,7 @@ frontend_stlib_code = declare_dependency( frontend_shlib_code = declare_dependency( include_directories: [postgres_inc], link_with: [common_shlib, pgport_shlib], - sources: generated_headers, + sources: generated_headers_stamp, dependencies: [shlib_code, os_deps, libintl], ) @@ -3130,7 +3132,7 @@ frontend_shlib_code = declare_dependency( frontend_no_fe_utils_code = declare_dependency( include_directories: [postgres_inc], link_with: [common_static, pgport_static], - sources: generated_headers, + sources: generated_headers_stamp, dependencies: [os_deps, libintl], ) @@ -3156,7 +3158,7 @@ subdir('src/fe_utils') frontend_code = declare_dependency( include_directories: [postgres_inc], link_with: [fe_utils, common_static, pgport_static], - sources: generated_headers, + sources: generated_headers_stamp, dependencies: [os_deps, libintl], ) @@ -3184,7 +3186,7 @@ backend_code = declare_dependency( include_directories: [postgres_inc], link_args: ldflags_be, link_with: [], - sources: generated_headers + generated_backend_headers, + sources: [generated_backend_headers_stamp], dependencies: os_deps + backend_both_deps + backend_deps, ) -- 2.47.2 [text/x-patch] v4-0002-meson-Add-postgresql-extension.pc-for-building-ex.patch (4.5K, 3-v4-0002-meson-Add-postgresql-extension.pc-for-building-ex.patch) download | inline diff: From d453e2e219cd0323217699002b6db990a7421da2 Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Sat, 27 Aug 2022 09:52:03 -0700 Subject: [PATCH v4 2/7] meson: Add postgresql-extension.pc for building extension libraries This should work with several other buildsystems. TODO: Docs Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/backend/meson.build | 110 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 110 insertions(+) diff --git a/src/backend/meson.build b/src/backend/meson.build index 7fc649c3ebd..9d79d4d058c 100644 --- a/src/backend/meson.build +++ b/src/backend/meson.build @@ -188,6 +188,116 @@ pg_test_mod_args = pg_mod_args + { +############################################################### +# Define a .pc file that can be used to build server extensions +############################################################### + +pg_ext_vars = [] +pg_ext_vars_inst = [] +pg_ext_vars_uninst = [] + +pg_ext_cflags = pg_mod_c_args + cppflags +pg_ext_libs = [backend_mod_deps, thread_dep, ldflags, ldflags_mod] +pg_ext_subdirs = [''] + +# Compute directories to add include directories to the .pc files for. +# This is a bit more complicated due to port/win32 etc. +i = 0 +foreach incdir : postgres_inc_d + if fs.is_absolute(incdir) + # an absolute path from -Dextra_include_dirs + pg_ext_cflags += '-I@0@'.format(incdir) + continue + elif incdir.startswith('src/include') + subincdir = dir_include_pkg_rel / 'server' / incdir.split('src/include/').get(1, '') + else + subincdir = '' + endif + pg_ext_subdirs += subincdir + + # Add directories in source / build dir containing headers to cflags for the + # -uninstalled.pc. Older versions of pkg-config complain if a referenced + # variable is not defined, so we emit an empty one for the installed .pc + # file. + pg_ext_vars += [ + 'build_inc@0@=""'.format(i), + 'src_inc@0@=""'.format(i), + ] + pg_ext_vars_uninst += [ + 'build_inc@0@=-I${prefix}/@1@'.format(i, incdir), + 'src_inc@0@=-I${srcdir}/@1@'.format(i, incdir), + ] + pg_ext_cflags += [ + '${build_inc@0@}'.format(i), + '${src_inc@0@}'.format(i) + ] + + i += 1 +endforeach + + +# Extension modules should likely also use -fwrapv etc. But it it's a bit odd +# to expose it to a .pc file? +pg_ext_cflags_warn = pg_ext_cflags + cflags_warn +pg_ext_cflags += cflags + +# Directories for extensions to install into +# XXX: more needed +pg_ext_vars += 'pkglibdir=${prefix}/@0@'.format(dir_lib_pkg) +pg_ext_vars += 'dir_mod=${pkglibdir}' +pg_ext_vars += 'dir_data=${prefix}/@0@'.format(dir_data_extension) +pg_ext_vars += 'dir_include=${prefix}/@0@'.format(dir_include_extension) +pg_ext_vars += 'dir_doc=${prefix}/@0@'.format(dir_doc_extension) +pg_ext_vars += 'dir_bitcode=${prefix}/@0@'.format(dir_bitcode) +# referenced on some platforms, via mod_link_with_dir +pg_ext_vars += 'bindir=${prefix}/@0@'.format(dir_bin) + +# XXX: Define variables making it easy to define tests, too + +# Some platforms need linker flags to link with binary, they are the same +# between building with meson and .pc file, except that we have have to +# reference a variable to make it work for both normal and -uninstalled .pc +# files. +if mod_link_args_fmt.length() != 0 + assert(link_with_inst != '') + assert(link_with_uninst != '') + + pg_ext_vars_inst += 'mod_link_with=@0@'.format(link_with_inst) + pg_ext_vars_uninst += 'mod_link_with=@0@'.format(link_with_uninst) + + foreach el : mod_link_args_fmt + pg_ext_libs += el.format('${mod_link_with}') + endforeach +endif + +# main .pc to build extensions +pkgconfig.generate( + name: 'postgresql-extension', + description: 'PostgreSQL Extension Support', + url: pg_url, + + subdirs: pg_ext_subdirs, + libraries: pg_ext_libs, + extra_cflags: pg_ext_cflags, + + variables: pg_ext_vars + pg_ext_vars_inst, + uninstalled_variables: pg_ext_vars + pg_ext_vars_uninst, +) + +# a .pc depending on the above, but with all our warnings enabled +pkgconfig.generate( + name: 'postgresql-extension-warnings', + description: 'PostgreSQL Extension Support - Compiler Warnings', + requires: 'postgresql-extension', + url: pg_url, + extra_cflags: pg_ext_cflags_warn, + + variables: pg_ext_vars + pg_ext_vars_inst, + uninstalled_variables: pg_ext_vars + pg_ext_vars_uninst, +) + + + # Shared modules that, on some system, link against the server binary. Only # enter these after we defined the server build. -- 2.47.2 [text/x-patch] v4-0003-meson-Test-building-extensions-by-using-postgresq.patch (10.7K, 4-v4-0003-meson-Test-building-extensions-by-using-postgresq.patch) download | inline diff: From 2a55bf50b86e5872b05c259a84b4803f9c39b1a7 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Thu, 27 Feb 2025 17:45:31 +0300 Subject: [PATCH v4 3/7] meson: Test building extensions by using postgresql-extension.pc The 'test_meson_extensions' pyton wrapper is added to run these tests. It compiles and builds extensions at ${build}/testrun/meson_extensions/${extension_name} path. The tests for building amcheck, auth_delay and postgres_fdw extensions are added. These are also examples of how to build extensions by using postgresql-extension.pc. Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/test/modules/meson.build | 1 + .../modules/test_meson_extensions/meson.build | 3 + .../amcheck/meson.build | 28 ++++++++ .../auth_delay/meson.build | 17 +++++ .../test_pkg_config_extensions/meson.build | 24 +++++++ .../postgres_fdw/meson.build | 31 ++++++++ meson.build | 32 +++++++++ src/tools/ci/test_meson_extensions | 70 +++++++++++++++++++ 8 files changed, 206 insertions(+) create mode 100644 src/test/modules/test_meson_extensions/meson.build create mode 100644 src/test/modules/test_meson_extensions/test_pkg_config_extensions/amcheck/meson.build create mode 100644 src/test/modules/test_meson_extensions/test_pkg_config_extensions/auth_delay/meson.build create mode 100644 src/test/modules/test_meson_extensions/test_pkg_config_extensions/meson.build create mode 100644 src/test/modules/test_meson_extensions/test_pkg_config_extensions/postgres_fdw/meson.build create mode 100644 src/tools/ci/test_meson_extensions diff --git a/src/test/modules/meson.build b/src/test/modules/meson.build index 2b057451473..22ec5e969ae 100644 --- a/src/test/modules/meson.build +++ b/src/test/modules/meson.build @@ -25,6 +25,7 @@ subdir('test_ginpostinglist') subdir('test_integerset') subdir('test_json_parser') subdir('test_lfind') +subdir('test_meson_extensions') subdir('test_misc') subdir('test_oat_hooks') subdir('test_parser') diff --git a/src/test/modules/test_meson_extensions/meson.build b/src/test/modules/test_meson_extensions/meson.build new file mode 100644 index 00000000000..06cf5d555df --- /dev/null +++ b/src/test/modules/test_meson_extensions/meson.build @@ -0,0 +1,3 @@ +# Copyright (c) 2022-2025, PostgreSQL Global Development Group + +subdir('test_pkg_config_extensions') diff --git a/src/test/modules/test_meson_extensions/test_pkg_config_extensions/amcheck/meson.build b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/amcheck/meson.build new file mode 100644 index 00000000000..482a543eb86 --- /dev/null +++ b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/amcheck/meson.build @@ -0,0 +1,28 @@ +# Copyright (c) 2022-2025, PostgreSQL Global Development Group + +project('amcheck', 'c') + +amcheck_path = '../../../../../../contrib/amcheck/' + +amcheck_sources = files( + amcheck_path / 'verify_heapam.c', + amcheck_path / 'verify_nbtree.c', +) + +pg_ext = dependency('postgresql-extension-warnings') + +amcheck = shared_module('amcheck', + amcheck_sources, + dependencies: pg_ext, + install_dir: pg_ext.get_variable(pkgconfig: 'dir_mod'), +) + +install_data( + amcheck_path / 'amcheck.control', + amcheck_path / 'amcheck--1.0.sql', + amcheck_path / 'amcheck--1.0--1.1.sql', + amcheck_path / 'amcheck--1.1--1.2.sql', + amcheck_path / 'amcheck--1.2--1.3.sql', + amcheck_path / 'amcheck--1.3--1.4.sql', + install_dir: pg_ext.get_variable(pkgconfig: 'dir_data'), +) diff --git a/src/test/modules/test_meson_extensions/test_pkg_config_extensions/auth_delay/meson.build b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/auth_delay/meson.build new file mode 100644 index 00000000000..98ad24cc183 --- /dev/null +++ b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/auth_delay/meson.build @@ -0,0 +1,17 @@ +# Copyright (c) 2022-2025, PostgreSQL Global Development Group + +project('auth_delay', 'c') + +auth_delay_path = '../../../../../../contrib/auth_delay/' + +auth_delay_sources = files( + auth_delay_path / 'auth_delay.c', +) + +pg_ext = dependency('postgresql-extension-warnings') + +auth_delay = shared_module('auth_delay', + auth_delay_sources, + dependencies: pg_ext, + install_dir: pg_ext.get_variable(pkgconfig: 'dir_mod'), +) diff --git a/src/test/modules/test_meson_extensions/test_pkg_config_extensions/meson.build b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/meson.build new file mode 100644 index 00000000000..dae94a384a1 --- /dev/null +++ b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/meson.build @@ -0,0 +1,24 @@ +# Copyright (c) 2022-2025, PostgreSQL Global Development Group + +# pkgconfig is not available on Windows, so skip it. +if host_machine.system() == 'windows' + subdir_done() +endif + +meson_extension_tests += { + 'name': 'amcheck', + 'kind': 'pkg_config', + 'sd': meson.current_source_dir() / 'amcheck', +} + +meson_extension_tests += { + 'name': 'auth_delay', + 'kind': 'pkg_config', + 'sd': meson.current_source_dir() / 'auth_delay', +} + +meson_extension_tests += { + 'name': 'postgres_fdw', + 'kind': 'pkg_config', + 'sd': meson.current_source_dir() / 'postgres_fdw', +} diff --git a/src/test/modules/test_meson_extensions/test_pkg_config_extensions/postgres_fdw/meson.build b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/postgres_fdw/meson.build new file mode 100644 index 00000000000..d49625e0611 --- /dev/null +++ b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/postgres_fdw/meson.build @@ -0,0 +1,31 @@ +# Copyright (c) 2022-2025, PostgreSQL Global Development Group + +project('auth_delay', 'c') + +postgres_fdw_path = '../../../../../../contrib/postgres_fdw/' + +postgres_fdw_sources = files( + postgres_fdw_path / 'connection.c', + postgres_fdw_path / 'deparse.c', + postgres_fdw_path / 'option.c', + postgres_fdw_path / 'postgres_fdw.c', + postgres_fdw_path / 'shippable.c', +) + +pg_ext = dependency('postgresql-extension-warnings') +libpq = dependency('libpq') + +postgres_fdw = shared_module('postgres_fdw', + postgres_fdw_sources, + dependencies: [pg_ext, libpq], + install_dir: pg_ext.get_variable(pkgconfig: 'dir_mod'), +) + +install_data( + postgres_fdw_path / 'postgres_fdw.control', + postgres_fdw_path / 'postgres_fdw--1.0.sql', + postgres_fdw_path / 'postgres_fdw--1.0--1.1.sql', + postgres_fdw_path / 'postgres_fdw--1.1--1.2.sql', + install_dir: pg_ext.get_variable(pkgconfig: 'dir_data'), +) + diff --git a/meson.build b/meson.build index 9bfa96ad255..765b7f6a207 100644 --- a/meson.build +++ b/meson.build @@ -2891,6 +2891,7 @@ nls_targets = [] # Define the tests to distribute them to the correct test styles later test_deps = [] tests = [] +meson_extension_tests = [] # Default options for targets @@ -3443,6 +3444,37 @@ sys.exit(sp.returncode) suite: ['setup']) +# it seems freebsd doesn't use libdir for pkgconfig path +if host_system == 'freebsd' + pkgconf_installdir = dir_prefix / 'libdata' / 'pkgconfig' +else + pkgconf_installdir = dir_prefix / dir_lib / 'pkgconfig' +endif +test_pkg_conf_file = files('src/tools/ci/test_meson_extensions') + +foreach test : meson_extension_tests + if test['kind'] not in ['pkg_config'] + error('unknown kind @0@ of test in @1@'.format(test['kind'], test['sd'])) + endif + + test_group = 'meson_@0@_extensions'.format(test['kind']) + + test(test_group / test['name'], + test_pkg_conf_file, + args: [ + '--meson', meson_bin.path(), + '--meson_args', meson_args, + '--test_dir', test['sd'], + '--test_out_dir', test_result_dir / 'meson_extensions' / test['name'], + '--builddir', meson.build_root(), + '--pkg_conf_path', get_option('pkg_config_path'), + '--', + cc.cmd_array(), + ], + suite: test_group, + ) + +endforeach ############################################################### # Test Generation diff --git a/src/tools/ci/test_meson_extensions b/src/tools/ci/test_meson_extensions new file mode 100644 index 00000000000..50358121f49 --- /dev/null +++ b/src/tools/ci/test_meson_extensions @@ -0,0 +1,70 @@ +#!/usr/bin/env python3 + +import argparse +import os +import shutil +import subprocess + +parser = argparse.ArgumentParser() + +parser.add_argument('--meson', help='path to meson binary', + type=str, required=True) +parser.add_argument('--meson_args', help='args of meson binary', + type=str, nargs='*', required=False) +parser.add_argument('--test_dir', help='test source directory', + type=str, required=True) +parser.add_argument('--test_out_dir', help='test output directory', + type=str, required=True) +parser.add_argument('--builddir', help='meson build directory', + type=str, required=True) +parser.add_argument('--pkg_conf_path', + help='PKG_CONF_PATH from surrounding meson build', + type=str, nargs='?', const='', required=False) +parser.add_argument('c_args', help='c_args from surrounding meson build', + nargs='*') + +args = parser.parse_args() + +meson_bin = args.meson +meson_args = ' '.join(args.meson_args) +test_source_dir = args.test_dir +test_out_dir = args.test_out_dir +build_dir = args.builddir +pkg_conf_path = args.pkg_conf_path +c_args = ' '.join(args.c_args) + +exit_code = 0 + +def remove_duplicates(duplicate_str): + words = duplicate_str.split() + return ' '.join(sorted(set(words), key=words.index)) + + +def run_tests(pkg_conf_path_local, message): + print('\n{}\n{}\n'.format('#' * 60, message), flush=True) + + env = {**os.environ, } + env['PKG_CONFIG_PATH'] = '{}:{}:{}'.format( + pkg_conf_path_local, pkg_conf_path, env.get('PKG_CONFIG_PATH', ''), + ).strip(': ') + env['CC'] = '{} {}'.format( + c_args, env.get('CC', ''), + ) + env['CC'] = remove_duplicates(env['CC']) + + # Clear the build directory beforehand. + if os.path.exists(test_out_dir): + shutil.rmtree(test_out_dir) + + if meson_args: + meson_setup_command = [meson_bin, meson_args, 'setup', test_out_dir] + else: + meson_setup_command = [meson_bin, 'setup', test_out_dir] + + meson_compile_command = ['meson', 'compile', '-C', test_out_dir, '-v'] + + subprocess.run(meson_setup_command, env=env, cwd=test_source_dir, check=True) + subprocess.run(meson_compile_command, cwd=test_source_dir, check=True) + +run_tests(os.path.join(build_dir, 'meson-uninstalled'), + message='Testing postgresql-extension-warnings-uninstalled') -- 2.47.2 [text/x-patch] v4-0004-meson-WIP-Add-docs-for-postgresql-extension.pc.patch (12.3K, 5-v4-0004-meson-WIP-Add-docs-for-postgresql-extension.pc.patch) download | inline diff: From 04ac04f711231b6189219389b27cfb4853df0d6a Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Thu, 6 Mar 2025 17:46:57 +0300 Subject: [PATCH v4 4/7] meson: [WIP] Add docs for postgresql-extension.pc Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- doc/src/sgml/acronyms.sgml | 2 +- doc/src/sgml/extend.sgml | 101 +++++++++++++++++++++++-------------- doc/src/sgml/jit.sgml | 2 +- 3 files changed, 66 insertions(+), 39 deletions(-) diff --git a/doc/src/sgml/acronyms.sgml b/doc/src/sgml/acronyms.sgml index 58d0d90fece..240db5cb827 100644 --- a/doc/src/sgml/acronyms.sgml +++ b/doc/src/sgml/acronyms.sgml @@ -561,7 +561,7 @@ <term><acronym>PGXS</acronym></term> <listitem> <para> - <link linkend="extend-pgxs"><productname>PostgreSQL</productname> Extension System</link> + <link linkend="extend-postgres-pgxs"><productname>PostgreSQL</productname> Extension System</link> </para> </listitem> </varlistentry> diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml index ba492ca27c0..4a8c19b3f53 100644 --- a/doc/src/sgml/extend.sgml +++ b/doc/src/sgml/extend.sgml @@ -1421,7 +1421,7 @@ include $(PGXS) </programlisting> This makefile relies on <acronym>PGXS</acronym>, which is described - in <xref linkend="extend-pgxs"/>. The command <literal>make install</literal> + in <xref linkend="extend-postgres-pgxs"/>. The command <literal>make install</literal> will install the control and script files into the correct directory as reported by <application>pg_config</application>. </para> @@ -1434,21 +1434,26 @@ include $(PGXS) </sect2> </sect1> - <sect1 id="extend-pgxs"> + <sect1 id="extend-postgres"> <title>Extension Building Infrastructure</title> - <indexterm zone="extend-pgxs"> - <primary>pgxs</primary> - </indexterm> - <para> If you are thinking about distributing your <productname>PostgreSQL</productname> extension modules, setting up a portable build system for them can be fairly difficult. Therefore the <productname>PostgreSQL</productname> installation provides a build - infrastructure for extensions, called <acronym>PGXS</acronym>, so - that simple extension modules can be built simply against an - already installed server. <acronym>PGXS</acronym> is mainly intended + infrastructure for extensions, called <literal>PGXS</literal> + (<xref linkend="extend-postgres-pgxs"/>) and + its meson counterpart <literal>postgresql-extension.pc</literal> + (<xref linkend="extend-postgres-meson"/>). + </para> + + </sect1> + + <sect1 id="extend-postgres-pgxs"> + <title>PGXS</title> + + <para> <acronym>PGXS</acronym> is mainly intended for extensions that include C code, although it can be used for pure-SQL extensions too. Note that <acronym>PGXS</acronym> is not intended to be a universal build system framework that can be used @@ -1488,7 +1493,7 @@ include $(PGXS) Set one of these three variables to specify what is built: <variablelist> - <varlistentry id="extend-pgxs-modules"> + <varlistentry id="extend-postgres-pgxs-modules"> <term><varname>MODULES</varname></term> <listitem> <para> @@ -1498,7 +1503,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-module-big"> + <varlistentry id="extend-postgres-pgxs-module-big"> <term><varname>MODULE_big</varname></term> <listitem> <para> @@ -1508,7 +1513,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-program"> + <varlistentry id="extend-postgres-pgxs-program"> <term><varname>PROGRAM</varname></term> <listitem> <para> @@ -1522,7 +1527,7 @@ include $(PGXS) The following variables can also be set: <variablelist> - <varlistentry id="extend-pgxs-extension"> + <varlistentry id="extend-postgres-pgxs-extension"> <term><varname>EXTENSION</varname></term> <listitem> <para> @@ -1534,7 +1539,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-moduledir"> + <varlistentry id="extend-postgres-pgxs-moduledir"> <term><varname>MODULEDIR</varname></term> <listitem> <para> @@ -1547,7 +1552,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-data"> + <varlistentry id="extend-postgres-pgxs-data"> <term><varname>DATA</varname></term> <listitem> <para> @@ -1556,7 +1561,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-data-built"> + <varlistentry id="extend-postgres-pgxs-data-built"> <term><varname>DATA_built</varname></term> <listitem> <para> @@ -1567,7 +1572,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-data-tsearch"> + <varlistentry id="extend-postgres-pgxs-data-tsearch"> <term><varname>DATA_TSEARCH</varname></term> <listitem> <para> @@ -1577,7 +1582,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-docs"> + <varlistentry id="extend-postgres-pgxs-docs"> <term><varname>DOCS</varname></term> <listitem> <para> @@ -1587,7 +1592,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-headers"> + <varlistentry id="extend-postgres-pgxs-headers"> <term><varname>HEADERS</varname></term> <term><varname>HEADERS_built</varname></term> <listitem> @@ -1603,7 +1608,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-headers-module"> + <varlistentry id="extend-postgres-pgxs-headers-module"> <term><varname>HEADERS_$MODULE</varname></term> <term><varname>HEADERS_built_$MODULE</varname></term> <listitem> @@ -1629,7 +1634,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-scripts"> + <varlistentry id="extend-postgres-pgxs-scripts"> <term><varname>SCRIPTS</varname></term> <listitem> <para> @@ -1639,7 +1644,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-scripts-built"> + <varlistentry id="extend-postgres-pgxs-scripts-built"> <term><varname>SCRIPTS_built</varname></term> <listitem> <para> @@ -1650,7 +1655,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-regress"> + <varlistentry id="extend-postgres-pgxs-regress"> <term><varname>REGRESS</varname></term> <listitem> <para> @@ -1659,7 +1664,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-regress-opts"> + <varlistentry id="extend-postgres-pgxs-regress-opts"> <term><varname>REGRESS_OPTS</varname></term> <listitem> <para> @@ -1668,7 +1673,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-isolation"> + <varlistentry id="extend-postgres-pgxs-isolation"> <term><varname>ISOLATION</varname></term> <listitem> <para> @@ -1677,7 +1682,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-isolation-opts"> + <varlistentry id="extend-postgres-pgxs-isolation-opts"> <term><varname>ISOLATION_OPTS</varname></term> <listitem> <para> @@ -1687,7 +1692,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-tap-tests"> + <varlistentry id="extend-postgres-pgxs-tap-tests"> <term><varname>TAP_TESTS</varname></term> <listitem> <para> @@ -1696,7 +1701,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-no-install"> + <varlistentry id="extend-postgres-pgxs-no-install"> <term><varname>NO_INSTALL</varname></term> <listitem> <para> @@ -1706,7 +1711,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-no-installcheck"> + <varlistentry id="extend-postgres-pgxs-no-installcheck"> <term><varname>NO_INSTALLCHECK</varname></term> <listitem> <para> @@ -1715,7 +1720,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-extra-clean"> + <varlistentry id="extend-postgres-pgxs-extra-clean"> <term><varname>EXTRA_CLEAN</varname></term> <listitem> <para> @@ -1724,7 +1729,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-cppflags"> + <varlistentry id="extend-postgres-pgxs-pg-cppflags"> <term><varname>PG_CPPFLAGS</varname></term> <listitem> <para> @@ -1733,7 +1738,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-cflags"> + <varlistentry id="extend-postgres-pgxs-pg-cflags"> <term><varname>PG_CFLAGS</varname></term> <listitem> <para> @@ -1742,7 +1747,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-cxxflags"> + <varlistentry id="extend-postgres-pgxs-pg-cxxflags"> <term><varname>PG_CXXFLAGS</varname></term> <listitem> <para> @@ -1751,7 +1756,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-ldflags"> + <varlistentry id="extend-postgres-pgxs-pg-ldflags"> <term><varname>PG_LDFLAGS</varname></term> <listitem> <para> @@ -1760,7 +1765,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-libs"> + <varlistentry id="extend-postgres-pgxs-pg-libs"> <term><varname>PG_LIBS</varname></term> <listitem> <para> @@ -1769,7 +1774,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-shlib-link"> + <varlistentry id="extend-postgres-pgxs-shlib-link"> <term><varname>SHLIB_LINK</varname></term> <listitem> <para> @@ -1778,7 +1783,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-config"> + <varlistentry id="extend-postgres-pgxs-pg-config"> <term><varname>PG_CONFIG</varname></term> <listitem> <para> @@ -1894,4 +1899,26 @@ make VPATH=/path/to/extension/source/tree install </tip> </sect1> + <sect1 id="extend-postgres-meson"> + <title>postgresql-extension.pc</title> + + <para> + When Postgres is built by using meson, it generates + <literal>postgresql-extension.pc</literal> pkg-config file. Extension + libraries can use this file like <literal>PGXS</literal> + (<xref linkend="extend-postgres-pgxs"/>). + + To use the <literal>postgresql-extension.pc</literal> infrastructure for + your extension, you must write a simple meson.build file. In the + meson.build file, you need to include the + <literal>postgresql-extension.pc</literal> pkg-config file. Here is an + example that builds an extension module named isbn_issn, consisting of a + shared library containing some C code, an extension control file, an SQL + script, an include file (only needed if other modules might need to access + the extension functions without going via SQL), and a documentation text + file: + </para> + + </sect1> + </chapter> diff --git a/doc/src/sgml/jit.sgml b/doc/src/sgml/jit.sgml index 44e18bf1a6f..81a4644a97d 100644 --- a/doc/src/sgml/jit.sgml +++ b/doc/src/sgml/jit.sgml @@ -223,7 +223,7 @@ SET of types <literal>C</literal> and <literal>internal</literal>, as well as operators based on such functions. To do so for functions in extensions, the definitions of those functions need to be made available. - When using <link linkend="extend-pgxs">PGXS</link> to build an extension + When using <link linkend="extend-postgres-pgxs">PGXS</link> to build an extension against a server that has been compiled with LLVM JIT support, the relevant files will be built and installed automatically. </para> -- 2.47.2 [text/x-patch] v4-0005-meson-Add-architecture-for-LLVM-bitcode-emission.patch (9.0K, 6-v4-0005-meson-Add-architecture-for-LLVM-bitcode-emission.patch) download | inline diff: From 455378f915bc04c9ee2b629500fd47bf7ea7b83d Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Fri, 7 Mar 2025 12:10:58 +0300 Subject: [PATCH v4 5/7] meson: Add architecture for LLVM bitcode emission This commit adds suport for bitcode emission for both normal and generated source files (processed by bison, flex, etc). These bitcode files are installed into $pkglibdir/bitcode/ directory if the LLVM is found. New variable `bitcode_modules` is introduced to generate bitcode files. All required information is gathered in this variable. Then, this variable is processed by the main meson LLVM bitcode emission scripts: src/backend/jit/llvm/bitcode/meson.build -> src/tools/irlink. An example of a possible structure of bitcode_modules is: ``` bitcode_modules = [ { 'name': '...', 'target': ..., 'srcfiles': [ '...', '...', ], 'additional_flags': [ '-I...', '-I...', ], 'gen_srcfiles': [ { 'srcfiles': [ <custom_target for ...>, <custom_target for ...>, ], 'additional_flags': [ '-I...', '-I...', ] } ] } ] ``` Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Author: Diego Fronza <[email protected]> Reviewed-by: Diego Fronza <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/backend/jit/llvm/bitcode/meson.build | 69 ++++++++++++++++++++++++ src/backend/jit/llvm/meson.build | 31 +++++++---- src/backend/meson.build | 6 +++ meson.build | 21 ++++++++ src/tools/irlink | 25 +++++++++ 5 files changed, 141 insertions(+), 11 deletions(-) create mode 100644 src/backend/jit/llvm/bitcode/meson.build create mode 100644 src/tools/irlink diff --git a/src/backend/jit/llvm/bitcode/meson.build b/src/backend/jit/llvm/bitcode/meson.build new file mode 100644 index 00000000000..546e4d8b898 --- /dev/null +++ b/src/backend/jit/llvm/bitcode/meson.build @@ -0,0 +1,69 @@ +# Copyright (c) 2022-2024, PostgreSQL Global Development Group +# +# emit LLVM bitcode for JIT inlining + +assert(llvm.found()) + +foreach bitcode_module : bitcode_modules + bitcode_targets = [] + bitcode_obj = bitcode_module['target'] + bitcode_cflags_local = bitcode_cflags + bitcode_module.get('additional_flags', []) + bitcode_name = bitcode_module.get('name', bitcode_obj.name()) + + foreach srcfile : bitcode_module['srcfiles'] + if meson.version().version_compare('>=0.59') + srcfilename = fs.parent(srcfile) / fs.name(srcfile) + else + srcfilename = '@0@'.format(srcfile) + endif + + targetname = '@0@_@[email protected]'.format( + bitcode_name, + srcfilename.underscorify(), + ) + bitcode_targets += custom_target( + targetname, + depends: [bitcode_obj], + input: [srcfile], + output: targetname, + command: [llvm_irgen_command, bitcode_cflags_local], + install: true, + install_dir: dir_bitcode, + ) + endforeach + + # Process generated sources, which may include custom compilation flags. + foreach gen_sources: bitcode_module.get('gen_sources', []) + bitcode_cflags_gen_local = bitcode_cflags_local + gen_sources.get('additional_flags', []) + + foreach srcfile: gen_sources['srcfiles'] + # Generated sources are stored in some folder under meson.build_root()/**, + # remove the build prefix from the string. + srcfilename = srcfile.full_path().split(meson.build_root() + '/')[1] + + targetname = '@0@_@[email protected]'.format( + bitcode_name, + srcfilename.underscorify(), + ) + bitcode_targets += custom_target( + targetname, + depends: [bitcode_obj], + input: [srcfile], + output: targetname, + command: [llvm_irgen_command, bitcode_cflags_gen_local], + install: true, + install_dir: dir_bitcode, + ) + endforeach + endforeach + + index_name = '@[email protected]'.format(bitcode_name) + bitcode_index = custom_target('@0@'.format(bitcode_name), + output: index_name, + input: bitcode_targets, + command: [irlink, '--lto', llvm_lto, '--outdir', '@OUTDIR@', '--index', index_name, '@INPUT@'], + install: true, + install_dir: dir_bitcode, + ) + backend_targets += bitcode_index +endforeach diff --git a/src/backend/jit/llvm/meson.build b/src/backend/jit/llvm/meson.build index c8e06dfbe35..2dd922e573b 100644 --- a/src/backend/jit/llvm/meson.build +++ b/src/backend/jit/llvm/meson.build @@ -42,21 +42,22 @@ backend_targets += llvmjit # Define a few bits and pieces used here and elsewhere to generate bitcode -llvm_irgen_args = [ - '-c', '-o', '@OUTPUT@', '@INPUT@', +llvm_irgen_command = [] +if ccache.found() + llvm_irgen_command += ccache +endif + +llvm_irgen_command += [ + clang, + '-c', '-o', '@OUTPUT0@', '@INPUT0@', '-flto=thin', '-emit-llvm', - '-MD', '-MQ', '@OUTPUT@', '-MF', '@DEPFILE@', '-O2', '-Wno-ignored-attributes', '-Wno-empty-body', + '-Wno-unknown-warning-option', + '-Wno-compound-token-split-by-macro', ] - -if ccache.found() - llvm_irgen_command = ccache - llvm_irgen_args = [clang.path()] + llvm_irgen_args -else - llvm_irgen_command = clang -endif +llvm_irgen_dep_args = ['-MD', '-MQ', '@OUTPUT0@', '-MF', '@DEPFILE@'] # XXX: Need to determine proper version of the function cflags for clang @@ -73,7 +74,7 @@ bitcode_cflags += '-I@SOURCE_ROOT@/src/include' # Note this is intentionally not installed to bitcodedir, as it's not for # inlining llvmjit_types = custom_target('llvmjit_types.bc', - command: [llvm_irgen_command] + llvm_irgen_args + bitcode_cflags, + command: llvm_irgen_command + llvm_irgen_dep_args + bitcode_cflags, input: 'llvmjit_types.c', output: 'llvmjit_types.bc', depends: [postgres], @@ -82,3 +83,11 @@ llvmjit_types = custom_target('llvmjit_types.bc', depfile: '@[email protected]', ) backend_targets += llvmjit_types + +# Figure out -I's needed to build all postgres code, including all its +# dependencies +pkg_config = find_program(['pkg-config', 'pkgconf'], required: true) +r = run_command(pkg_config, + ['--cflags-only-I', meson.build_root() / 'meson-uninstalled/postgresql-extension-uninstalled.pc'], + check: true) +bitcode_cflags += r.stdout().split() diff --git a/src/backend/meson.build b/src/backend/meson.build index 9d79d4d058c..5fb33660d3d 100644 --- a/src/backend/meson.build +++ b/src/backend/meson.build @@ -140,6 +140,12 @@ postgres = executable('postgres', backend_targets += postgres +bitcode_modules += { + 'name': 'postgres', + 'target': postgres_lib, + 'srcfiles': backend_sources, +} + pg_mod_c_args = cflags_mod pg_mod_cpp_args = cxxflags_mod pg_mod_link_args = ldflags_sl + ldflags_mod diff --git a/meson.build b/meson.build index 765b7f6a207..a01dbca66d1 100644 --- a/meson.build +++ b/meson.build @@ -814,6 +814,8 @@ if add_languages('cpp', required: llvmopt, native: false) # Some distros put LLVM and clang in different paths, so fallback to # find via PATH, too. clang = find_program(llvm_binpath / 'clang', 'clang', required: true) + llvm_lto = find_program(llvm_binpath / 'llvm-lto', required: true) + irlink = find_program('src/tools/irlink', native: true) endif elif llvmopt.auto() message('llvm requires a C++ compiler') @@ -2893,6 +2895,11 @@ test_deps = [] tests = [] meson_extension_tests = [] +# List of object files + source files to generated LLVM IR for inlining. +# Each element is a hash of: +# {'target': target, 'srcfiles': ..., 'additional_flags': ...}. +bitcode_modules = [] + # Default options for targets @@ -3212,6 +3219,11 @@ subdir('src/interfaces/ecpg/test') subdir('doc/src/sgml') +# generate bitcode for JIT inlining after giving contrib modules etc a chance +# to add themselves to bitcode_modules[] +subdir('src/backend/jit/llvm/bitcode', if_found: llvm) + + generated_sources_ac += {'': ['GNUmakefile']} # After processing src/test, add test_install_libs to the testprep_targets @@ -3847,6 +3859,15 @@ if meson.version().version_compare('>=0.57') section: 'Programs', ) + if llvm.found() + summary( + { + 'clang': clang, + }, + section: 'Programs', + ) + endif + summary( { 'bonjour': bonjour, diff --git a/src/tools/irlink b/src/tools/irlink new file mode 100644 index 00000000000..793c0abf91a --- /dev/null +++ b/src/tools/irlink @@ -0,0 +1,25 @@ +#!/usr/bin/env python3 + +import argparse +import os +import shutil +import subprocess +import sys + +parser = argparse.ArgumentParser( + description='generate PostgreSQL JIT IR module') + +parser.add_argument('--index', type=str, required=True) +parser.add_argument('--lto', type=str, required=True) +parser.add_argument('--outdir', type=str, required=True) +parser.add_argument('INPUT', type=str, nargs='+') + +args = parser.parse_args() + +file_names = [os.path.basename(f) for f in args.INPUT] +command = [args.lto, + '-thinlto', '-thinlto-action=thinlink', + '-o', args.index] + file_names +res = subprocess.run(command, cwd=args.outdir) + +exit(res.returncode) -- 2.47.2 [text/x-patch] v4-0006-meson-Add-LLVM-bitcode-emissions-for-contrib-libr.patch (22.6K, 7-v4-0006-meson-Add-LLVM-bitcode-emissions-for-contrib-libr.patch) download | inline diff: From 9d4ba41fa305027fd5a18e284894b5444182756f Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Wed, 12 Mar 2025 10:49:18 +0300 Subject: [PATCH v4 6/7] meson: Add LLVM bitcode emissions for contrib libraries The libraries which the bitcode files will be generated in are selected manually. Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Author: Diego Fronza <[email protected]> Reviewed-by: Diego Fronza <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/interfaces/libpq/meson.build | 3 +++ contrib/bloom/meson.build | 5 +++++ contrib/bool_plperl/meson.build | 9 +++++++++ contrib/btree_gin/meson.build | 5 +++++ contrib/btree_gist/meson.build | 5 +++++ contrib/citext/meson.build | 5 +++++ contrib/cube/meson.build | 13 +++++++++++++ contrib/dict_int/meson.build | 5 +++++ contrib/dict_xsyn/meson.build | 5 +++++ contrib/earthdistance/meson.build | 6 ++++++ contrib/fuzzystrmatch/meson.build | 9 +++++++++ contrib/hstore/meson.build | 7 +++++++ contrib/hstore_plperl/meson.build | 10 ++++++++++ contrib/hstore_plpython/meson.build | 12 ++++++++++++ contrib/intarray/meson.build | 5 +++++ contrib/isn/meson.build | 5 +++++ contrib/jsonb_plperl/meson.build | 8 ++++++++ contrib/jsonb_plpython/meson.build | 10 ++++++++++ contrib/lo/meson.build | 5 +++++ contrib/ltree/meson.build | 10 ++++++++++ contrib/ltree_plpython/meson.build | 11 +++++++++++ contrib/pg_buffercache/meson.build | 5 +++++ contrib/pg_freespacemap/meson.build | 5 +++++ contrib/pg_logicalinspect/meson.build | 5 +++++ contrib/pg_surgery/meson.build | 4 ++++ contrib/pg_trgm/meson.build | 5 +++++ contrib/pgcrypto/meson.build | 4 ++++ contrib/seg/meson.build | 12 ++++++++++++ contrib/spi/meson.build | 20 ++++++++++++++++++++ contrib/sslinfo/meson.build | 5 +++++ contrib/tablefunc/meson.build | 5 +++++ contrib/tcn/meson.build | 5 +++++ contrib/tsm_system_rows/meson.build | 5 +++++ contrib/tsm_system_time/meson.build | 5 +++++ contrib/unaccent/meson.build | 5 +++++ contrib/uuid-ossp/meson.build | 5 +++++ contrib/xml2/meson.build | 5 +++++ src/pl/plperl/meson.build | 1 + src/pl/plpython/meson.build | 1 + meson.build | 1 + 40 files changed, 256 insertions(+) diff --git a/src/interfaces/libpq/meson.build b/src/interfaces/libpq/meson.build index 19f4a52a97a..4742489e0ce 100644 --- a/src/interfaces/libpq/meson.build +++ b/src/interfaces/libpq/meson.build @@ -50,6 +50,9 @@ export_file = custom_target('libpq.exports', libpq_inc = include_directories('.', '../../port') libpq_c_args = ['-DSO_MAJOR_VERSION=5'] +# libpq-fe.h is needed to generate bitcode for some extensions +libpq_dir = meson.current_source_dir() + # Not using both_libraries() here as # 1) resource files should only be in the shared library # 2) we want the .pc file to include a dependency to {pgport,common}_static for diff --git a/contrib/bloom/meson.build b/contrib/bloom/meson.build index 695712a455e..1090c281532 100644 --- a/contrib/bloom/meson.build +++ b/contrib/bloom/meson.build @@ -28,6 +28,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': bloom, + 'srcfiles': bloom_sources, +} + tests += { 'name': 'bloom', 'sd': meson.current_source_dir(), diff --git a/contrib/bool_plperl/meson.build b/contrib/bool_plperl/meson.build index f489d99044c..1758adb7489 100644 --- a/contrib/bool_plperl/meson.build +++ b/contrib/bool_plperl/meson.build @@ -37,6 +37,15 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': bool_plperl, + 'srcfiles': bool_plperl_sources, + 'additional_flags': [ + '-I@0@'.format(plperl_dir), + perl_ccflags + ] +} + tests += { 'name': 'bool_plperl', 'sd': meson.current_source_dir(), diff --git a/contrib/btree_gin/meson.build b/contrib/btree_gin/meson.build index b2749f6e669..26d49d005ab 100644 --- a/contrib/btree_gin/meson.build +++ b/contrib/btree_gin/meson.build @@ -25,6 +25,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': btree_gin, + 'srcfiles': btree_gin_sources, +} + tests += { 'name': 'btree_gin', 'sd': meson.current_source_dir(), diff --git a/contrib/btree_gist/meson.build b/contrib/btree_gist/meson.build index f4fa9574f1f..2649d5a7ffb 100644 --- a/contrib/btree_gist/meson.build +++ b/contrib/btree_gist/meson.build @@ -54,6 +54,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': btree_gist, + 'srcfiles': btree_gist_sources, +} + tests += { 'name': 'btree_gist', 'sd': meson.current_source_dir(), diff --git a/contrib/citext/meson.build b/contrib/citext/meson.build index 7fff34f2368..fa07ff72be4 100644 --- a/contrib/citext/meson.build +++ b/contrib/citext/meson.build @@ -30,6 +30,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': citext, + 'srcfiles': citext_sources, +} + tests += { 'name': 'citext', 'sd': meson.current_source_dir(), diff --git a/contrib/cube/meson.build b/contrib/cube/meson.build index fd3c057f469..a7649a74a1f 100644 --- a/contrib/cube/meson.build +++ b/contrib/cube/meson.build @@ -3,6 +3,7 @@ cube_sources = files( 'cube.c', ) +bc_cube_sources = cube_sources cube_scan = custom_target('cubescan', input: 'cubescan.l', @@ -11,6 +12,7 @@ cube_scan = custom_target('cubescan', ) generated_sources += cube_scan cube_sources += cube_scan +bc_cube_gen_sources = [{'srcfiles': [cube_scan]}] cube_parse = custom_target('cubeparse', input: 'cubeparse.y', @@ -18,6 +20,7 @@ cube_parse = custom_target('cubeparse', ) generated_sources += cube_parse.to_list() cube_sources += cube_parse +bc_cube_gen_sources += {'srcfiles': [cube_parse[0]]} if host_system == 'windows' cube_sources += rc_lib_gen.process(win32ver_rc, extra_args: [ @@ -48,6 +51,16 @@ install_headers( install_dir: dir_include_extension / 'cube', ) +cube_dir = meson.current_source_dir() +bitcode_modules += { + 'target': cube, + 'srcfiles': bc_cube_sources, + 'gen_sources': bc_cube_gen_sources, + 'additional_flags': [ + '-I@0@'.format(cube_dir), + ] +} + tests += { 'name': 'cube', 'sd': meson.current_source_dir(), diff --git a/contrib/dict_int/meson.build b/contrib/dict_int/meson.build index ab41588547b..894c8a9c94d 100644 --- a/contrib/dict_int/meson.build +++ b/contrib/dict_int/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': dict_int, + 'srcfiles': dict_int_sources, +} + tests += { 'name': 'dict_int', 'sd': meson.current_source_dir(), diff --git a/contrib/dict_xsyn/meson.build b/contrib/dict_xsyn/meson.build index 93f41b1f963..d4bd4a6ef5a 100644 --- a/contrib/dict_xsyn/meson.build +++ b/contrib/dict_xsyn/meson.build @@ -29,6 +29,11 @@ install_data( } ) +bitcode_modules += { + 'target': dict_xsyn, + 'srcfiles': dict_xsyn_sources, +} + tests += { 'name': 'dict_xsyn', 'sd': meson.current_source_dir(), diff --git a/contrib/earthdistance/meson.build b/contrib/earthdistance/meson.build index a5f8d1a3ffa..c101359b118 100644 --- a/contrib/earthdistance/meson.build +++ b/contrib/earthdistance/meson.build @@ -24,6 +24,12 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': earthdistance, + 'srcfiles': earthdistance_sources, +} + + tests += { 'name': 'earthdistance', 'sd': meson.current_source_dir(), diff --git a/contrib/fuzzystrmatch/meson.build b/contrib/fuzzystrmatch/meson.build index f52daa4ea1c..fc5e88d0c92 100644 --- a/contrib/fuzzystrmatch/meson.build +++ b/contrib/fuzzystrmatch/meson.build @@ -5,6 +5,7 @@ fuzzystrmatch_sources = files( 'dmetaphone.c', 'fuzzystrmatch.c', ) +bc_fuzzystrmatch_sources = fuzzystrmatch_sources daitch_mokotoff_h = custom_target('daitch_mokotoff', input: 'daitch_mokotoff_header.pl', @@ -35,6 +36,14 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': fuzzystrmatch, + 'srcfiles': bc_fuzzystrmatch_sources, + 'additional_flags': [ + '-I@0@'.format(meson.current_build_dir()) + ] +} + tests += { 'name': 'fuzzystrmatch', 'sd': meson.current_source_dir(), diff --git a/contrib/hstore/meson.build b/contrib/hstore/meson.build index 39622d1024d..344a620273f 100644 --- a/contrib/hstore/meson.build +++ b/contrib/hstore/meson.build @@ -43,6 +43,13 @@ install_headers( install_dir: dir_include_extension / 'hstore', ) +# some libraries include "hstore/hstore.h" instead of "hstore.h" +hstore_dir_up = join_paths(meson.current_source_dir(), '..') +bitcode_modules += { + 'target': hstore, + 'srcfiles': hstore_sources, +} + tests += { 'name': 'hstore', 'sd': meson.current_source_dir(), diff --git a/contrib/hstore_plperl/meson.build b/contrib/hstore_plperl/meson.build index 60b8ad23569..b689b55e1c9 100644 --- a/contrib/hstore_plperl/meson.build +++ b/contrib/hstore_plperl/meson.build @@ -37,6 +37,16 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': hstore_plperl, + 'srcfiles': hstore_plperl_sources, + 'additional_flags': [ + '-I@0@'.format(hstore_dir_up), + '-I@0@'.format(plperl_dir), + perl_ccflags, + ] +} + tests += { 'name': 'hstore_plperl', 'sd': meson.current_source_dir(), diff --git a/contrib/hstore_plpython/meson.build b/contrib/hstore_plpython/meson.build index ea8e20a377b..d68de3dc46b 100644 --- a/contrib/hstore_plpython/meson.build +++ b/contrib/hstore_plpython/meson.build @@ -30,6 +30,18 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': hstore_plpython, + 'srcfiles': hstore_plpython_sources, + 'additional_flags': [ + '-I@0@'.format(hstore_dir_up), + '-DPLPYTHON_LIBNAME="plpython3"', + '-I@0@'.format(python3_inc_dir), + '-I@0@'.format(plpython_dir), + perl_ccflags, + ] +} + hstore_plpython_regress = [ 'hstore_plpython' ] diff --git a/contrib/intarray/meson.build b/contrib/intarray/meson.build index fae9add981d..3d7da0c9c9a 100644 --- a/contrib/intarray/meson.build +++ b/contrib/intarray/meson.build @@ -33,6 +33,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': intarray, + 'srcfiles': intarray_sources, +} + tests += { 'name': 'intarray', 'sd': meson.current_source_dir(), diff --git a/contrib/isn/meson.build b/contrib/isn/meson.build index fbbeeff01bb..d42473134ea 100644 --- a/contrib/isn/meson.build +++ b/contrib/isn/meson.build @@ -29,6 +29,11 @@ install_headers( install_dir: dir_include_extension / 'isn', ) +bitcode_modules += { + 'target': isn, + 'srcfiles': isn_sources, +} + tests += { 'name': 'isn', 'sd': meson.current_source_dir(), diff --git a/contrib/jsonb_plperl/meson.build b/contrib/jsonb_plperl/meson.build index 95a9a7bc082..febafc73a99 100644 --- a/contrib/jsonb_plperl/meson.build +++ b/contrib/jsonb_plperl/meson.build @@ -37,6 +37,14 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': jsonb_plperl, + 'srcfiles': jsonb_plperl_sources, + 'additional_flags': [ + '-I@0@'.format(plperl_dir), + perl_ccflags, + ] +} tests += { 'name': 'jsonb_plperl', diff --git a/contrib/jsonb_plpython/meson.build b/contrib/jsonb_plpython/meson.build index 5fe80483e58..6be92221272 100644 --- a/contrib/jsonb_plpython/meson.build +++ b/contrib/jsonb_plpython/meson.build @@ -30,6 +30,16 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': jsonb_plpython, + 'srcfiles': jsonb_plpython_sources, + 'additional_flags': [ + '-DPLPYTHON_LIBNAME="plpython3"', + '-I@0@'.format(python3_inc_dir), + '-I@0@'.format(plpython_dir), + ] +} + jsonb_plpython_regress = [ 'jsonb_plpython' ] diff --git a/contrib/lo/meson.build b/contrib/lo/meson.build index 2d78907ba12..1222faa9169 100644 --- a/contrib/lo/meson.build +++ b/contrib/lo/meson.build @@ -24,6 +24,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': lo, + 'srcfiles': lo_sources, +} + tests += { 'name': 'lo', 'sd': meson.current_source_dir(), diff --git a/contrib/ltree/meson.build b/contrib/ltree/meson.build index f9b06302839..cda86935544 100644 --- a/contrib/ltree/meson.build +++ b/contrib/ltree/meson.build @@ -41,6 +41,16 @@ install_headers( install_dir: dir_include_extension / 'ltree', ) +ltree_dir = meson.current_source_dir() +ltree_dir_up = join_paths(ltree_dir, '..') +bitcode_modules += { + 'target': ltree, + 'srcfiles': ltree_sources, + 'additional_flags': [ + '-I@0@'.format(ltree_dir) + ] +} + tests += { 'name': 'ltree', 'sd': meson.current_source_dir(), diff --git a/contrib/ltree_plpython/meson.build b/contrib/ltree_plpython/meson.build index a37732c486b..74b285da05c 100644 --- a/contrib/ltree_plpython/meson.build +++ b/contrib/ltree_plpython/meson.build @@ -30,6 +30,17 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': ltree_plpython, + 'srcfiles': ltree_plpython_sources, + 'additional_flags': [ + '-I@0@'.format(ltree_dir_up), + '-DPLPYTHON_LIBNAME="plpython3"', + '-I@0@'.format(python3_inc_dir), + '-I@0@'.format(plpython_dir), + ] +} + ltree_plpython_regress = [ 'ltree_plpython' ] diff --git a/contrib/pg_buffercache/meson.build b/contrib/pg_buffercache/meson.build index 12d1fe48717..5385318fdc9 100644 --- a/contrib/pg_buffercache/meson.build +++ b/contrib/pg_buffercache/meson.build @@ -27,6 +27,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pg_buffercache, + 'srcfiles': pg_buffercache_sources, +} + tests += { 'name': 'pg_buffercache', 'sd': meson.current_source_dir(), diff --git a/contrib/pg_freespacemap/meson.build b/contrib/pg_freespacemap/meson.build index ff8eda3580e..120def1ad2e 100644 --- a/contrib/pg_freespacemap/meson.build +++ b/contrib/pg_freespacemap/meson.build @@ -25,6 +25,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pg_freespacemap, + 'srcfiles': pg_freespacemap_sources, +} + tests += { 'name': 'pg_freespacemap', 'sd': meson.current_source_dir(), diff --git a/contrib/pg_logicalinspect/meson.build b/contrib/pg_logicalinspect/meson.build index 5c0528a8c63..1b4f46277ed 100644 --- a/contrib/pg_logicalinspect/meson.build +++ b/contrib/pg_logicalinspect/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pg_logicalinspect, + 'srcfiles': pg_logicalinspect_sources, +} + tests += { 'name': 'pg_logicalinspect', 'sd': meson.current_source_dir(), diff --git a/contrib/pg_surgery/meson.build b/contrib/pg_surgery/meson.build index c6cfa9c4694..9d7199392c7 100644 --- a/contrib/pg_surgery/meson.build +++ b/contrib/pg_surgery/meson.build @@ -22,6 +22,10 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pg_surgery, + 'srcfiles': pg_surgery_sources, +} tests += { 'name': 'pg_surgery', diff --git a/contrib/pg_trgm/meson.build b/contrib/pg_trgm/meson.build index a31aa5c574d..408e287172e 100644 --- a/contrib/pg_trgm/meson.build +++ b/contrib/pg_trgm/meson.build @@ -32,6 +32,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pg_trgm, + 'srcfiles': pg_trgm_sources, +} + tests += { 'name': 'pg_trgm', 'sd': meson.current_source_dir(), diff --git a/contrib/pgcrypto/meson.build b/contrib/pgcrypto/meson.build index 7a4e8e76d64..d6b0559f232 100644 --- a/contrib/pgcrypto/meson.build +++ b/contrib/pgcrypto/meson.build @@ -98,6 +98,10 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pgcrypto, + 'srcfiles': pgcrypto_sources, +} tests += { 'name': 'pgcrypto', diff --git a/contrib/seg/meson.build b/contrib/seg/meson.build index e331e097230..371cd272a06 100644 --- a/contrib/seg/meson.build +++ b/contrib/seg/meson.build @@ -3,6 +3,7 @@ seg_sources = files( 'seg.c', ) +bc_seg_sources = seg_sources seg_scan = custom_target('segscan', input: 'segscan.l', @@ -11,6 +12,7 @@ seg_scan = custom_target('segscan', ) generated_sources += seg_scan seg_sources += seg_scan +bc_seg_gen_sources = [{'srcfiles': [seg_scan]}] seg_parse = custom_target('segparse', input: 'segparse.y', @@ -18,6 +20,7 @@ seg_parse = custom_target('segparse', ) generated_sources += seg_parse.to_list() seg_sources += seg_parse +bc_seg_gen_sources += {'srcfiles': [seg_parse[0]]} if host_system == 'windows' seg_sources += rc_lib_gen.process(win32ver_rc, extra_args: [ @@ -47,6 +50,15 @@ install_headers( install_dir: dir_include_extension / 'seg', ) +bitcode_modules += { + 'target': seg, + 'srcfiles': bc_seg_sources, + 'gen_sources': bc_seg_gen_sources, + 'additional_flags': [ + '-I@0@'.format(meson.current_source_dir()), + ] +} + tests += { 'name': 'seg', 'sd': meson.current_source_dir(), diff --git a/contrib/spi/meson.build b/contrib/spi/meson.build index eeab1ab210b..b16c4384bb0 100644 --- a/contrib/spi/meson.build +++ b/contrib/spi/meson.build @@ -24,6 +24,11 @@ install_data('autoinc.example', kwargs: contrib_doc_args, ) +bitcode_modules += { + 'target': autoinc, + 'srcfiles': autoinc_sources, +} + insert_username_sources = files( 'insert_username.c', @@ -51,6 +56,11 @@ install_data('insert_username.example', kwargs: contrib_doc_args, ) +bitcode_modules += { + 'target': insert_username, + 'srcfiles': insert_username_sources, +} + moddatetime_sources = files( 'moddatetime.c', @@ -78,6 +88,11 @@ install_data('moddatetime.example', kwargs: contrib_doc_args, ) +bitcode_modules += { + 'target': moddatetime, + 'srcfiles': moddatetime_sources, +} + # this is needed for the regression tests; # comment out if you want a quieter refint package for other uses @@ -107,3 +122,8 @@ install_data('refint.control', 'refint--1.0.sql', install_data('refint.example', kwargs: contrib_doc_args, ) + +bitcode_modules += { + 'target': refint, + 'srcfiles': refint_sources, +} diff --git a/contrib/sslinfo/meson.build b/contrib/sslinfo/meson.build index 4c513759200..5750d783b92 100644 --- a/contrib/sslinfo/meson.build +++ b/contrib/sslinfo/meson.build @@ -29,3 +29,8 @@ install_data( 'sslinfo.control', kwargs: contrib_data_args, ) + +bitcode_modules += { + 'target': sslinfo, + 'srcfiles': sslinfo_sources, +} diff --git a/contrib/tablefunc/meson.build b/contrib/tablefunc/meson.build index ee67272ec0a..6c26ffafddf 100644 --- a/contrib/tablefunc/meson.build +++ b/contrib/tablefunc/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': tablefunc, + 'srcfiles': tablefunc_sources, +} + tests += { 'name': 'tablefunc', 'sd': meson.current_source_dir(), diff --git a/contrib/tcn/meson.build b/contrib/tcn/meson.build index 6ffb136af90..df71e8a91f9 100644 --- a/contrib/tcn/meson.build +++ b/contrib/tcn/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': tcn, + 'srcfiles': tcn_sources, +} + tests += { 'name': 'tcn', 'sd': meson.current_source_dir(), diff --git a/contrib/tsm_system_rows/meson.build b/contrib/tsm_system_rows/meson.build index b8cece3d80f..1ea74cce949 100644 --- a/contrib/tsm_system_rows/meson.build +++ b/contrib/tsm_system_rows/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': tsm_system_rows, + 'srcfiles': tsm_system_rows_sources, +} + tests += { 'name': 'tsm_system_rows', 'sd': meson.current_source_dir(), diff --git a/contrib/tsm_system_time/meson.build b/contrib/tsm_system_time/meson.build index 8a143a8f8e6..1f6b98d3ec4 100644 --- a/contrib/tsm_system_time/meson.build +++ b/contrib/tsm_system_time/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': tsm_system_time, + 'srcfiles': tsm_system_time_sources, +} + tests += { 'name': 'tsm_system_time', 'sd': meson.current_source_dir(), diff --git a/contrib/unaccent/meson.build b/contrib/unaccent/meson.build index 33d4649bae1..1f5d120e3d8 100644 --- a/contrib/unaccent/meson.build +++ b/contrib/unaccent/meson.build @@ -28,6 +28,11 @@ install_data( install_dir: dir_data / 'tsearch_data' ) +bitcode_modules += { + 'target': unaccent, + 'srcfiles': unaccent_sources, +} + # XXX: Implement downlo tests += { 'name': 'unaccent', diff --git a/contrib/uuid-ossp/meson.build b/contrib/uuid-ossp/meson.build index 982f27c085f..8d21bb0a996 100644 --- a/contrib/uuid-ossp/meson.build +++ b/contrib/uuid-ossp/meson.build @@ -29,6 +29,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': uuid_ossp, + 'srcfiles': uuid_ossp_sources, +} + tests += { 'name': 'uuid-ossp', 'sd': meson.current_source_dir(), diff --git a/contrib/xml2/meson.build b/contrib/xml2/meson.build index 08d3c3b8e30..b23a8fe0cd7 100644 --- a/contrib/xml2/meson.build +++ b/contrib/xml2/meson.build @@ -32,6 +32,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': xml2, + 'srcfiles': xml2_sources, +} + tests += { 'name': 'xml2', 'sd': meson.current_source_dir(), diff --git a/src/pl/plperl/meson.build b/src/pl/plperl/meson.build index b463d4d56c5..279277dee73 100644 --- a/src/pl/plperl/meson.build +++ b/src/pl/plperl/meson.build @@ -37,6 +37,7 @@ foreach n : ['SPI', 'Util'] plperl_sources += xs_c endforeach +plperl_dir = meson.current_source_dir() plperl_inc = include_directories('.') if host_system == 'windows' diff --git a/src/pl/plpython/meson.build b/src/pl/plpython/meson.build index 709e5932a93..e2c19771d85 100644 --- a/src/pl/plpython/meson.build +++ b/src/pl/plpython/meson.build @@ -29,6 +29,7 @@ plpython_sources += custom_target('spiexceptions.h', # FIXME: need to duplicate import library ugliness? plpython_inc = include_directories('.') +plpython_dir = meson.current_source_dir() if host_system == 'windows' plpython_sources += rc_lib_gen.process(win32ver_rc, extra_args: [ diff --git a/meson.build b/meson.build index a01dbca66d1..9e20be4590d 100644 --- a/meson.build +++ b/meson.build @@ -1244,6 +1244,7 @@ if not pyopt.disabled() python3_inst = pm.find_installation(python.path(), required: pyopt) if python3_inst.found() python3_dep = python3_inst.dependency(embed: true, required: pyopt) + python3_inc_dir = python3_inst.get_variable('INCLUDEPY') # Remove this check after we depend on Meson >= 1.1.0 if not cc.check_header('Python.h', dependencies: python3_dep, required: pyopt, include_directories: postgres_inc) python3_dep = not_found_dep -- 2.47.2 [text/x-patch] v4-0007-meson-Add-LLVM-bitcode-emission-for-backend-sourc.patch (5.2K, 8-v4-0007-meson-Add-LLVM-bitcode-emission-for-backend-sourc.patch) download | inline diff: From b64d31ff236c6419f5450698490387f30ebdce19 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Wed, 12 Mar 2025 10:44:46 +0300 Subject: [PATCH v4 7/7] meson: Add LLVM bitcode emission for backend sources Since generated backend sources may have their own compilation flags and must also be included in the postgres.index.bc, the way to make it work with current code was to create a new variable, called `bc_generated_backend_sources`, which is a list of dictionaries, each one having an optional 'additional_flags' and a `srclist` pointing to the list of custom_target generated sources. An example of a possible structure of bitcode_modules which is processed by the main meson llvm bitcode emission file src/backend/jit/llvm/bitcode/meson.build: ``` bitcode_modules = [ { 'name': 'postgres', 'target': postgres_lib, 'src_file': backend_sources, 'gen_srcfiles': [ { 'additional_flags': [ '-I/path/postgresl/src/backend/parser', '-I/path/postgresl/build/src/backend/parser', ], 'srcfiles': [ <custom_target for scan.c>, <custom_target for gram.c> ] } ] } ] ``` Author: Diego Fronza <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/backend/bootstrap/meson.build | 2 ++ src/backend/meson.build | 2 ++ src/backend/parser/meson.build | 8 ++++++++ src/backend/replication/meson.build | 4 ++++ src/backend/utils/fmgr/meson.build | 1 + 5 files changed, 17 insertions(+) diff --git a/src/backend/bootstrap/meson.build b/src/backend/bootstrap/meson.build index 29726c1ab4f..389c75f8081 100644 --- a/src/backend/bootstrap/meson.build +++ b/src/backend/bootstrap/meson.build @@ -13,6 +13,7 @@ bootscanner = custom_target('bootscanner', ) generated_sources += bootscanner boot_parser_sources += bootscanner +bc_generated_backend_sources += {'srcfiles': [bootscanner]} bootparse = custom_target('bootparse', input: 'bootparse.y', @@ -20,6 +21,7 @@ bootparse = custom_target('bootparse', ) generated_sources += bootparse.to_list() boot_parser_sources += bootparse +bc_generated_backend_sources += {'srcfiles': [bootparse[0]]} boot_parser = static_library('boot_parser', boot_parser_sources, diff --git a/src/backend/meson.build b/src/backend/meson.build index 5fb33660d3d..85453de1928 100644 --- a/src/backend/meson.build +++ b/src/backend/meson.build @@ -5,6 +5,7 @@ backend_sources = [] backend_link_with = [pgport_srv, common_srv] generated_backend_sources = [] +bc_generated_backend_sources = [] post_export_backend_sources = [] subdir('access') @@ -144,6 +145,7 @@ bitcode_modules += { 'name': 'postgres', 'target': postgres_lib, 'srcfiles': backend_sources, + 'gen_sources': bc_generated_backend_sources, } pg_mod_c_args = cflags_mod diff --git a/src/backend/parser/meson.build b/src/backend/parser/meson.build index 874aa749aa6..add472a0cd8 100644 --- a/src/backend/parser/meson.build +++ b/src/backend/parser/meson.build @@ -42,6 +42,14 @@ backend_parser = custom_target('gram', generated_sources += backend_parser.to_list() parser_sources += backend_parser +bc_generated_backend_sources += { + 'additional_flags': [ + '-I@0@'.format(meson.current_build_dir()), + '-I@0@'.format(meson.current_source_dir()), + ], + 'srcfiles': [backend_scanner, backend_parser[0]], +} + parser = static_library('parser', parser_sources, dependencies: [backend_code], diff --git a/src/backend/replication/meson.build b/src/backend/replication/meson.build index b0601498865..71ab1164960 100644 --- a/src/backend/replication/meson.build +++ b/src/backend/replication/meson.build @@ -19,6 +19,7 @@ repl_scanner = custom_target('repl_scanner', ) generated_sources += repl_scanner repl_parser_sources += repl_scanner +bc_generated_backend_sources += {'srcfiles': [repl_scanner]} repl_gram = custom_target('repl_gram', input: 'repl_gram.y', @@ -26,6 +27,7 @@ repl_gram = custom_target('repl_gram', ) generated_sources += repl_gram.to_list() repl_parser_sources += repl_gram +bc_generated_backend_sources += {'srcfiles': [repl_gram[0]]} syncrep_scanner = custom_target('syncrep_scanner', input: 'syncrep_scanner.l', @@ -34,6 +36,7 @@ syncrep_scanner = custom_target('syncrep_scanner', ) generated_sources += syncrep_scanner repl_parser_sources += syncrep_scanner +bc_generated_backend_sources += {'srcfiles': [syncrep_scanner]} syncrep_gram = custom_target('syncrep_gram', input: 'syncrep_gram.y', @@ -41,6 +44,7 @@ syncrep_gram = custom_target('syncrep_gram', ) generated_sources += syncrep_gram.to_list() repl_parser_sources += syncrep_gram +bc_generated_backend_sources += {'srcfiles': [syncrep_gram[0]]} repl_parser = static_library('repl_parser', repl_parser_sources, diff --git a/src/backend/utils/fmgr/meson.build b/src/backend/utils/fmgr/meson.build index b1dcab93e70..080f59988e9 100644 --- a/src/backend/utils/fmgr/meson.build +++ b/src/backend/utils/fmgr/meson.build @@ -8,3 +8,4 @@ backend_sources += files( # fmgrtab.c generated_backend_sources += fmgrtab_target[2] +bc_generated_backend_sources += {'srcfiles': [fmgrtab_target[2]]} -- 2.47.2 ^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: meson vs. llvm bitcode files @ 2025-04-29 08:23 Nazir Bilal Yavuz <[email protected]> parent: Nazir Bilal Yavuz <[email protected]> 0 siblings, 1 reply; 10+ messages in thread From: Nazir Bilal Yavuz @ 2025-04-29 08:23 UTC (permalink / raw) To: Diego Fronza <[email protected]>; +Cc: Peter Eisentraut <[email protected]>; pgsql-hackers; Andres Freund <[email protected]> Hi, On Thu, 13 Mar 2025 at 13:11, Nazir Bilal Yavuz <[email protected]> wrote: > One thing I noticed is that gen_srcfiles['srcfiles'] seems wrong. > gen_sources is a better name compared to gen_srcfiles. So, I changed > it to gen_sources in v4. Rebase is needed due to b1720fe63f, v5 is attached. -- Regards, Nazir Bilal Yavuz Microsoft Attachments: [text/x-patch] v5-0001-meson-Add-generated-header-stamps.patch (5.0K, 2-v5-0001-meson-Add-generated-header-stamps.patch) download | inline diff: From 8965a194e7449c4731c896941994500565fb6f26 Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Thu, 24 Oct 2024 07:23:05 -0400 Subject: [PATCH v5 1/7] meson: Add generated header stamps Otherwise build commands become too long and this has visible effect on creation time of meson build files. Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/include/meson.build | 18 ++++++++++++++++++ src/backend/meson.build | 2 +- src/fe_utils/meson.build | 2 +- meson.build | 16 +++++++++------- 4 files changed, 29 insertions(+), 9 deletions(-) diff --git a/src/include/meson.build b/src/include/meson.build index 2e4b7aa529e..c488a5dc4c9 100644 --- a/src/include/meson.build +++ b/src/include/meson.build @@ -177,3 +177,21 @@ install_subdir('catalog', # autoconf generates the file there, ensure we get a conflict generated_sources_ac += {'src/include': ['stamp-h']} + +# Instead of having targets depending directly on the generated headers, have +# them depend on a stamp files for all of them. Dependencies on headers are +# implemented as order-only dependencies in meson (later using compiler +# generated dependencies). The benefit of using a stamp file is that it makes +# ninja.build smaller and meson setup faster. +generated_headers_stamp = custom_target('generated-headers-stamp.h', + output: 'generated-headers-stamp.h', + input: generated_headers, + command: stamp_cmd, +) + +generated_backend_headers_stamp = custom_target('generated-backend-headers-stamp.h', + output: 'generated-backend-headers-stamp.h', + input: generated_backend_headers, + depends: generated_headers_stamp, + command: stamp_cmd, +) diff --git a/src/backend/meson.build b/src/backend/meson.build index 2b0db214804..7fc649c3ebd 100644 --- a/src/backend/meson.build +++ b/src/backend/meson.build @@ -169,7 +169,7 @@ backend_mod_code = declare_dependency( compile_args: pg_mod_c_args, include_directories: postgres_inc, link_args: pg_mod_link_args, - sources: generated_headers + generated_backend_headers, + sources: [generated_backend_headers_stamp], dependencies: backend_mod_deps, ) diff --git a/src/fe_utils/meson.build b/src/fe_utils/meson.build index a18cbc939e4..5a9ddb73463 100644 --- a/src/fe_utils/meson.build +++ b/src/fe_utils/meson.build @@ -29,7 +29,7 @@ generated_sources += psqlscan fe_utils_sources += psqlscan fe_utils = static_library('libpgfeutils', - fe_utils_sources + generated_headers, + fe_utils_sources, c_pch: pch_postgres_fe_h, include_directories: [postgres_inc, libpq_inc], c_args: host_system == 'windows' ? ['-DFD_SETSIZE=1024'] : [], diff --git a/meson.build b/meson.build index a1516e54529..35cf7d8b948 100644 --- a/meson.build +++ b/meson.build @@ -3105,6 +3105,8 @@ gen_export_kwargs = { 'install': false, } +# command to create stamp files on all OSs +stamp_cmd = [python, '-c', 'import sys; open(sys.argv[1], "w")', '@OUTPUT0@'] ### @@ -3222,14 +3224,14 @@ subdir('src/port') frontend_common_code = declare_dependency( compile_args: ['-DFRONTEND'], include_directories: [postgres_inc], - sources: generated_headers, + sources: generated_headers_stamp, dependencies: [os_deps, zlib, zstd, lz4], ) backend_common_code = declare_dependency( compile_args: ['-DBUILDING_DLL'], include_directories: [postgres_inc], - sources: generated_headers, + sources: generated_headers_stamp, dependencies: [os_deps, zlib, zstd], ) @@ -3244,7 +3246,7 @@ shlib_code = declare_dependency( frontend_stlib_code = declare_dependency( include_directories: [postgres_inc], link_with: [common_static, pgport_static], - sources: generated_headers, + sources: generated_headers_stamp, dependencies: [os_deps, libintl], ) @@ -3252,7 +3254,7 @@ frontend_stlib_code = declare_dependency( frontend_shlib_code = declare_dependency( include_directories: [postgres_inc], link_with: [common_shlib, pgport_shlib], - sources: generated_headers, + sources: generated_headers_stamp, dependencies: [shlib_code, os_deps, libintl], ) @@ -3262,7 +3264,7 @@ frontend_shlib_code = declare_dependency( frontend_no_fe_utils_code = declare_dependency( include_directories: [postgres_inc], link_with: [common_static, pgport_static], - sources: generated_headers, + sources: generated_headers_stamp, dependencies: [os_deps, libintl], ) @@ -3288,7 +3290,7 @@ subdir('src/fe_utils') frontend_code = declare_dependency( include_directories: [postgres_inc], link_with: [fe_utils, common_static, pgport_static], - sources: generated_headers, + sources: generated_headers_stamp, dependencies: [os_deps, libintl], ) @@ -3318,7 +3320,7 @@ backend_code = declare_dependency( include_directories: [postgres_inc], link_args: ldflags_be, link_with: [], - sources: generated_headers + generated_backend_headers, + sources: [generated_backend_headers_stamp], dependencies: os_deps + backend_both_deps + backend_deps, ) -- 2.49.0 [text/x-patch] v5-0002-meson-Add-postgresql-extension.pc-for-building-ex.patch (4.5K, 3-v5-0002-meson-Add-postgresql-extension.pc-for-building-ex.patch) download | inline diff: From de2990f424c394b93bdf41f8ab06f60305898313 Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Sat, 27 Aug 2022 09:52:03 -0700 Subject: [PATCH v5 2/7] meson: Add postgresql-extension.pc for building extension libraries This should work with several other buildsystems. TODO: Docs Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/backend/meson.build | 110 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 110 insertions(+) diff --git a/src/backend/meson.build b/src/backend/meson.build index 7fc649c3ebd..9d79d4d058c 100644 --- a/src/backend/meson.build +++ b/src/backend/meson.build @@ -188,6 +188,116 @@ pg_test_mod_args = pg_mod_args + { +############################################################### +# Define a .pc file that can be used to build server extensions +############################################################### + +pg_ext_vars = [] +pg_ext_vars_inst = [] +pg_ext_vars_uninst = [] + +pg_ext_cflags = pg_mod_c_args + cppflags +pg_ext_libs = [backend_mod_deps, thread_dep, ldflags, ldflags_mod] +pg_ext_subdirs = [''] + +# Compute directories to add include directories to the .pc files for. +# This is a bit more complicated due to port/win32 etc. +i = 0 +foreach incdir : postgres_inc_d + if fs.is_absolute(incdir) + # an absolute path from -Dextra_include_dirs + pg_ext_cflags += '-I@0@'.format(incdir) + continue + elif incdir.startswith('src/include') + subincdir = dir_include_pkg_rel / 'server' / incdir.split('src/include/').get(1, '') + else + subincdir = '' + endif + pg_ext_subdirs += subincdir + + # Add directories in source / build dir containing headers to cflags for the + # -uninstalled.pc. Older versions of pkg-config complain if a referenced + # variable is not defined, so we emit an empty one for the installed .pc + # file. + pg_ext_vars += [ + 'build_inc@0@=""'.format(i), + 'src_inc@0@=""'.format(i), + ] + pg_ext_vars_uninst += [ + 'build_inc@0@=-I${prefix}/@1@'.format(i, incdir), + 'src_inc@0@=-I${srcdir}/@1@'.format(i, incdir), + ] + pg_ext_cflags += [ + '${build_inc@0@}'.format(i), + '${src_inc@0@}'.format(i) + ] + + i += 1 +endforeach + + +# Extension modules should likely also use -fwrapv etc. But it it's a bit odd +# to expose it to a .pc file? +pg_ext_cflags_warn = pg_ext_cflags + cflags_warn +pg_ext_cflags += cflags + +# Directories for extensions to install into +# XXX: more needed +pg_ext_vars += 'pkglibdir=${prefix}/@0@'.format(dir_lib_pkg) +pg_ext_vars += 'dir_mod=${pkglibdir}' +pg_ext_vars += 'dir_data=${prefix}/@0@'.format(dir_data_extension) +pg_ext_vars += 'dir_include=${prefix}/@0@'.format(dir_include_extension) +pg_ext_vars += 'dir_doc=${prefix}/@0@'.format(dir_doc_extension) +pg_ext_vars += 'dir_bitcode=${prefix}/@0@'.format(dir_bitcode) +# referenced on some platforms, via mod_link_with_dir +pg_ext_vars += 'bindir=${prefix}/@0@'.format(dir_bin) + +# XXX: Define variables making it easy to define tests, too + +# Some platforms need linker flags to link with binary, they are the same +# between building with meson and .pc file, except that we have have to +# reference a variable to make it work for both normal and -uninstalled .pc +# files. +if mod_link_args_fmt.length() != 0 + assert(link_with_inst != '') + assert(link_with_uninst != '') + + pg_ext_vars_inst += 'mod_link_with=@0@'.format(link_with_inst) + pg_ext_vars_uninst += 'mod_link_with=@0@'.format(link_with_uninst) + + foreach el : mod_link_args_fmt + pg_ext_libs += el.format('${mod_link_with}') + endforeach +endif + +# main .pc to build extensions +pkgconfig.generate( + name: 'postgresql-extension', + description: 'PostgreSQL Extension Support', + url: pg_url, + + subdirs: pg_ext_subdirs, + libraries: pg_ext_libs, + extra_cflags: pg_ext_cflags, + + variables: pg_ext_vars + pg_ext_vars_inst, + uninstalled_variables: pg_ext_vars + pg_ext_vars_uninst, +) + +# a .pc depending on the above, but with all our warnings enabled +pkgconfig.generate( + name: 'postgresql-extension-warnings', + description: 'PostgreSQL Extension Support - Compiler Warnings', + requires: 'postgresql-extension', + url: pg_url, + extra_cflags: pg_ext_cflags_warn, + + variables: pg_ext_vars + pg_ext_vars_inst, + uninstalled_variables: pg_ext_vars + pg_ext_vars_uninst, +) + + + # Shared modules that, on some system, link against the server binary. Only # enter these after we defined the server build. -- 2.49.0 [text/x-patch] v5-0003-meson-Test-building-extensions-by-using-postgresq.patch (10.7K, 4-v5-0003-meson-Test-building-extensions-by-using-postgresq.patch) download | inline diff: From 171d16416031a8649079a727c3aad5fb4b517c46 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Thu, 27 Feb 2025 17:45:31 +0300 Subject: [PATCH v5 3/7] meson: Test building extensions by using postgresql-extension.pc The 'test_meson_extensions' pyton wrapper is added to run these tests. It compiles and builds extensions at ${build}/testrun/meson_extensions/${extension_name} path. The tests for building amcheck, auth_delay and postgres_fdw extensions are added. These are also examples of how to build extensions by using postgresql-extension.pc. Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/test/modules/meson.build | 1 + .../modules/test_meson_extensions/meson.build | 3 + .../amcheck/meson.build | 28 ++++++++ .../auth_delay/meson.build | 17 +++++ .../test_pkg_config_extensions/meson.build | 24 +++++++ .../postgres_fdw/meson.build | 30 ++++++++ meson.build | 32 +++++++++ src/tools/ci/test_meson_extensions | 70 +++++++++++++++++++ 8 files changed, 205 insertions(+) create mode 100644 src/test/modules/test_meson_extensions/meson.build create mode 100644 src/test/modules/test_meson_extensions/test_pkg_config_extensions/amcheck/meson.build create mode 100644 src/test/modules/test_meson_extensions/test_pkg_config_extensions/auth_delay/meson.build create mode 100644 src/test/modules/test_meson_extensions/test_pkg_config_extensions/meson.build create mode 100644 src/test/modules/test_meson_extensions/test_pkg_config_extensions/postgres_fdw/meson.build create mode 100644 src/tools/ci/test_meson_extensions diff --git a/src/test/modules/meson.build b/src/test/modules/meson.build index 9de0057bd1d..94db733c4ad 100644 --- a/src/test/modules/meson.build +++ b/src/test/modules/meson.build @@ -26,6 +26,7 @@ subdir('test_ginpostinglist') subdir('test_integerset') subdir('test_json_parser') subdir('test_lfind') +subdir('test_meson_extensions') subdir('test_misc') subdir('test_oat_hooks') subdir('test_parser') diff --git a/src/test/modules/test_meson_extensions/meson.build b/src/test/modules/test_meson_extensions/meson.build new file mode 100644 index 00000000000..06cf5d555df --- /dev/null +++ b/src/test/modules/test_meson_extensions/meson.build @@ -0,0 +1,3 @@ +# Copyright (c) 2022-2025, PostgreSQL Global Development Group + +subdir('test_pkg_config_extensions') diff --git a/src/test/modules/test_meson_extensions/test_pkg_config_extensions/amcheck/meson.build b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/amcheck/meson.build new file mode 100644 index 00000000000..482a543eb86 --- /dev/null +++ b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/amcheck/meson.build @@ -0,0 +1,28 @@ +# Copyright (c) 2022-2025, PostgreSQL Global Development Group + +project('amcheck', 'c') + +amcheck_path = '../../../../../../contrib/amcheck/' + +amcheck_sources = files( + amcheck_path / 'verify_heapam.c', + amcheck_path / 'verify_nbtree.c', +) + +pg_ext = dependency('postgresql-extension-warnings') + +amcheck = shared_module('amcheck', + amcheck_sources, + dependencies: pg_ext, + install_dir: pg_ext.get_variable(pkgconfig: 'dir_mod'), +) + +install_data( + amcheck_path / 'amcheck.control', + amcheck_path / 'amcheck--1.0.sql', + amcheck_path / 'amcheck--1.0--1.1.sql', + amcheck_path / 'amcheck--1.1--1.2.sql', + amcheck_path / 'amcheck--1.2--1.3.sql', + amcheck_path / 'amcheck--1.3--1.4.sql', + install_dir: pg_ext.get_variable(pkgconfig: 'dir_data'), +) diff --git a/src/test/modules/test_meson_extensions/test_pkg_config_extensions/auth_delay/meson.build b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/auth_delay/meson.build new file mode 100644 index 00000000000..98ad24cc183 --- /dev/null +++ b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/auth_delay/meson.build @@ -0,0 +1,17 @@ +# Copyright (c) 2022-2025, PostgreSQL Global Development Group + +project('auth_delay', 'c') + +auth_delay_path = '../../../../../../contrib/auth_delay/' + +auth_delay_sources = files( + auth_delay_path / 'auth_delay.c', +) + +pg_ext = dependency('postgresql-extension-warnings') + +auth_delay = shared_module('auth_delay', + auth_delay_sources, + dependencies: pg_ext, + install_dir: pg_ext.get_variable(pkgconfig: 'dir_mod'), +) diff --git a/src/test/modules/test_meson_extensions/test_pkg_config_extensions/meson.build b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/meson.build new file mode 100644 index 00000000000..dae94a384a1 --- /dev/null +++ b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/meson.build @@ -0,0 +1,24 @@ +# Copyright (c) 2022-2025, PostgreSQL Global Development Group + +# pkgconfig is not available on Windows, so skip it. +if host_machine.system() == 'windows' + subdir_done() +endif + +meson_extension_tests += { + 'name': 'amcheck', + 'kind': 'pkg_config', + 'sd': meson.current_source_dir() / 'amcheck', +} + +meson_extension_tests += { + 'name': 'auth_delay', + 'kind': 'pkg_config', + 'sd': meson.current_source_dir() / 'auth_delay', +} + +meson_extension_tests += { + 'name': 'postgres_fdw', + 'kind': 'pkg_config', + 'sd': meson.current_source_dir() / 'postgres_fdw', +} diff --git a/src/test/modules/test_meson_extensions/test_pkg_config_extensions/postgres_fdw/meson.build b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/postgres_fdw/meson.build new file mode 100644 index 00000000000..6d561e3789b --- /dev/null +++ b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/postgres_fdw/meson.build @@ -0,0 +1,30 @@ +# Copyright (c) 2022-2025, PostgreSQL Global Development Group + +project('auth_delay', 'c') + +postgres_fdw_path = '../../../../../../contrib/postgres_fdw/' + +postgres_fdw_sources = files( + postgres_fdw_path / 'connection.c', + postgres_fdw_path / 'deparse.c', + postgres_fdw_path / 'option.c', + postgres_fdw_path / 'postgres_fdw.c', + postgres_fdw_path / 'shippable.c', +) + +pg_ext = dependency('postgresql-extension-warnings') +libpq = dependency('libpq') + +postgres_fdw = shared_module('postgres_fdw', + postgres_fdw_sources, + dependencies: [pg_ext, libpq], + install_dir: pg_ext.get_variable(pkgconfig: 'dir_mod'), +) + +install_data( + postgres_fdw_path / 'postgres_fdw.control', + postgres_fdw_path / 'postgres_fdw--1.0.sql', + postgres_fdw_path / 'postgres_fdw--1.0--1.1.sql', + postgres_fdw_path / 'postgres_fdw--1.1--1.2.sql', + install_dir: pg_ext.get_variable(pkgconfig: 'dir_data'), +) diff --git a/meson.build b/meson.build index 35cf7d8b948..1b7388096a6 100644 --- a/meson.build +++ b/meson.build @@ -3023,6 +3023,7 @@ nls_targets = [] # Define the tests to distribute them to the correct test styles later test_deps = [] tests = [] +meson_extension_tests = [] # Default options for targets @@ -3577,6 +3578,37 @@ sys.exit(sp.returncode) suite: ['setup']) +# it seems freebsd doesn't use libdir for pkgconfig path +if host_system == 'freebsd' + pkgconf_installdir = dir_prefix / 'libdata' / 'pkgconfig' +else + pkgconf_installdir = dir_prefix / dir_lib / 'pkgconfig' +endif +test_pkg_conf_file = files('src/tools/ci/test_meson_extensions') + +foreach test : meson_extension_tests + if test['kind'] not in ['pkg_config'] + error('unknown kind @0@ of test in @1@'.format(test['kind'], test['sd'])) + endif + + test_group = 'meson_@0@_extensions'.format(test['kind']) + + test(test_group / test['name'], + test_pkg_conf_file, + args: [ + '--meson', meson_bin.path(), + '--meson_args', meson_args, + '--test_dir', test['sd'], + '--test_out_dir', test_result_dir / 'meson_extensions' / test['name'], + '--builddir', meson.build_root(), + '--pkg_conf_path', get_option('pkg_config_path'), + '--', + cc.cmd_array(), + ], + suite: test_group, + ) + +endforeach ############################################################### # Test Generation diff --git a/src/tools/ci/test_meson_extensions b/src/tools/ci/test_meson_extensions new file mode 100644 index 00000000000..50358121f49 --- /dev/null +++ b/src/tools/ci/test_meson_extensions @@ -0,0 +1,70 @@ +#!/usr/bin/env python3 + +import argparse +import os +import shutil +import subprocess + +parser = argparse.ArgumentParser() + +parser.add_argument('--meson', help='path to meson binary', + type=str, required=True) +parser.add_argument('--meson_args', help='args of meson binary', + type=str, nargs='*', required=False) +parser.add_argument('--test_dir', help='test source directory', + type=str, required=True) +parser.add_argument('--test_out_dir', help='test output directory', + type=str, required=True) +parser.add_argument('--builddir', help='meson build directory', + type=str, required=True) +parser.add_argument('--pkg_conf_path', + help='PKG_CONF_PATH from surrounding meson build', + type=str, nargs='?', const='', required=False) +parser.add_argument('c_args', help='c_args from surrounding meson build', + nargs='*') + +args = parser.parse_args() + +meson_bin = args.meson +meson_args = ' '.join(args.meson_args) +test_source_dir = args.test_dir +test_out_dir = args.test_out_dir +build_dir = args.builddir +pkg_conf_path = args.pkg_conf_path +c_args = ' '.join(args.c_args) + +exit_code = 0 + +def remove_duplicates(duplicate_str): + words = duplicate_str.split() + return ' '.join(sorted(set(words), key=words.index)) + + +def run_tests(pkg_conf_path_local, message): + print('\n{}\n{}\n'.format('#' * 60, message), flush=True) + + env = {**os.environ, } + env['PKG_CONFIG_PATH'] = '{}:{}:{}'.format( + pkg_conf_path_local, pkg_conf_path, env.get('PKG_CONFIG_PATH', ''), + ).strip(': ') + env['CC'] = '{} {}'.format( + c_args, env.get('CC', ''), + ) + env['CC'] = remove_duplicates(env['CC']) + + # Clear the build directory beforehand. + if os.path.exists(test_out_dir): + shutil.rmtree(test_out_dir) + + if meson_args: + meson_setup_command = [meson_bin, meson_args, 'setup', test_out_dir] + else: + meson_setup_command = [meson_bin, 'setup', test_out_dir] + + meson_compile_command = ['meson', 'compile', '-C', test_out_dir, '-v'] + + subprocess.run(meson_setup_command, env=env, cwd=test_source_dir, check=True) + subprocess.run(meson_compile_command, cwd=test_source_dir, check=True) + +run_tests(os.path.join(build_dir, 'meson-uninstalled'), + message='Testing postgresql-extension-warnings-uninstalled') -- 2.49.0 [text/x-patch] v5-0004-meson-WIP-Add-docs-for-postgresql-extension.pc.patch (12.3K, 5-v5-0004-meson-WIP-Add-docs-for-postgresql-extension.pc.patch) download | inline diff: From 05f7e8733b9b2f0aa5404636c2241a7e56800692 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Thu, 6 Mar 2025 17:46:57 +0300 Subject: [PATCH v5 4/7] meson: [WIP] Add docs for postgresql-extension.pc Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- doc/src/sgml/acronyms.sgml | 2 +- doc/src/sgml/extend.sgml | 101 +++++++++++++++++++++++-------------- doc/src/sgml/jit.sgml | 2 +- 3 files changed, 66 insertions(+), 39 deletions(-) diff --git a/doc/src/sgml/acronyms.sgml b/doc/src/sgml/acronyms.sgml index 2f906e9f018..57f49c06a19 100644 --- a/doc/src/sgml/acronyms.sgml +++ b/doc/src/sgml/acronyms.sgml @@ -579,7 +579,7 @@ <term><acronym>PGXS</acronym></term> <listitem> <para> - <link linkend="extend-pgxs"><productname>PostgreSQL</productname> Extension System</link> + <link linkend="extend-postgres-pgxs"><productname>PostgreSQL</productname> Extension System</link> </para> </listitem> </varlistentry> diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml index 64f8e133cae..8be5313d9fd 100644 --- a/doc/src/sgml/extend.sgml +++ b/doc/src/sgml/extend.sgml @@ -1426,7 +1426,7 @@ include $(PGXS) </programlisting> This makefile relies on <acronym>PGXS</acronym>, which is described - in <xref linkend="extend-pgxs"/>. The command <literal>make install</literal> + in <xref linkend="extend-postgres-pgxs"/>. The command <literal>make install</literal> will install the control and script files into the correct directory as reported by <application>pg_config</application>. </para> @@ -1439,21 +1439,26 @@ include $(PGXS) </sect2> </sect1> - <sect1 id="extend-pgxs"> + <sect1 id="extend-postgres"> <title>Extension Building Infrastructure</title> - <indexterm zone="extend-pgxs"> - <primary>pgxs</primary> - </indexterm> - <para> If you are thinking about distributing your <productname>PostgreSQL</productname> extension modules, setting up a portable build system for them can be fairly difficult. Therefore the <productname>PostgreSQL</productname> installation provides a build - infrastructure for extensions, called <acronym>PGXS</acronym>, so - that simple extension modules can be built simply against an - already installed server. <acronym>PGXS</acronym> is mainly intended + infrastructure for extensions, called <literal>PGXS</literal> + (<xref linkend="extend-postgres-pgxs"/>) and + its meson counterpart <literal>postgresql-extension.pc</literal> + (<xref linkend="extend-postgres-meson"/>). + </para> + + </sect1> + + <sect1 id="extend-postgres-pgxs"> + <title>PGXS</title> + + <para> <acronym>PGXS</acronym> is mainly intended for extensions that include C code, although it can be used for pure-SQL extensions too. Note that <acronym>PGXS</acronym> is not intended to be a universal build system framework that can be used @@ -1493,7 +1498,7 @@ include $(PGXS) Set one of these three variables to specify what is built: <variablelist> - <varlistentry id="extend-pgxs-modules"> + <varlistentry id="extend-postgres-pgxs-modules"> <term><varname>MODULES</varname></term> <listitem> <para> @@ -1503,7 +1508,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-module-big"> + <varlistentry id="extend-postgres-pgxs-module-big"> <term><varname>MODULE_big</varname></term> <listitem> <para> @@ -1513,7 +1518,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-program"> + <varlistentry id="extend-postgres-pgxs-program"> <term><varname>PROGRAM</varname></term> <listitem> <para> @@ -1527,7 +1532,7 @@ include $(PGXS) The following variables can also be set: <variablelist> - <varlistentry id="extend-pgxs-extension"> + <varlistentry id="extend-postgres-pgxs-extension"> <term><varname>EXTENSION</varname></term> <listitem> <para> @@ -1539,7 +1544,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-moduledir"> + <varlistentry id="extend-postgres-pgxs-moduledir"> <term><varname>MODULEDIR</varname></term> <listitem> <para> @@ -1552,7 +1557,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-data"> + <varlistentry id="extend-postgres-pgxs-data"> <term><varname>DATA</varname></term> <listitem> <para> @@ -1561,7 +1566,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-data-built"> + <varlistentry id="extend-postgres-pgxs-data-built"> <term><varname>DATA_built</varname></term> <listitem> <para> @@ -1572,7 +1577,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-data-tsearch"> + <varlistentry id="extend-postgres-pgxs-data-tsearch"> <term><varname>DATA_TSEARCH</varname></term> <listitem> <para> @@ -1582,7 +1587,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-docs"> + <varlistentry id="extend-postgres-pgxs-docs"> <term><varname>DOCS</varname></term> <listitem> <para> @@ -1592,7 +1597,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-headers"> + <varlistentry id="extend-postgres-pgxs-headers"> <term><varname>HEADERS</varname></term> <term><varname>HEADERS_built</varname></term> <listitem> @@ -1608,7 +1613,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-headers-module"> + <varlistentry id="extend-postgres-pgxs-headers-module"> <term><varname>HEADERS_$MODULE</varname></term> <term><varname>HEADERS_built_$MODULE</varname></term> <listitem> @@ -1634,7 +1639,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-scripts"> + <varlistentry id="extend-postgres-pgxs-scripts"> <term><varname>SCRIPTS</varname></term> <listitem> <para> @@ -1644,7 +1649,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-scripts-built"> + <varlistentry id="extend-postgres-pgxs-scripts-built"> <term><varname>SCRIPTS_built</varname></term> <listitem> <para> @@ -1655,7 +1660,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-regress"> + <varlistentry id="extend-postgres-pgxs-regress"> <term><varname>REGRESS</varname></term> <listitem> <para> @@ -1664,7 +1669,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-regress-opts"> + <varlistentry id="extend-postgres-pgxs-regress-opts"> <term><varname>REGRESS_OPTS</varname></term> <listitem> <para> @@ -1673,7 +1678,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-isolation"> + <varlistentry id="extend-postgres-pgxs-isolation"> <term><varname>ISOLATION</varname></term> <listitem> <para> @@ -1682,7 +1687,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-isolation-opts"> + <varlistentry id="extend-postgres-pgxs-isolation-opts"> <term><varname>ISOLATION_OPTS</varname></term> <listitem> <para> @@ -1692,7 +1697,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-tap-tests"> + <varlistentry id="extend-postgres-pgxs-tap-tests"> <term><varname>TAP_TESTS</varname></term> <listitem> <para> @@ -1701,7 +1706,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-no-install"> + <varlistentry id="extend-postgres-pgxs-no-install"> <term><varname>NO_INSTALL</varname></term> <listitem> <para> @@ -1711,7 +1716,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-no-installcheck"> + <varlistentry id="extend-postgres-pgxs-no-installcheck"> <term><varname>NO_INSTALLCHECK</varname></term> <listitem> <para> @@ -1720,7 +1725,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-extra-clean"> + <varlistentry id="extend-postgres-pgxs-extra-clean"> <term><varname>EXTRA_CLEAN</varname></term> <listitem> <para> @@ -1729,7 +1734,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-cppflags"> + <varlistentry id="extend-postgres-pgxs-pg-cppflags"> <term><varname>PG_CPPFLAGS</varname></term> <listitem> <para> @@ -1738,7 +1743,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-cflags"> + <varlistentry id="extend-postgres-pgxs-pg-cflags"> <term><varname>PG_CFLAGS</varname></term> <listitem> <para> @@ -1747,7 +1752,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-cxxflags"> + <varlistentry id="extend-postgres-pgxs-pg-cxxflags"> <term><varname>PG_CXXFLAGS</varname></term> <listitem> <para> @@ -1756,7 +1761,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-ldflags"> + <varlistentry id="extend-postgres-pgxs-pg-ldflags"> <term><varname>PG_LDFLAGS</varname></term> <listitem> <para> @@ -1765,7 +1770,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-libs"> + <varlistentry id="extend-postgres-pgxs-pg-libs"> <term><varname>PG_LIBS</varname></term> <listitem> <para> @@ -1774,7 +1779,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-shlib-link"> + <varlistentry id="extend-postgres-pgxs-shlib-link"> <term><varname>SHLIB_LINK</varname></term> <listitem> <para> @@ -1783,7 +1788,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-config"> + <varlistentry id="extend-postgres-pgxs-pg-config"> <term><varname>PG_CONFIG</varname></term> <listitem> <para> @@ -1903,4 +1908,26 @@ make VPATH=/path/to/extension/source/tree install </tip> </sect1> + <sect1 id="extend-postgres-meson"> + <title>postgresql-extension.pc</title> + + <para> + When Postgres is built by using meson, it generates + <literal>postgresql-extension.pc</literal> pkg-config file. Extension + libraries can use this file like <literal>PGXS</literal> + (<xref linkend="extend-postgres-pgxs"/>). + + To use the <literal>postgresql-extension.pc</literal> infrastructure for + your extension, you must write a simple meson.build file. In the + meson.build file, you need to include the + <literal>postgresql-extension.pc</literal> pkg-config file. Here is an + example that builds an extension module named isbn_issn, consisting of a + shared library containing some C code, an extension control file, an SQL + script, an include file (only needed if other modules might need to access + the extension functions without going via SQL), and a documentation text + file: + </para> + + </sect1> + </chapter> diff --git a/doc/src/sgml/jit.sgml b/doc/src/sgml/jit.sgml index 44e18bf1a6f..81a4644a97d 100644 --- a/doc/src/sgml/jit.sgml +++ b/doc/src/sgml/jit.sgml @@ -223,7 +223,7 @@ SET of types <literal>C</literal> and <literal>internal</literal>, as well as operators based on such functions. To do so for functions in extensions, the definitions of those functions need to be made available. - When using <link linkend="extend-pgxs">PGXS</link> to build an extension + When using <link linkend="extend-postgres-pgxs">PGXS</link> to build an extension against a server that has been compiled with LLVM JIT support, the relevant files will be built and installed automatically. </para> -- 2.49.0 [text/x-patch] v5-0005-meson-Add-architecture-for-LLVM-bitcode-emission.patch (9.0K, 6-v5-0005-meson-Add-architecture-for-LLVM-bitcode-emission.patch) download | inline diff: From a9fc008dfb3babb14dfd88aee2a281b776cdc784 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Fri, 7 Mar 2025 12:10:58 +0300 Subject: [PATCH v5 5/7] meson: Add architecture for LLVM bitcode emission This commit adds suport for bitcode emission for both normal and generated source files (processed by bison, flex, etc). These bitcode files are installed into $pkglibdir/bitcode/ directory if the LLVM is found. New variable `bitcode_modules` is introduced to generate bitcode files. All required information is gathered in this variable. Then, this variable is processed by the main meson LLVM bitcode emission scripts: src/backend/jit/llvm/bitcode/meson.build -> src/tools/irlink. An example of a possible structure of bitcode_modules is: ``` bitcode_modules = [ { 'name': '...', 'target': ..., 'srcfiles': [ '...', '...', ], 'additional_flags': [ '-I...', '-I...', ], 'gen_srcfiles': [ { 'srcfiles': [ <custom_target for ...>, <custom_target for ...>, ], 'additional_flags': [ '-I...', '-I...', ] } ] } ] ``` Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Author: Diego Fronza <[email protected]> Reviewed-by: Diego Fronza <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/backend/jit/llvm/bitcode/meson.build | 69 ++++++++++++++++++++++++ src/backend/jit/llvm/meson.build | 31 +++++++---- src/backend/meson.build | 6 +++ meson.build | 21 ++++++++ src/tools/irlink | 25 +++++++++ 5 files changed, 141 insertions(+), 11 deletions(-) create mode 100644 src/backend/jit/llvm/bitcode/meson.build create mode 100644 src/tools/irlink diff --git a/src/backend/jit/llvm/bitcode/meson.build b/src/backend/jit/llvm/bitcode/meson.build new file mode 100644 index 00000000000..546e4d8b898 --- /dev/null +++ b/src/backend/jit/llvm/bitcode/meson.build @@ -0,0 +1,69 @@ +# Copyright (c) 2022-2024, PostgreSQL Global Development Group +# +# emit LLVM bitcode for JIT inlining + +assert(llvm.found()) + +foreach bitcode_module : bitcode_modules + bitcode_targets = [] + bitcode_obj = bitcode_module['target'] + bitcode_cflags_local = bitcode_cflags + bitcode_module.get('additional_flags', []) + bitcode_name = bitcode_module.get('name', bitcode_obj.name()) + + foreach srcfile : bitcode_module['srcfiles'] + if meson.version().version_compare('>=0.59') + srcfilename = fs.parent(srcfile) / fs.name(srcfile) + else + srcfilename = '@0@'.format(srcfile) + endif + + targetname = '@0@_@[email protected]'.format( + bitcode_name, + srcfilename.underscorify(), + ) + bitcode_targets += custom_target( + targetname, + depends: [bitcode_obj], + input: [srcfile], + output: targetname, + command: [llvm_irgen_command, bitcode_cflags_local], + install: true, + install_dir: dir_bitcode, + ) + endforeach + + # Process generated sources, which may include custom compilation flags. + foreach gen_sources: bitcode_module.get('gen_sources', []) + bitcode_cflags_gen_local = bitcode_cflags_local + gen_sources.get('additional_flags', []) + + foreach srcfile: gen_sources['srcfiles'] + # Generated sources are stored in some folder under meson.build_root()/**, + # remove the build prefix from the string. + srcfilename = srcfile.full_path().split(meson.build_root() + '/')[1] + + targetname = '@0@_@[email protected]'.format( + bitcode_name, + srcfilename.underscorify(), + ) + bitcode_targets += custom_target( + targetname, + depends: [bitcode_obj], + input: [srcfile], + output: targetname, + command: [llvm_irgen_command, bitcode_cflags_gen_local], + install: true, + install_dir: dir_bitcode, + ) + endforeach + endforeach + + index_name = '@[email protected]'.format(bitcode_name) + bitcode_index = custom_target('@0@'.format(bitcode_name), + output: index_name, + input: bitcode_targets, + command: [irlink, '--lto', llvm_lto, '--outdir', '@OUTDIR@', '--index', index_name, '@INPUT@'], + install: true, + install_dir: dir_bitcode, + ) + backend_targets += bitcode_index +endforeach diff --git a/src/backend/jit/llvm/meson.build b/src/backend/jit/llvm/meson.build index c8e06dfbe35..2dd922e573b 100644 --- a/src/backend/jit/llvm/meson.build +++ b/src/backend/jit/llvm/meson.build @@ -42,21 +42,22 @@ backend_targets += llvmjit # Define a few bits and pieces used here and elsewhere to generate bitcode -llvm_irgen_args = [ - '-c', '-o', '@OUTPUT@', '@INPUT@', +llvm_irgen_command = [] +if ccache.found() + llvm_irgen_command += ccache +endif + +llvm_irgen_command += [ + clang, + '-c', '-o', '@OUTPUT0@', '@INPUT0@', '-flto=thin', '-emit-llvm', - '-MD', '-MQ', '@OUTPUT@', '-MF', '@DEPFILE@', '-O2', '-Wno-ignored-attributes', '-Wno-empty-body', + '-Wno-unknown-warning-option', + '-Wno-compound-token-split-by-macro', ] - -if ccache.found() - llvm_irgen_command = ccache - llvm_irgen_args = [clang.path()] + llvm_irgen_args -else - llvm_irgen_command = clang -endif +llvm_irgen_dep_args = ['-MD', '-MQ', '@OUTPUT0@', '-MF', '@DEPFILE@'] # XXX: Need to determine proper version of the function cflags for clang @@ -73,7 +74,7 @@ bitcode_cflags += '-I@SOURCE_ROOT@/src/include' # Note this is intentionally not installed to bitcodedir, as it's not for # inlining llvmjit_types = custom_target('llvmjit_types.bc', - command: [llvm_irgen_command] + llvm_irgen_args + bitcode_cflags, + command: llvm_irgen_command + llvm_irgen_dep_args + bitcode_cflags, input: 'llvmjit_types.c', output: 'llvmjit_types.bc', depends: [postgres], @@ -82,3 +83,11 @@ llvmjit_types = custom_target('llvmjit_types.bc', depfile: '@[email protected]', ) backend_targets += llvmjit_types + +# Figure out -I's needed to build all postgres code, including all its +# dependencies +pkg_config = find_program(['pkg-config', 'pkgconf'], required: true) +r = run_command(pkg_config, + ['--cflags-only-I', meson.build_root() / 'meson-uninstalled/postgresql-extension-uninstalled.pc'], + check: true) +bitcode_cflags += r.stdout().split() diff --git a/src/backend/meson.build b/src/backend/meson.build index 9d79d4d058c..5fb33660d3d 100644 --- a/src/backend/meson.build +++ b/src/backend/meson.build @@ -140,6 +140,12 @@ postgres = executable('postgres', backend_targets += postgres +bitcode_modules += { + 'name': 'postgres', + 'target': postgres_lib, + 'srcfiles': backend_sources, +} + pg_mod_c_args = cflags_mod pg_mod_cpp_args = cxxflags_mod pg_mod_link_args = ldflags_sl + ldflags_mod diff --git a/meson.build b/meson.build index 1b7388096a6..19e46df5d07 100644 --- a/meson.build +++ b/meson.build @@ -814,6 +814,8 @@ if add_languages('cpp', required: llvmopt, native: false) # Some distros put LLVM and clang in different paths, so fallback to # find via PATH, too. clang = find_program(llvm_binpath / 'clang', 'clang', required: true) + llvm_lto = find_program(llvm_binpath / 'llvm-lto', required: true) + irlink = find_program('src/tools/irlink', native: true) endif elif llvmopt.auto() message('llvm requires a C++ compiler') @@ -3025,6 +3027,11 @@ test_deps = [] tests = [] meson_extension_tests = [] +# List of object files + source files to generated LLVM IR for inlining. +# Each element is a hash of: +# {'target': target, 'srcfiles': ..., 'additional_flags': ...}. +bitcode_modules = [] + # Default options for targets @@ -3346,6 +3353,11 @@ subdir('src/interfaces/ecpg/test') subdir('doc/src/sgml') +# generate bitcode for JIT inlining after giving contrib modules etc a chance +# to add themselves to bitcode_modules[] +subdir('src/backend/jit/llvm/bitcode', if_found: llvm) + + generated_sources_ac += {'': ['GNUmakefile']} # After processing src/test, add test_install_libs to the testprep_targets @@ -3981,6 +3993,15 @@ if meson.version().version_compare('>=0.57') section: 'Programs', ) + if llvm.found() + summary( + { + 'clang': clang, + }, + section: 'Programs', + ) + endif + summary( { 'bonjour': bonjour, diff --git a/src/tools/irlink b/src/tools/irlink new file mode 100644 index 00000000000..793c0abf91a --- /dev/null +++ b/src/tools/irlink @@ -0,0 +1,25 @@ +#!/usr/bin/env python3 + +import argparse +import os +import shutil +import subprocess +import sys + +parser = argparse.ArgumentParser( + description='generate PostgreSQL JIT IR module') + +parser.add_argument('--index', type=str, required=True) +parser.add_argument('--lto', type=str, required=True) +parser.add_argument('--outdir', type=str, required=True) +parser.add_argument('INPUT', type=str, nargs='+') + +args = parser.parse_args() + +file_names = [os.path.basename(f) for f in args.INPUT] +command = [args.lto, + '-thinlto', '-thinlto-action=thinlink', + '-o', args.index] + file_names +res = subprocess.run(command, cwd=args.outdir) + +exit(res.returncode) -- 2.49.0 [text/x-patch] v5-0006-meson-Add-LLVM-bitcode-emissions-for-contrib-libr.patch (22.7K, 7-v5-0006-meson-Add-LLVM-bitcode-emissions-for-contrib-libr.patch) download | inline diff: From 31801011c5a9698aded26829081906156dfa1ef1 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Wed, 12 Mar 2025 10:49:18 +0300 Subject: [PATCH v5 6/7] meson: Add LLVM bitcode emissions for contrib libraries The libraries which the bitcode files will be generated in are selected manually. Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Author: Diego Fronza <[email protected]> Reviewed-by: Diego Fronza <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/interfaces/libpq/meson.build | 3 +++ contrib/bloom/meson.build | 5 +++++ contrib/bool_plperl/meson.build | 9 +++++++++ contrib/btree_gin/meson.build | 5 +++++ contrib/btree_gist/meson.build | 5 +++++ contrib/citext/meson.build | 5 +++++ contrib/cube/meson.build | 13 +++++++++++++ contrib/dict_int/meson.build | 5 +++++ contrib/dict_xsyn/meson.build | 5 +++++ contrib/earthdistance/meson.build | 6 ++++++ contrib/fuzzystrmatch/meson.build | 9 +++++++++ contrib/hstore/meson.build | 7 +++++++ contrib/hstore_plperl/meson.build | 10 ++++++++++ contrib/hstore_plpython/meson.build | 12 ++++++++++++ contrib/intarray/meson.build | 5 +++++ contrib/isn/meson.build | 5 +++++ contrib/jsonb_plperl/meson.build | 8 ++++++++ contrib/jsonb_plpython/meson.build | 10 ++++++++++ contrib/lo/meson.build | 5 +++++ contrib/ltree/meson.build | 10 ++++++++++ contrib/ltree_plpython/meson.build | 11 +++++++++++ contrib/pg_buffercache/meson.build | 5 +++++ contrib/pg_freespacemap/meson.build | 5 +++++ contrib/pg_logicalinspect/meson.build | 5 +++++ contrib/pg_surgery/meson.build | 4 ++++ contrib/pg_trgm/meson.build | 5 +++++ contrib/pgcrypto/meson.build | 4 ++++ contrib/seg/meson.build | 12 ++++++++++++ contrib/spi/meson.build | 20 ++++++++++++++++++++ contrib/sslinfo/meson.build | 5 +++++ contrib/tablefunc/meson.build | 5 +++++ contrib/tcn/meson.build | 5 +++++ contrib/tsm_system_rows/meson.build | 5 +++++ contrib/tsm_system_time/meson.build | 5 +++++ contrib/unaccent/meson.build | 5 +++++ contrib/uuid-ossp/meson.build | 5 +++++ contrib/xml2/meson.build | 5 +++++ src/pl/plperl/meson.build | 1 + src/pl/plpython/meson.build | 1 + meson.build | 1 + 40 files changed, 256 insertions(+) diff --git a/src/interfaces/libpq/meson.build b/src/interfaces/libpq/meson.build index 292fecf3320..ced3151edfe 100644 --- a/src/interfaces/libpq/meson.build +++ b/src/interfaces/libpq/meson.build @@ -50,6 +50,9 @@ export_file = custom_target('libpq.exports', libpq_inc = include_directories('.', '../../port') libpq_c_args = ['-DSO_MAJOR_VERSION=5'] +# libpq-fe.h is needed to generate bitcode for some extensions +libpq_dir = meson.current_source_dir() + # Not using both_libraries() here as # 1) resource files should only be in the shared library # 2) we want the .pc file to include a dependency to {pgport,common}_static for diff --git a/contrib/bloom/meson.build b/contrib/bloom/meson.build index 695712a455e..1090c281532 100644 --- a/contrib/bloom/meson.build +++ b/contrib/bloom/meson.build @@ -28,6 +28,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': bloom, + 'srcfiles': bloom_sources, +} + tests += { 'name': 'bloom', 'sd': meson.current_source_dir(), diff --git a/contrib/bool_plperl/meson.build b/contrib/bool_plperl/meson.build index f489d99044c..1758adb7489 100644 --- a/contrib/bool_plperl/meson.build +++ b/contrib/bool_plperl/meson.build @@ -37,6 +37,15 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': bool_plperl, + 'srcfiles': bool_plperl_sources, + 'additional_flags': [ + '-I@0@'.format(plperl_dir), + perl_ccflags + ] +} + tests += { 'name': 'bool_plperl', 'sd': meson.current_source_dir(), diff --git a/contrib/btree_gin/meson.build b/contrib/btree_gin/meson.build index b2749f6e669..26d49d005ab 100644 --- a/contrib/btree_gin/meson.build +++ b/contrib/btree_gin/meson.build @@ -25,6 +25,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': btree_gin, + 'srcfiles': btree_gin_sources, +} + tests += { 'name': 'btree_gin', 'sd': meson.current_source_dir(), diff --git a/contrib/btree_gist/meson.build b/contrib/btree_gist/meson.build index 89932dd3844..6a276ec02f4 100644 --- a/contrib/btree_gist/meson.build +++ b/contrib/btree_gist/meson.build @@ -55,6 +55,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': btree_gist, + 'srcfiles': btree_gist_sources, +} + tests += { 'name': 'btree_gist', 'sd': meson.current_source_dir(), diff --git a/contrib/citext/meson.build b/contrib/citext/meson.build index 7fff34f2368..fa07ff72be4 100644 --- a/contrib/citext/meson.build +++ b/contrib/citext/meson.build @@ -30,6 +30,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': citext, + 'srcfiles': citext_sources, +} + tests += { 'name': 'citext', 'sd': meson.current_source_dir(), diff --git a/contrib/cube/meson.build b/contrib/cube/meson.build index fd3c057f469..a7649a74a1f 100644 --- a/contrib/cube/meson.build +++ b/contrib/cube/meson.build @@ -3,6 +3,7 @@ cube_sources = files( 'cube.c', ) +bc_cube_sources = cube_sources cube_scan = custom_target('cubescan', input: 'cubescan.l', @@ -11,6 +12,7 @@ cube_scan = custom_target('cubescan', ) generated_sources += cube_scan cube_sources += cube_scan +bc_cube_gen_sources = [{'srcfiles': [cube_scan]}] cube_parse = custom_target('cubeparse', input: 'cubeparse.y', @@ -18,6 +20,7 @@ cube_parse = custom_target('cubeparse', ) generated_sources += cube_parse.to_list() cube_sources += cube_parse +bc_cube_gen_sources += {'srcfiles': [cube_parse[0]]} if host_system == 'windows' cube_sources += rc_lib_gen.process(win32ver_rc, extra_args: [ @@ -48,6 +51,16 @@ install_headers( install_dir: dir_include_extension / 'cube', ) +cube_dir = meson.current_source_dir() +bitcode_modules += { + 'target': cube, + 'srcfiles': bc_cube_sources, + 'gen_sources': bc_cube_gen_sources, + 'additional_flags': [ + '-I@0@'.format(cube_dir), + ] +} + tests += { 'name': 'cube', 'sd': meson.current_source_dir(), diff --git a/contrib/dict_int/meson.build b/contrib/dict_int/meson.build index ab41588547b..894c8a9c94d 100644 --- a/contrib/dict_int/meson.build +++ b/contrib/dict_int/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': dict_int, + 'srcfiles': dict_int_sources, +} + tests += { 'name': 'dict_int', 'sd': meson.current_source_dir(), diff --git a/contrib/dict_xsyn/meson.build b/contrib/dict_xsyn/meson.build index 93f41b1f963..d4bd4a6ef5a 100644 --- a/contrib/dict_xsyn/meson.build +++ b/contrib/dict_xsyn/meson.build @@ -29,6 +29,11 @@ install_data( } ) +bitcode_modules += { + 'target': dict_xsyn, + 'srcfiles': dict_xsyn_sources, +} + tests += { 'name': 'dict_xsyn', 'sd': meson.current_source_dir(), diff --git a/contrib/earthdistance/meson.build b/contrib/earthdistance/meson.build index a5f8d1a3ffa..c101359b118 100644 --- a/contrib/earthdistance/meson.build +++ b/contrib/earthdistance/meson.build @@ -24,6 +24,12 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': earthdistance, + 'srcfiles': earthdistance_sources, +} + + tests += { 'name': 'earthdistance', 'sd': meson.current_source_dir(), diff --git a/contrib/fuzzystrmatch/meson.build b/contrib/fuzzystrmatch/meson.build index f52daa4ea1c..fc5e88d0c92 100644 --- a/contrib/fuzzystrmatch/meson.build +++ b/contrib/fuzzystrmatch/meson.build @@ -5,6 +5,7 @@ fuzzystrmatch_sources = files( 'dmetaphone.c', 'fuzzystrmatch.c', ) +bc_fuzzystrmatch_sources = fuzzystrmatch_sources daitch_mokotoff_h = custom_target('daitch_mokotoff', input: 'daitch_mokotoff_header.pl', @@ -35,6 +36,14 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': fuzzystrmatch, + 'srcfiles': bc_fuzzystrmatch_sources, + 'additional_flags': [ + '-I@0@'.format(meson.current_build_dir()) + ] +} + tests += { 'name': 'fuzzystrmatch', 'sd': meson.current_source_dir(), diff --git a/contrib/hstore/meson.build b/contrib/hstore/meson.build index 39622d1024d..344a620273f 100644 --- a/contrib/hstore/meson.build +++ b/contrib/hstore/meson.build @@ -43,6 +43,13 @@ install_headers( install_dir: dir_include_extension / 'hstore', ) +# some libraries include "hstore/hstore.h" instead of "hstore.h" +hstore_dir_up = join_paths(meson.current_source_dir(), '..') +bitcode_modules += { + 'target': hstore, + 'srcfiles': hstore_sources, +} + tests += { 'name': 'hstore', 'sd': meson.current_source_dir(), diff --git a/contrib/hstore_plperl/meson.build b/contrib/hstore_plperl/meson.build index 60b8ad23569..b689b55e1c9 100644 --- a/contrib/hstore_plperl/meson.build +++ b/contrib/hstore_plperl/meson.build @@ -37,6 +37,16 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': hstore_plperl, + 'srcfiles': hstore_plperl_sources, + 'additional_flags': [ + '-I@0@'.format(hstore_dir_up), + '-I@0@'.format(plperl_dir), + perl_ccflags, + ] +} + tests += { 'name': 'hstore_plperl', 'sd': meson.current_source_dir(), diff --git a/contrib/hstore_plpython/meson.build b/contrib/hstore_plpython/meson.build index ea8e20a377b..d68de3dc46b 100644 --- a/contrib/hstore_plpython/meson.build +++ b/contrib/hstore_plpython/meson.build @@ -30,6 +30,18 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': hstore_plpython, + 'srcfiles': hstore_plpython_sources, + 'additional_flags': [ + '-I@0@'.format(hstore_dir_up), + '-DPLPYTHON_LIBNAME="plpython3"', + '-I@0@'.format(python3_inc_dir), + '-I@0@'.format(plpython_dir), + perl_ccflags, + ] +} + hstore_plpython_regress = [ 'hstore_plpython' ] diff --git a/contrib/intarray/meson.build b/contrib/intarray/meson.build index fae9add981d..3d7da0c9c9a 100644 --- a/contrib/intarray/meson.build +++ b/contrib/intarray/meson.build @@ -33,6 +33,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': intarray, + 'srcfiles': intarray_sources, +} + tests += { 'name': 'intarray', 'sd': meson.current_source_dir(), diff --git a/contrib/isn/meson.build b/contrib/isn/meson.build index 39cf781684e..71dc8cf1a76 100644 --- a/contrib/isn/meson.build +++ b/contrib/isn/meson.build @@ -30,6 +30,11 @@ install_headers( install_dir: dir_include_extension / 'isn', ) +bitcode_modules += { + 'target': isn, + 'srcfiles': isn_sources, +} + tests += { 'name': 'isn', 'sd': meson.current_source_dir(), diff --git a/contrib/jsonb_plperl/meson.build b/contrib/jsonb_plperl/meson.build index 95a9a7bc082..febafc73a99 100644 --- a/contrib/jsonb_plperl/meson.build +++ b/contrib/jsonb_plperl/meson.build @@ -37,6 +37,14 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': jsonb_plperl, + 'srcfiles': jsonb_plperl_sources, + 'additional_flags': [ + '-I@0@'.format(plperl_dir), + perl_ccflags, + ] +} tests += { 'name': 'jsonb_plperl', diff --git a/contrib/jsonb_plpython/meson.build b/contrib/jsonb_plpython/meson.build index 5fe80483e58..6be92221272 100644 --- a/contrib/jsonb_plpython/meson.build +++ b/contrib/jsonb_plpython/meson.build @@ -30,6 +30,16 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': jsonb_plpython, + 'srcfiles': jsonb_plpython_sources, + 'additional_flags': [ + '-DPLPYTHON_LIBNAME="plpython3"', + '-I@0@'.format(python3_inc_dir), + '-I@0@'.format(plpython_dir), + ] +} + jsonb_plpython_regress = [ 'jsonb_plpython' ] diff --git a/contrib/lo/meson.build b/contrib/lo/meson.build index 2d78907ba12..1222faa9169 100644 --- a/contrib/lo/meson.build +++ b/contrib/lo/meson.build @@ -24,6 +24,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': lo, + 'srcfiles': lo_sources, +} + tests += { 'name': 'lo', 'sd': meson.current_source_dir(), diff --git a/contrib/ltree/meson.build b/contrib/ltree/meson.build index f9b06302839..cda86935544 100644 --- a/contrib/ltree/meson.build +++ b/contrib/ltree/meson.build @@ -41,6 +41,16 @@ install_headers( install_dir: dir_include_extension / 'ltree', ) +ltree_dir = meson.current_source_dir() +ltree_dir_up = join_paths(ltree_dir, '..') +bitcode_modules += { + 'target': ltree, + 'srcfiles': ltree_sources, + 'additional_flags': [ + '-I@0@'.format(ltree_dir) + ] +} + tests += { 'name': 'ltree', 'sd': meson.current_source_dir(), diff --git a/contrib/ltree_plpython/meson.build b/contrib/ltree_plpython/meson.build index a37732c486b..74b285da05c 100644 --- a/contrib/ltree_plpython/meson.build +++ b/contrib/ltree_plpython/meson.build @@ -30,6 +30,17 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': ltree_plpython, + 'srcfiles': ltree_plpython_sources, + 'additional_flags': [ + '-I@0@'.format(ltree_dir_up), + '-DPLPYTHON_LIBNAME="plpython3"', + '-I@0@'.format(python3_inc_dir), + '-I@0@'.format(plpython_dir), + ] +} + ltree_plpython_regress = [ 'ltree_plpython' ] diff --git a/contrib/pg_buffercache/meson.build b/contrib/pg_buffercache/meson.build index 7cd039a1df9..65dcc690601 100644 --- a/contrib/pg_buffercache/meson.build +++ b/contrib/pg_buffercache/meson.build @@ -28,6 +28,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pg_buffercache, + 'srcfiles': pg_buffercache_sources, +} + tests += { 'name': 'pg_buffercache', 'sd': meson.current_source_dir(), diff --git a/contrib/pg_freespacemap/meson.build b/contrib/pg_freespacemap/meson.build index ff8eda3580e..120def1ad2e 100644 --- a/contrib/pg_freespacemap/meson.build +++ b/contrib/pg_freespacemap/meson.build @@ -25,6 +25,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pg_freespacemap, + 'srcfiles': pg_freespacemap_sources, +} + tests += { 'name': 'pg_freespacemap', 'sd': meson.current_source_dir(), diff --git a/contrib/pg_logicalinspect/meson.build b/contrib/pg_logicalinspect/meson.build index 5c0528a8c63..1b4f46277ed 100644 --- a/contrib/pg_logicalinspect/meson.build +++ b/contrib/pg_logicalinspect/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pg_logicalinspect, + 'srcfiles': pg_logicalinspect_sources, +} + tests += { 'name': 'pg_logicalinspect', 'sd': meson.current_source_dir(), diff --git a/contrib/pg_surgery/meson.build b/contrib/pg_surgery/meson.build index c6cfa9c4694..9d7199392c7 100644 --- a/contrib/pg_surgery/meson.build +++ b/contrib/pg_surgery/meson.build @@ -22,6 +22,10 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pg_surgery, + 'srcfiles': pg_surgery_sources, +} tests += { 'name': 'pg_surgery', diff --git a/contrib/pg_trgm/meson.build b/contrib/pg_trgm/meson.build index a31aa5c574d..408e287172e 100644 --- a/contrib/pg_trgm/meson.build +++ b/contrib/pg_trgm/meson.build @@ -32,6 +32,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pg_trgm, + 'srcfiles': pg_trgm_sources, +} + tests += { 'name': 'pg_trgm', 'sd': meson.current_source_dir(), diff --git a/contrib/pgcrypto/meson.build b/contrib/pgcrypto/meson.build index 7d5ef9b6d32..65d94174f7e 100644 --- a/contrib/pgcrypto/meson.build +++ b/contrib/pgcrypto/meson.build @@ -100,6 +100,10 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pgcrypto, + 'srcfiles': pgcrypto_sources, +} tests += { 'name': 'pgcrypto', diff --git a/contrib/seg/meson.build b/contrib/seg/meson.build index e331e097230..371cd272a06 100644 --- a/contrib/seg/meson.build +++ b/contrib/seg/meson.build @@ -3,6 +3,7 @@ seg_sources = files( 'seg.c', ) +bc_seg_sources = seg_sources seg_scan = custom_target('segscan', input: 'segscan.l', @@ -11,6 +12,7 @@ seg_scan = custom_target('segscan', ) generated_sources += seg_scan seg_sources += seg_scan +bc_seg_gen_sources = [{'srcfiles': [seg_scan]}] seg_parse = custom_target('segparse', input: 'segparse.y', @@ -18,6 +20,7 @@ seg_parse = custom_target('segparse', ) generated_sources += seg_parse.to_list() seg_sources += seg_parse +bc_seg_gen_sources += {'srcfiles': [seg_parse[0]]} if host_system == 'windows' seg_sources += rc_lib_gen.process(win32ver_rc, extra_args: [ @@ -47,6 +50,15 @@ install_headers( install_dir: dir_include_extension / 'seg', ) +bitcode_modules += { + 'target': seg, + 'srcfiles': bc_seg_sources, + 'gen_sources': bc_seg_gen_sources, + 'additional_flags': [ + '-I@0@'.format(meson.current_source_dir()), + ] +} + tests += { 'name': 'seg', 'sd': meson.current_source_dir(), diff --git a/contrib/spi/meson.build b/contrib/spi/meson.build index 3832a91019a..c8216fcbcd0 100644 --- a/contrib/spi/meson.build +++ b/contrib/spi/meson.build @@ -24,6 +24,11 @@ install_data('autoinc.example', kwargs: contrib_doc_args, ) +bitcode_modules += { + 'target': autoinc, + 'srcfiles': autoinc_sources, +} + insert_username_sources = files( 'insert_username.c', @@ -51,6 +56,11 @@ install_data('insert_username.example', kwargs: contrib_doc_args, ) +bitcode_modules += { + 'target': insert_username, + 'srcfiles': insert_username_sources, +} + moddatetime_sources = files( 'moddatetime.c', @@ -78,6 +88,11 @@ install_data('moddatetime.example', kwargs: contrib_doc_args, ) +bitcode_modules += { + 'target': moddatetime, + 'srcfiles': moddatetime_sources, +} + # this is needed for the regression tests; # comment out if you want a quieter refint package for other uses @@ -108,6 +123,11 @@ install_data('refint.example', kwargs: contrib_doc_args, ) +bitcode_modules += { + 'target': refint, + 'srcfiles': refint_sources, +} + tests += { 'name': 'spi', 'sd': meson.current_source_dir(), diff --git a/contrib/sslinfo/meson.build b/contrib/sslinfo/meson.build index 4c513759200..5750d783b92 100644 --- a/contrib/sslinfo/meson.build +++ b/contrib/sslinfo/meson.build @@ -29,3 +29,8 @@ install_data( 'sslinfo.control', kwargs: contrib_data_args, ) + +bitcode_modules += { + 'target': sslinfo, + 'srcfiles': sslinfo_sources, +} diff --git a/contrib/tablefunc/meson.build b/contrib/tablefunc/meson.build index ee67272ec0a..6c26ffafddf 100644 --- a/contrib/tablefunc/meson.build +++ b/contrib/tablefunc/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': tablefunc, + 'srcfiles': tablefunc_sources, +} + tests += { 'name': 'tablefunc', 'sd': meson.current_source_dir(), diff --git a/contrib/tcn/meson.build b/contrib/tcn/meson.build index 6ffb136af90..df71e8a91f9 100644 --- a/contrib/tcn/meson.build +++ b/contrib/tcn/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': tcn, + 'srcfiles': tcn_sources, +} + tests += { 'name': 'tcn', 'sd': meson.current_source_dir(), diff --git a/contrib/tsm_system_rows/meson.build b/contrib/tsm_system_rows/meson.build index b8cece3d80f..1ea74cce949 100644 --- a/contrib/tsm_system_rows/meson.build +++ b/contrib/tsm_system_rows/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': tsm_system_rows, + 'srcfiles': tsm_system_rows_sources, +} + tests += { 'name': 'tsm_system_rows', 'sd': meson.current_source_dir(), diff --git a/contrib/tsm_system_time/meson.build b/contrib/tsm_system_time/meson.build index 8a143a8f8e6..1f6b98d3ec4 100644 --- a/contrib/tsm_system_time/meson.build +++ b/contrib/tsm_system_time/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': tsm_system_time, + 'srcfiles': tsm_system_time_sources, +} + tests += { 'name': 'tsm_system_time', 'sd': meson.current_source_dir(), diff --git a/contrib/unaccent/meson.build b/contrib/unaccent/meson.build index 33d4649bae1..1f5d120e3d8 100644 --- a/contrib/unaccent/meson.build +++ b/contrib/unaccent/meson.build @@ -28,6 +28,11 @@ install_data( install_dir: dir_data / 'tsearch_data' ) +bitcode_modules += { + 'target': unaccent, + 'srcfiles': unaccent_sources, +} + # XXX: Implement downlo tests += { 'name': 'unaccent', diff --git a/contrib/uuid-ossp/meson.build b/contrib/uuid-ossp/meson.build index 982f27c085f..8d21bb0a996 100644 --- a/contrib/uuid-ossp/meson.build +++ b/contrib/uuid-ossp/meson.build @@ -29,6 +29,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': uuid_ossp, + 'srcfiles': uuid_ossp_sources, +} + tests += { 'name': 'uuid-ossp', 'sd': meson.current_source_dir(), diff --git a/contrib/xml2/meson.build b/contrib/xml2/meson.build index 08d3c3b8e30..b23a8fe0cd7 100644 --- a/contrib/xml2/meson.build +++ b/contrib/xml2/meson.build @@ -32,6 +32,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': xml2, + 'srcfiles': xml2_sources, +} + tests += { 'name': 'xml2', 'sd': meson.current_source_dir(), diff --git a/src/pl/plperl/meson.build b/src/pl/plperl/meson.build index b463d4d56c5..279277dee73 100644 --- a/src/pl/plperl/meson.build +++ b/src/pl/plperl/meson.build @@ -37,6 +37,7 @@ foreach n : ['SPI', 'Util'] plperl_sources += xs_c endforeach +plperl_dir = meson.current_source_dir() plperl_inc = include_directories('.') if host_system == 'windows' diff --git a/src/pl/plpython/meson.build b/src/pl/plpython/meson.build index 709e5932a93..e2c19771d85 100644 --- a/src/pl/plpython/meson.build +++ b/src/pl/plpython/meson.build @@ -29,6 +29,7 @@ plpython_sources += custom_target('spiexceptions.h', # FIXME: need to duplicate import library ugliness? plpython_inc = include_directories('.') +plpython_dir = meson.current_source_dir() if host_system == 'windows' plpython_sources += rc_lib_gen.process(win32ver_rc, extra_args: [ diff --git a/meson.build b/meson.build index 19e46df5d07..18ba5fe9f0b 100644 --- a/meson.build +++ b/meson.build @@ -1272,6 +1272,7 @@ if not pyopt.disabled() python3_inst = pm.find_installation(python.path(), required: pyopt) if python3_inst.found() python3_dep = python3_inst.dependency(embed: true, required: pyopt) + python3_inc_dir = python3_inst.get_variable('INCLUDEPY') # Remove this check after we depend on Meson >= 1.1.0 if not cc.check_header('Python.h', dependencies: python3_dep, required: pyopt, include_directories: postgres_inc) python3_dep = not_found_dep -- 2.49.0 [text/x-patch] v5-0007-meson-Add-LLVM-bitcode-emission-for-backend-sourc.patch (5.2K, 8-v5-0007-meson-Add-LLVM-bitcode-emission-for-backend-sourc.patch) download | inline diff: From a53a675701fce5f2591fa1d454d6691a90853265 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Wed, 12 Mar 2025 10:44:46 +0300 Subject: [PATCH v5 7/7] meson: Add LLVM bitcode emission for backend sources Since generated backend sources may have their own compilation flags and must also be included in the postgres.index.bc, the way to make it work with current code was to create a new variable, called `bc_generated_backend_sources`, which is a list of dictionaries, each one having an optional 'additional_flags' and a `srclist` pointing to the list of custom_target generated sources. An example of a possible structure of bitcode_modules which is processed by the main meson llvm bitcode emission file src/backend/jit/llvm/bitcode/meson.build: ``` bitcode_modules = [ { 'name': 'postgres', 'target': postgres_lib, 'src_file': backend_sources, 'gen_srcfiles': [ { 'additional_flags': [ '-I/path/postgresl/src/backend/parser', '-I/path/postgresl/build/src/backend/parser', ], 'srcfiles': [ <custom_target for scan.c>, <custom_target for gram.c> ] } ] } ] ``` Author: Diego Fronza <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/backend/bootstrap/meson.build | 2 ++ src/backend/meson.build | 2 ++ src/backend/parser/meson.build | 8 ++++++++ src/backend/replication/meson.build | 4 ++++ src/backend/utils/fmgr/meson.build | 1 + 5 files changed, 17 insertions(+) diff --git a/src/backend/bootstrap/meson.build b/src/backend/bootstrap/meson.build index 29726c1ab4f..389c75f8081 100644 --- a/src/backend/bootstrap/meson.build +++ b/src/backend/bootstrap/meson.build @@ -13,6 +13,7 @@ bootscanner = custom_target('bootscanner', ) generated_sources += bootscanner boot_parser_sources += bootscanner +bc_generated_backend_sources += {'srcfiles': [bootscanner]} bootparse = custom_target('bootparse', input: 'bootparse.y', @@ -20,6 +21,7 @@ bootparse = custom_target('bootparse', ) generated_sources += bootparse.to_list() boot_parser_sources += bootparse +bc_generated_backend_sources += {'srcfiles': [bootparse[0]]} boot_parser = static_library('boot_parser', boot_parser_sources, diff --git a/src/backend/meson.build b/src/backend/meson.build index 5fb33660d3d..85453de1928 100644 --- a/src/backend/meson.build +++ b/src/backend/meson.build @@ -5,6 +5,7 @@ backend_sources = [] backend_link_with = [pgport_srv, common_srv] generated_backend_sources = [] +bc_generated_backend_sources = [] post_export_backend_sources = [] subdir('access') @@ -144,6 +145,7 @@ bitcode_modules += { 'name': 'postgres', 'target': postgres_lib, 'srcfiles': backend_sources, + 'gen_sources': bc_generated_backend_sources, } pg_mod_c_args = cflags_mod diff --git a/src/backend/parser/meson.build b/src/backend/parser/meson.build index 874aa749aa6..add472a0cd8 100644 --- a/src/backend/parser/meson.build +++ b/src/backend/parser/meson.build @@ -42,6 +42,14 @@ backend_parser = custom_target('gram', generated_sources += backend_parser.to_list() parser_sources += backend_parser +bc_generated_backend_sources += { + 'additional_flags': [ + '-I@0@'.format(meson.current_build_dir()), + '-I@0@'.format(meson.current_source_dir()), + ], + 'srcfiles': [backend_scanner, backend_parser[0]], +} + parser = static_library('parser', parser_sources, dependencies: [backend_code], diff --git a/src/backend/replication/meson.build b/src/backend/replication/meson.build index b0601498865..71ab1164960 100644 --- a/src/backend/replication/meson.build +++ b/src/backend/replication/meson.build @@ -19,6 +19,7 @@ repl_scanner = custom_target('repl_scanner', ) generated_sources += repl_scanner repl_parser_sources += repl_scanner +bc_generated_backend_sources += {'srcfiles': [repl_scanner]} repl_gram = custom_target('repl_gram', input: 'repl_gram.y', @@ -26,6 +27,7 @@ repl_gram = custom_target('repl_gram', ) generated_sources += repl_gram.to_list() repl_parser_sources += repl_gram +bc_generated_backend_sources += {'srcfiles': [repl_gram[0]]} syncrep_scanner = custom_target('syncrep_scanner', input: 'syncrep_scanner.l', @@ -34,6 +36,7 @@ syncrep_scanner = custom_target('syncrep_scanner', ) generated_sources += syncrep_scanner repl_parser_sources += syncrep_scanner +bc_generated_backend_sources += {'srcfiles': [syncrep_scanner]} syncrep_gram = custom_target('syncrep_gram', input: 'syncrep_gram.y', @@ -41,6 +44,7 @@ syncrep_gram = custom_target('syncrep_gram', ) generated_sources += syncrep_gram.to_list() repl_parser_sources += syncrep_gram +bc_generated_backend_sources += {'srcfiles': [syncrep_gram[0]]} repl_parser = static_library('repl_parser', repl_parser_sources, diff --git a/src/backend/utils/fmgr/meson.build b/src/backend/utils/fmgr/meson.build index b1dcab93e70..080f59988e9 100644 --- a/src/backend/utils/fmgr/meson.build +++ b/src/backend/utils/fmgr/meson.build @@ -8,3 +8,4 @@ backend_sources += files( # fmgrtab.c generated_backend_sources += fmgrtab_target[2] +bc_generated_backend_sources += {'srcfiles': [fmgrtab_target[2]]} -- 2.49.0 ^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: meson vs. llvm bitcode files @ 2025-07-07 08:45 Nazir Bilal Yavuz <[email protected]> parent: Nazir Bilal Yavuz <[email protected]> 0 siblings, 1 reply; 10+ messages in thread From: Nazir Bilal Yavuz @ 2025-07-07 08:45 UTC (permalink / raw) To: Diego Fronza <[email protected]>; +Cc: Peter Eisentraut <[email protected]>; pgsql-hackers; Andres Freund <[email protected]> Hi, On Tue, 29 Apr 2025 at 11:23, Nazir Bilal Yavuz <[email protected]> wrote: > > Hi, > > On Thu, 13 Mar 2025 at 13:11, Nazir Bilal Yavuz <[email protected]> wrote: > > One thing I noticed is that gen_srcfiles['srcfiles'] seems wrong. > > gen_sources is a better name compared to gen_srcfiles. So, I changed > > it to gen_sources in v4. > > Rebase is needed due to b1720fe63f, v5 is attached. Mandatory rebase, v6 is attached. -- Regards, Nazir Bilal Yavuz Microsoft Attachments: [text/x-patch] v6-0001-meson-Add-generated-header-stamps.patch (5.0K, 2-v6-0001-meson-Add-generated-header-stamps.patch) download | inline diff: From 5730e84c6a1d3edaa43af8c714ce219449ee7cf2 Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Thu, 24 Oct 2024 07:23:05 -0400 Subject: [PATCH v6 1/7] meson: Add generated header stamps Otherwise build commands become too long and this has visible effect on creation time of meson build files. Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/include/meson.build | 18 ++++++++++++++++++ src/backend/meson.build | 2 +- src/fe_utils/meson.build | 2 +- meson.build | 16 +++++++++------- 4 files changed, 29 insertions(+), 9 deletions(-) diff --git a/src/include/meson.build b/src/include/meson.build index 2e4b7aa529e..c488a5dc4c9 100644 --- a/src/include/meson.build +++ b/src/include/meson.build @@ -177,3 +177,21 @@ install_subdir('catalog', # autoconf generates the file there, ensure we get a conflict generated_sources_ac += {'src/include': ['stamp-h']} + +# Instead of having targets depending directly on the generated headers, have +# them depend on a stamp files for all of them. Dependencies on headers are +# implemented as order-only dependencies in meson (later using compiler +# generated dependencies). The benefit of using a stamp file is that it makes +# ninja.build smaller and meson setup faster. +generated_headers_stamp = custom_target('generated-headers-stamp.h', + output: 'generated-headers-stamp.h', + input: generated_headers, + command: stamp_cmd, +) + +generated_backend_headers_stamp = custom_target('generated-backend-headers-stamp.h', + output: 'generated-backend-headers-stamp.h', + input: generated_backend_headers, + depends: generated_headers_stamp, + command: stamp_cmd, +) diff --git a/src/backend/meson.build b/src/backend/meson.build index 2b0db214804..7fc649c3ebd 100644 --- a/src/backend/meson.build +++ b/src/backend/meson.build @@ -169,7 +169,7 @@ backend_mod_code = declare_dependency( compile_args: pg_mod_c_args, include_directories: postgres_inc, link_args: pg_mod_link_args, - sources: generated_headers + generated_backend_headers, + sources: [generated_backend_headers_stamp], dependencies: backend_mod_deps, ) diff --git a/src/fe_utils/meson.build b/src/fe_utils/meson.build index a18cbc939e4..5a9ddb73463 100644 --- a/src/fe_utils/meson.build +++ b/src/fe_utils/meson.build @@ -29,7 +29,7 @@ generated_sources += psqlscan fe_utils_sources += psqlscan fe_utils = static_library('libpgfeutils', - fe_utils_sources + generated_headers, + fe_utils_sources, c_pch: pch_postgres_fe_h, include_directories: [postgres_inc, libpq_inc], c_args: host_system == 'windows' ? ['-DFD_SETSIZE=1024'] : [], diff --git a/meson.build b/meson.build index a97854a947d..6a03bc461e6 100644 --- a/meson.build +++ b/meson.build @@ -3130,6 +3130,8 @@ gen_export_kwargs = { 'install': false, } +# command to create stamp files on all OSs +stamp_cmd = [python, '-c', 'import sys; open(sys.argv[1], "w")', '@OUTPUT0@'] ### @@ -3247,14 +3249,14 @@ subdir('src/port') frontend_common_code = declare_dependency( compile_args: ['-DFRONTEND'], include_directories: [postgres_inc], - sources: generated_headers, + sources: generated_headers_stamp, dependencies: [os_deps, zlib, zstd, lz4], ) backend_common_code = declare_dependency( compile_args: ['-DBUILDING_DLL'], include_directories: [postgres_inc], - sources: generated_headers, + sources: generated_headers_stamp, dependencies: [os_deps, zlib, zstd], ) @@ -3269,7 +3271,7 @@ shlib_code = declare_dependency( frontend_stlib_code = declare_dependency( include_directories: [postgres_inc], link_with: [common_static, pgport_static], - sources: generated_headers, + sources: generated_headers_stamp, dependencies: [os_deps, libintl], ) @@ -3277,7 +3279,7 @@ frontend_stlib_code = declare_dependency( frontend_shlib_code = declare_dependency( include_directories: [postgres_inc], link_with: [common_shlib, pgport_shlib], - sources: generated_headers, + sources: generated_headers_stamp, dependencies: [shlib_code, os_deps, libintl], ) @@ -3287,7 +3289,7 @@ frontend_shlib_code = declare_dependency( frontend_no_fe_utils_code = declare_dependency( include_directories: [postgres_inc], link_with: [common_static, pgport_static], - sources: generated_headers, + sources: generated_headers_stamp, dependencies: [os_deps, libintl], ) @@ -3314,7 +3316,7 @@ subdir('src/interfaces/libpq-oauth') frontend_code = declare_dependency( include_directories: [postgres_inc], link_with: [fe_utils, common_static, pgport_static], - sources: generated_headers, + sources: generated_headers_stamp, dependencies: [os_deps, libintl], ) @@ -3344,7 +3346,7 @@ backend_code = declare_dependency( include_directories: [postgres_inc], link_args: ldflags_be, link_with: [], - sources: generated_headers + generated_backend_headers, + sources: [generated_backend_headers_stamp], dependencies: os_deps + backend_both_deps + backend_deps, ) -- 2.49.0 [text/x-patch] v6-0002-meson-Add-postgresql-extension.pc-for-building-ex.patch (4.5K, 3-v6-0002-meson-Add-postgresql-extension.pc-for-building-ex.patch) download | inline diff: From ec7a71fa92befbadb016fe1b74575fddfea2f7bd Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Sat, 27 Aug 2022 09:52:03 -0700 Subject: [PATCH v6 2/7] meson: Add postgresql-extension.pc for building extension libraries This should work with several other buildsystems. TODO: Docs Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/backend/meson.build | 110 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 110 insertions(+) diff --git a/src/backend/meson.build b/src/backend/meson.build index 7fc649c3ebd..9d79d4d058c 100644 --- a/src/backend/meson.build +++ b/src/backend/meson.build @@ -188,6 +188,116 @@ pg_test_mod_args = pg_mod_args + { +############################################################### +# Define a .pc file that can be used to build server extensions +############################################################### + +pg_ext_vars = [] +pg_ext_vars_inst = [] +pg_ext_vars_uninst = [] + +pg_ext_cflags = pg_mod_c_args + cppflags +pg_ext_libs = [backend_mod_deps, thread_dep, ldflags, ldflags_mod] +pg_ext_subdirs = [''] + +# Compute directories to add include directories to the .pc files for. +# This is a bit more complicated due to port/win32 etc. +i = 0 +foreach incdir : postgres_inc_d + if fs.is_absolute(incdir) + # an absolute path from -Dextra_include_dirs + pg_ext_cflags += '-I@0@'.format(incdir) + continue + elif incdir.startswith('src/include') + subincdir = dir_include_pkg_rel / 'server' / incdir.split('src/include/').get(1, '') + else + subincdir = '' + endif + pg_ext_subdirs += subincdir + + # Add directories in source / build dir containing headers to cflags for the + # -uninstalled.pc. Older versions of pkg-config complain if a referenced + # variable is not defined, so we emit an empty one for the installed .pc + # file. + pg_ext_vars += [ + 'build_inc@0@=""'.format(i), + 'src_inc@0@=""'.format(i), + ] + pg_ext_vars_uninst += [ + 'build_inc@0@=-I${prefix}/@1@'.format(i, incdir), + 'src_inc@0@=-I${srcdir}/@1@'.format(i, incdir), + ] + pg_ext_cflags += [ + '${build_inc@0@}'.format(i), + '${src_inc@0@}'.format(i) + ] + + i += 1 +endforeach + + +# Extension modules should likely also use -fwrapv etc. But it it's a bit odd +# to expose it to a .pc file? +pg_ext_cflags_warn = pg_ext_cflags + cflags_warn +pg_ext_cflags += cflags + +# Directories for extensions to install into +# XXX: more needed +pg_ext_vars += 'pkglibdir=${prefix}/@0@'.format(dir_lib_pkg) +pg_ext_vars += 'dir_mod=${pkglibdir}' +pg_ext_vars += 'dir_data=${prefix}/@0@'.format(dir_data_extension) +pg_ext_vars += 'dir_include=${prefix}/@0@'.format(dir_include_extension) +pg_ext_vars += 'dir_doc=${prefix}/@0@'.format(dir_doc_extension) +pg_ext_vars += 'dir_bitcode=${prefix}/@0@'.format(dir_bitcode) +# referenced on some platforms, via mod_link_with_dir +pg_ext_vars += 'bindir=${prefix}/@0@'.format(dir_bin) + +# XXX: Define variables making it easy to define tests, too + +# Some platforms need linker flags to link with binary, they are the same +# between building with meson and .pc file, except that we have have to +# reference a variable to make it work for both normal and -uninstalled .pc +# files. +if mod_link_args_fmt.length() != 0 + assert(link_with_inst != '') + assert(link_with_uninst != '') + + pg_ext_vars_inst += 'mod_link_with=@0@'.format(link_with_inst) + pg_ext_vars_uninst += 'mod_link_with=@0@'.format(link_with_uninst) + + foreach el : mod_link_args_fmt + pg_ext_libs += el.format('${mod_link_with}') + endforeach +endif + +# main .pc to build extensions +pkgconfig.generate( + name: 'postgresql-extension', + description: 'PostgreSQL Extension Support', + url: pg_url, + + subdirs: pg_ext_subdirs, + libraries: pg_ext_libs, + extra_cflags: pg_ext_cflags, + + variables: pg_ext_vars + pg_ext_vars_inst, + uninstalled_variables: pg_ext_vars + pg_ext_vars_uninst, +) + +# a .pc depending on the above, but with all our warnings enabled +pkgconfig.generate( + name: 'postgresql-extension-warnings', + description: 'PostgreSQL Extension Support - Compiler Warnings', + requires: 'postgresql-extension', + url: pg_url, + extra_cflags: pg_ext_cflags_warn, + + variables: pg_ext_vars + pg_ext_vars_inst, + uninstalled_variables: pg_ext_vars + pg_ext_vars_uninst, +) + + + # Shared modules that, on some system, link against the server binary. Only # enter these after we defined the server build. -- 2.49.0 [text/x-patch] v6-0003-meson-Test-building-extensions-by-using-postgresq.patch (10.7K, 4-v6-0003-meson-Test-building-extensions-by-using-postgresq.patch) download | inline diff: From 912260657dc1a5ebc1534ac755f4f8065c5719aa Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Thu, 27 Feb 2025 17:45:31 +0300 Subject: [PATCH v6 3/7] meson: Test building extensions by using postgresql-extension.pc The 'test_meson_extensions' pyton wrapper is added to run these tests. It compiles and builds extensions at ${build}/testrun/meson_extensions/${extension_name} path. The tests for building amcheck, auth_delay and postgres_fdw extensions are added. These are also examples of how to build extensions by using postgresql-extension.pc. Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/test/modules/meson.build | 1 + .../modules/test_meson_extensions/meson.build | 3 + .../amcheck/meson.build | 28 ++++++++ .../auth_delay/meson.build | 17 +++++ .../test_pkg_config_extensions/meson.build | 24 +++++++ .../postgres_fdw/meson.build | 30 ++++++++ meson.build | 32 +++++++++ src/tools/ci/test_meson_extensions | 70 +++++++++++++++++++ 8 files changed, 205 insertions(+) create mode 100644 src/test/modules/test_meson_extensions/meson.build create mode 100644 src/test/modules/test_meson_extensions/test_pkg_config_extensions/amcheck/meson.build create mode 100644 src/test/modules/test_meson_extensions/test_pkg_config_extensions/auth_delay/meson.build create mode 100644 src/test/modules/test_meson_extensions/test_pkg_config_extensions/meson.build create mode 100644 src/test/modules/test_meson_extensions/test_pkg_config_extensions/postgres_fdw/meson.build create mode 100644 src/tools/ci/test_meson_extensions diff --git a/src/test/modules/meson.build b/src/test/modules/meson.build index 9de0057bd1d..94db733c4ad 100644 --- a/src/test/modules/meson.build +++ b/src/test/modules/meson.build @@ -26,6 +26,7 @@ subdir('test_ginpostinglist') subdir('test_integerset') subdir('test_json_parser') subdir('test_lfind') +subdir('test_meson_extensions') subdir('test_misc') subdir('test_oat_hooks') subdir('test_parser') diff --git a/src/test/modules/test_meson_extensions/meson.build b/src/test/modules/test_meson_extensions/meson.build new file mode 100644 index 00000000000..06cf5d555df --- /dev/null +++ b/src/test/modules/test_meson_extensions/meson.build @@ -0,0 +1,3 @@ +# Copyright (c) 2022-2025, PostgreSQL Global Development Group + +subdir('test_pkg_config_extensions') diff --git a/src/test/modules/test_meson_extensions/test_pkg_config_extensions/amcheck/meson.build b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/amcheck/meson.build new file mode 100644 index 00000000000..482a543eb86 --- /dev/null +++ b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/amcheck/meson.build @@ -0,0 +1,28 @@ +# Copyright (c) 2022-2025, PostgreSQL Global Development Group + +project('amcheck', 'c') + +amcheck_path = '../../../../../../contrib/amcheck/' + +amcheck_sources = files( + amcheck_path / 'verify_heapam.c', + amcheck_path / 'verify_nbtree.c', +) + +pg_ext = dependency('postgresql-extension-warnings') + +amcheck = shared_module('amcheck', + amcheck_sources, + dependencies: pg_ext, + install_dir: pg_ext.get_variable(pkgconfig: 'dir_mod'), +) + +install_data( + amcheck_path / 'amcheck.control', + amcheck_path / 'amcheck--1.0.sql', + amcheck_path / 'amcheck--1.0--1.1.sql', + amcheck_path / 'amcheck--1.1--1.2.sql', + amcheck_path / 'amcheck--1.2--1.3.sql', + amcheck_path / 'amcheck--1.3--1.4.sql', + install_dir: pg_ext.get_variable(pkgconfig: 'dir_data'), +) diff --git a/src/test/modules/test_meson_extensions/test_pkg_config_extensions/auth_delay/meson.build b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/auth_delay/meson.build new file mode 100644 index 00000000000..98ad24cc183 --- /dev/null +++ b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/auth_delay/meson.build @@ -0,0 +1,17 @@ +# Copyright (c) 2022-2025, PostgreSQL Global Development Group + +project('auth_delay', 'c') + +auth_delay_path = '../../../../../../contrib/auth_delay/' + +auth_delay_sources = files( + auth_delay_path / 'auth_delay.c', +) + +pg_ext = dependency('postgresql-extension-warnings') + +auth_delay = shared_module('auth_delay', + auth_delay_sources, + dependencies: pg_ext, + install_dir: pg_ext.get_variable(pkgconfig: 'dir_mod'), +) diff --git a/src/test/modules/test_meson_extensions/test_pkg_config_extensions/meson.build b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/meson.build new file mode 100644 index 00000000000..dae94a384a1 --- /dev/null +++ b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/meson.build @@ -0,0 +1,24 @@ +# Copyright (c) 2022-2025, PostgreSQL Global Development Group + +# pkgconfig is not available on Windows, so skip it. +if host_machine.system() == 'windows' + subdir_done() +endif + +meson_extension_tests += { + 'name': 'amcheck', + 'kind': 'pkg_config', + 'sd': meson.current_source_dir() / 'amcheck', +} + +meson_extension_tests += { + 'name': 'auth_delay', + 'kind': 'pkg_config', + 'sd': meson.current_source_dir() / 'auth_delay', +} + +meson_extension_tests += { + 'name': 'postgres_fdw', + 'kind': 'pkg_config', + 'sd': meson.current_source_dir() / 'postgres_fdw', +} diff --git a/src/test/modules/test_meson_extensions/test_pkg_config_extensions/postgres_fdw/meson.build b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/postgres_fdw/meson.build new file mode 100644 index 00000000000..6d561e3789b --- /dev/null +++ b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/postgres_fdw/meson.build @@ -0,0 +1,30 @@ +# Copyright (c) 2022-2025, PostgreSQL Global Development Group + +project('auth_delay', 'c') + +postgres_fdw_path = '../../../../../../contrib/postgres_fdw/' + +postgres_fdw_sources = files( + postgres_fdw_path / 'connection.c', + postgres_fdw_path / 'deparse.c', + postgres_fdw_path / 'option.c', + postgres_fdw_path / 'postgres_fdw.c', + postgres_fdw_path / 'shippable.c', +) + +pg_ext = dependency('postgresql-extension-warnings') +libpq = dependency('libpq') + +postgres_fdw = shared_module('postgres_fdw', + postgres_fdw_sources, + dependencies: [pg_ext, libpq], + install_dir: pg_ext.get_variable(pkgconfig: 'dir_mod'), +) + +install_data( + postgres_fdw_path / 'postgres_fdw.control', + postgres_fdw_path / 'postgres_fdw--1.0.sql', + postgres_fdw_path / 'postgres_fdw--1.0--1.1.sql', + postgres_fdw_path / 'postgres_fdw--1.1--1.2.sql', + install_dir: pg_ext.get_variable(pkgconfig: 'dir_data'), +) diff --git a/meson.build b/meson.build index 6a03bc461e6..455f4715bf3 100644 --- a/meson.build +++ b/meson.build @@ -3048,6 +3048,7 @@ nls_targets = [] # Define the tests to distribute them to the correct test styles later test_deps = [] tests = [] +meson_extension_tests = [] # Default options for targets @@ -3603,6 +3604,37 @@ sys.exit(sp.returncode) suite: ['setup']) +# it seems freebsd doesn't use libdir for pkgconfig path +if host_system == 'freebsd' + pkgconf_installdir = dir_prefix / 'libdata' / 'pkgconfig' +else + pkgconf_installdir = dir_prefix / dir_lib / 'pkgconfig' +endif +test_pkg_conf_file = files('src/tools/ci/test_meson_extensions') + +foreach test : meson_extension_tests + if test['kind'] not in ['pkg_config'] + error('unknown kind @0@ of test in @1@'.format(test['kind'], test['sd'])) + endif + + test_group = 'meson_@0@_extensions'.format(test['kind']) + + test(test_group / test['name'], + test_pkg_conf_file, + args: [ + '--meson', meson_bin.path(), + '--meson_args', meson_args, + '--test_dir', test['sd'], + '--test_out_dir', test_result_dir / 'meson_extensions' / test['name'], + '--builddir', meson.build_root(), + '--pkg_conf_path', get_option('pkg_config_path'), + '--', + cc.cmd_array(), + ], + suite: test_group, + ) + +endforeach ############################################################### # Test Generation diff --git a/src/tools/ci/test_meson_extensions b/src/tools/ci/test_meson_extensions new file mode 100644 index 00000000000..50358121f49 --- /dev/null +++ b/src/tools/ci/test_meson_extensions @@ -0,0 +1,70 @@ +#!/usr/bin/env python3 + +import argparse +import os +import shutil +import subprocess + +parser = argparse.ArgumentParser() + +parser.add_argument('--meson', help='path to meson binary', + type=str, required=True) +parser.add_argument('--meson_args', help='args of meson binary', + type=str, nargs='*', required=False) +parser.add_argument('--test_dir', help='test source directory', + type=str, required=True) +parser.add_argument('--test_out_dir', help='test output directory', + type=str, required=True) +parser.add_argument('--builddir', help='meson build directory', + type=str, required=True) +parser.add_argument('--pkg_conf_path', + help='PKG_CONF_PATH from surrounding meson build', + type=str, nargs='?', const='', required=False) +parser.add_argument('c_args', help='c_args from surrounding meson build', + nargs='*') + +args = parser.parse_args() + +meson_bin = args.meson +meson_args = ' '.join(args.meson_args) +test_source_dir = args.test_dir +test_out_dir = args.test_out_dir +build_dir = args.builddir +pkg_conf_path = args.pkg_conf_path +c_args = ' '.join(args.c_args) + +exit_code = 0 + +def remove_duplicates(duplicate_str): + words = duplicate_str.split() + return ' '.join(sorted(set(words), key=words.index)) + + +def run_tests(pkg_conf_path_local, message): + print('\n{}\n{}\n'.format('#' * 60, message), flush=True) + + env = {**os.environ, } + env['PKG_CONFIG_PATH'] = '{}:{}:{}'.format( + pkg_conf_path_local, pkg_conf_path, env.get('PKG_CONFIG_PATH', ''), + ).strip(': ') + env['CC'] = '{} {}'.format( + c_args, env.get('CC', ''), + ) + env['CC'] = remove_duplicates(env['CC']) + + # Clear the build directory beforehand. + if os.path.exists(test_out_dir): + shutil.rmtree(test_out_dir) + + if meson_args: + meson_setup_command = [meson_bin, meson_args, 'setup', test_out_dir] + else: + meson_setup_command = [meson_bin, 'setup', test_out_dir] + + meson_compile_command = ['meson', 'compile', '-C', test_out_dir, '-v'] + + subprocess.run(meson_setup_command, env=env, cwd=test_source_dir, check=True) + subprocess.run(meson_compile_command, cwd=test_source_dir, check=True) + +run_tests(os.path.join(build_dir, 'meson-uninstalled'), + message='Testing postgresql-extension-warnings-uninstalled') -- 2.49.0 [text/x-patch] v6-0004-meson-WIP-Add-docs-for-postgresql-extension.pc.patch (12.3K, 5-v6-0004-meson-WIP-Add-docs-for-postgresql-extension.pc.patch) download | inline diff: From 7d0f94729f20300e55375c8a05b52aeeaf00f1f0 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Thu, 6 Mar 2025 17:46:57 +0300 Subject: [PATCH v6 4/7] meson: [WIP] Add docs for postgresql-extension.pc Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- doc/src/sgml/acronyms.sgml | 2 +- doc/src/sgml/extend.sgml | 101 +++++++++++++++++++++++-------------- doc/src/sgml/jit.sgml | 2 +- 3 files changed, 66 insertions(+), 39 deletions(-) diff --git a/doc/src/sgml/acronyms.sgml b/doc/src/sgml/acronyms.sgml index 2f906e9f018..57f49c06a19 100644 --- a/doc/src/sgml/acronyms.sgml +++ b/doc/src/sgml/acronyms.sgml @@ -579,7 +579,7 @@ <term><acronym>PGXS</acronym></term> <listitem> <para> - <link linkend="extend-pgxs"><productname>PostgreSQL</productname> Extension System</link> + <link linkend="extend-postgres-pgxs"><productname>PostgreSQL</productname> Extension System</link> </para> </listitem> </varlistentry> diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml index 63c5ec6d1eb..6d9fe5e96ad 100644 --- a/doc/src/sgml/extend.sgml +++ b/doc/src/sgml/extend.sgml @@ -1426,7 +1426,7 @@ include $(PGXS) </programlisting> This makefile relies on <acronym>PGXS</acronym>, which is described - in <xref linkend="extend-pgxs"/>. The command <literal>make install</literal> + in <xref linkend="extend-postgres-pgxs"/>. The command <literal>make install</literal> will install the control and script files into the correct directory as reported by <application>pg_config</application>. </para> @@ -1439,21 +1439,26 @@ include $(PGXS) </sect2> </sect1> - <sect1 id="extend-pgxs"> + <sect1 id="extend-postgres"> <title>Extension Building Infrastructure</title> - <indexterm zone="extend-pgxs"> - <primary>pgxs</primary> - </indexterm> - <para> If you are thinking about distributing your <productname>PostgreSQL</productname> extension modules, setting up a portable build system for them can be fairly difficult. Therefore the <productname>PostgreSQL</productname> installation provides a build - infrastructure for extensions, called <acronym>PGXS</acronym>, so - that simple extension modules can be built simply against an - already installed server. <acronym>PGXS</acronym> is mainly intended + infrastructure for extensions, called <literal>PGXS</literal> + (<xref linkend="extend-postgres-pgxs"/>) and + its meson counterpart <literal>postgresql-extension.pc</literal> + (<xref linkend="extend-postgres-meson"/>). + </para> + + </sect1> + + <sect1 id="extend-postgres-pgxs"> + <title>PGXS</title> + + <para> <acronym>PGXS</acronym> is mainly intended for extensions that include C code, although it can be used for pure-SQL extensions too. Note that <acronym>PGXS</acronym> is not intended to be a universal build system framework that can be used @@ -1493,7 +1498,7 @@ include $(PGXS) Set one of these three variables to specify what is built: <variablelist> - <varlistentry id="extend-pgxs-modules"> + <varlistentry id="extend-postgres-pgxs-modules"> <term><varname>MODULES</varname></term> <listitem> <para> @@ -1503,7 +1508,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-module-big"> + <varlistentry id="extend-postgres-pgxs-module-big"> <term><varname>MODULE_big</varname></term> <listitem> <para> @@ -1513,7 +1518,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-program"> + <varlistentry id="extend-postgres-pgxs-program"> <term><varname>PROGRAM</varname></term> <listitem> <para> @@ -1527,7 +1532,7 @@ include $(PGXS) The following variables can also be set: <variablelist> - <varlistentry id="extend-pgxs-extension"> + <varlistentry id="extend-postgres-pgxs-extension"> <term><varname>EXTENSION</varname></term> <listitem> <para> @@ -1539,7 +1544,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-moduledir"> + <varlistentry id="extend-postgres-pgxs-moduledir"> <term><varname>MODULEDIR</varname></term> <listitem> <para> @@ -1552,7 +1557,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-data"> + <varlistentry id="extend-postgres-pgxs-data"> <term><varname>DATA</varname></term> <listitem> <para> @@ -1561,7 +1566,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-data-built"> + <varlistentry id="extend-postgres-pgxs-data-built"> <term><varname>DATA_built</varname></term> <listitem> <para> @@ -1572,7 +1577,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-data-tsearch"> + <varlistentry id="extend-postgres-pgxs-data-tsearch"> <term><varname>DATA_TSEARCH</varname></term> <listitem> <para> @@ -1582,7 +1587,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-docs"> + <varlistentry id="extend-postgres-pgxs-docs"> <term><varname>DOCS</varname></term> <listitem> <para> @@ -1592,7 +1597,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-headers"> + <varlistentry id="extend-postgres-pgxs-headers"> <term><varname>HEADERS</varname></term> <term><varname>HEADERS_built</varname></term> <listitem> @@ -1608,7 +1613,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-headers-module"> + <varlistentry id="extend-postgres-pgxs-headers-module"> <term><varname>HEADERS_$MODULE</varname></term> <term><varname>HEADERS_built_$MODULE</varname></term> <listitem> @@ -1634,7 +1639,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-scripts"> + <varlistentry id="extend-postgres-pgxs-scripts"> <term><varname>SCRIPTS</varname></term> <listitem> <para> @@ -1644,7 +1649,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-scripts-built"> + <varlistentry id="extend-postgres-pgxs-scripts-built"> <term><varname>SCRIPTS_built</varname></term> <listitem> <para> @@ -1655,7 +1660,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-regress"> + <varlistentry id="extend-postgres-pgxs-regress"> <term><varname>REGRESS</varname></term> <listitem> <para> @@ -1664,7 +1669,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-regress-opts"> + <varlistentry id="extend-postgres-pgxs-regress-opts"> <term><varname>REGRESS_OPTS</varname></term> <listitem> <para> @@ -1673,7 +1678,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-isolation"> + <varlistentry id="extend-postgres-pgxs-isolation"> <term><varname>ISOLATION</varname></term> <listitem> <para> @@ -1682,7 +1687,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-isolation-opts"> + <varlistentry id="extend-postgres-pgxs-isolation-opts"> <term><varname>ISOLATION_OPTS</varname></term> <listitem> <para> @@ -1692,7 +1697,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-tap-tests"> + <varlistentry id="extend-postgres-pgxs-tap-tests"> <term><varname>TAP_TESTS</varname></term> <listitem> <para> @@ -1701,7 +1706,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-no-install"> + <varlistentry id="extend-postgres-pgxs-no-install"> <term><varname>NO_INSTALL</varname></term> <listitem> <para> @@ -1711,7 +1716,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-no-installcheck"> + <varlistentry id="extend-postgres-pgxs-no-installcheck"> <term><varname>NO_INSTALLCHECK</varname></term> <listitem> <para> @@ -1720,7 +1725,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-extra-clean"> + <varlistentry id="extend-postgres-pgxs-extra-clean"> <term><varname>EXTRA_CLEAN</varname></term> <listitem> <para> @@ -1729,7 +1734,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-cppflags"> + <varlistentry id="extend-postgres-pgxs-pg-cppflags"> <term><varname>PG_CPPFLAGS</varname></term> <listitem> <para> @@ -1738,7 +1743,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-cflags"> + <varlistentry id="extend-postgres-pgxs-pg-cflags"> <term><varname>PG_CFLAGS</varname></term> <listitem> <para> @@ -1747,7 +1752,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-cxxflags"> + <varlistentry id="extend-postgres-pgxs-pg-cxxflags"> <term><varname>PG_CXXFLAGS</varname></term> <listitem> <para> @@ -1756,7 +1761,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-ldflags"> + <varlistentry id="extend-postgres-pgxs-pg-ldflags"> <term><varname>PG_LDFLAGS</varname></term> <listitem> <para> @@ -1765,7 +1770,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-libs"> + <varlistentry id="extend-postgres-pgxs-pg-libs"> <term><varname>PG_LIBS</varname></term> <listitem> <para> @@ -1774,7 +1779,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-shlib-link"> + <varlistentry id="extend-postgres-pgxs-shlib-link"> <term><varname>SHLIB_LINK</varname></term> <listitem> <para> @@ -1783,7 +1788,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-config"> + <varlistentry id="extend-postgres-pgxs-pg-config"> <term><varname>PG_CONFIG</varname></term> <listitem> <para> @@ -1929,4 +1934,26 @@ make VPATH=/path/to/extension/source/tree install </tip> </sect1> + <sect1 id="extend-postgres-meson"> + <title>postgresql-extension.pc</title> + + <para> + When Postgres is built by using meson, it generates + <literal>postgresql-extension.pc</literal> pkg-config file. Extension + libraries can use this file like <literal>PGXS</literal> + (<xref linkend="extend-postgres-pgxs"/>). + + To use the <literal>postgresql-extension.pc</literal> infrastructure for + your extension, you must write a simple meson.build file. In the + meson.build file, you need to include the + <literal>postgresql-extension.pc</literal> pkg-config file. Here is an + example that builds an extension module named isbn_issn, consisting of a + shared library containing some C code, an extension control file, an SQL + script, an include file (only needed if other modules might need to access + the extension functions without going via SQL), and a documentation text + file: + </para> + + </sect1> + </chapter> diff --git a/doc/src/sgml/jit.sgml b/doc/src/sgml/jit.sgml index 44e18bf1a6f..81a4644a97d 100644 --- a/doc/src/sgml/jit.sgml +++ b/doc/src/sgml/jit.sgml @@ -223,7 +223,7 @@ SET of types <literal>C</literal> and <literal>internal</literal>, as well as operators based on such functions. To do so for functions in extensions, the definitions of those functions need to be made available. - When using <link linkend="extend-pgxs">PGXS</link> to build an extension + When using <link linkend="extend-postgres-pgxs">PGXS</link> to build an extension against a server that has been compiled with LLVM JIT support, the relevant files will be built and installed automatically. </para> -- 2.49.0 [text/x-patch] v6-0005-meson-Add-architecture-for-LLVM-bitcode-emission.patch (9.0K, 6-v6-0005-meson-Add-architecture-for-LLVM-bitcode-emission.patch) download | inline diff: From 5e0b20943a0afaf6bccc8e022b82e25ebb906d12 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Fri, 7 Mar 2025 12:10:58 +0300 Subject: [PATCH v6 5/7] meson: Add architecture for LLVM bitcode emission This commit adds suport for bitcode emission for both normal and generated source files (processed by bison, flex, etc). These bitcode files are installed into $pkglibdir/bitcode/ directory if the LLVM is found. New variable `bitcode_modules` is introduced to generate bitcode files. All required information is gathered in this variable. Then, this variable is processed by the main meson LLVM bitcode emission scripts: src/backend/jit/llvm/bitcode/meson.build -> src/tools/irlink. An example of a possible structure of bitcode_modules is: ``` bitcode_modules = [ { 'name': '...', 'target': ..., 'srcfiles': [ '...', '...', ], 'additional_flags': [ '-I...', '-I...', ], 'gen_srcfiles': [ { 'srcfiles': [ <custom_target for ...>, <custom_target for ...>, ], 'additional_flags': [ '-I...', '-I...', ] } ] } ] ``` Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Author: Diego Fronza <[email protected]> Reviewed-by: Diego Fronza <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/backend/jit/llvm/bitcode/meson.build | 69 ++++++++++++++++++++++++ src/backend/jit/llvm/meson.build | 31 +++++++---- src/backend/meson.build | 6 +++ meson.build | 21 ++++++++ src/tools/irlink | 25 +++++++++ 5 files changed, 141 insertions(+), 11 deletions(-) create mode 100644 src/backend/jit/llvm/bitcode/meson.build create mode 100644 src/tools/irlink diff --git a/src/backend/jit/llvm/bitcode/meson.build b/src/backend/jit/llvm/bitcode/meson.build new file mode 100644 index 00000000000..546e4d8b898 --- /dev/null +++ b/src/backend/jit/llvm/bitcode/meson.build @@ -0,0 +1,69 @@ +# Copyright (c) 2022-2024, PostgreSQL Global Development Group +# +# emit LLVM bitcode for JIT inlining + +assert(llvm.found()) + +foreach bitcode_module : bitcode_modules + bitcode_targets = [] + bitcode_obj = bitcode_module['target'] + bitcode_cflags_local = bitcode_cflags + bitcode_module.get('additional_flags', []) + bitcode_name = bitcode_module.get('name', bitcode_obj.name()) + + foreach srcfile : bitcode_module['srcfiles'] + if meson.version().version_compare('>=0.59') + srcfilename = fs.parent(srcfile) / fs.name(srcfile) + else + srcfilename = '@0@'.format(srcfile) + endif + + targetname = '@0@_@[email protected]'.format( + bitcode_name, + srcfilename.underscorify(), + ) + bitcode_targets += custom_target( + targetname, + depends: [bitcode_obj], + input: [srcfile], + output: targetname, + command: [llvm_irgen_command, bitcode_cflags_local], + install: true, + install_dir: dir_bitcode, + ) + endforeach + + # Process generated sources, which may include custom compilation flags. + foreach gen_sources: bitcode_module.get('gen_sources', []) + bitcode_cflags_gen_local = bitcode_cflags_local + gen_sources.get('additional_flags', []) + + foreach srcfile: gen_sources['srcfiles'] + # Generated sources are stored in some folder under meson.build_root()/**, + # remove the build prefix from the string. + srcfilename = srcfile.full_path().split(meson.build_root() + '/')[1] + + targetname = '@0@_@[email protected]'.format( + bitcode_name, + srcfilename.underscorify(), + ) + bitcode_targets += custom_target( + targetname, + depends: [bitcode_obj], + input: [srcfile], + output: targetname, + command: [llvm_irgen_command, bitcode_cflags_gen_local], + install: true, + install_dir: dir_bitcode, + ) + endforeach + endforeach + + index_name = '@[email protected]'.format(bitcode_name) + bitcode_index = custom_target('@0@'.format(bitcode_name), + output: index_name, + input: bitcode_targets, + command: [irlink, '--lto', llvm_lto, '--outdir', '@OUTDIR@', '--index', index_name, '@INPUT@'], + install: true, + install_dir: dir_bitcode, + ) + backend_targets += bitcode_index +endforeach diff --git a/src/backend/jit/llvm/meson.build b/src/backend/jit/llvm/meson.build index 805fbd69006..2dd922e573b 100644 --- a/src/backend/jit/llvm/meson.build +++ b/src/backend/jit/llvm/meson.build @@ -42,21 +42,22 @@ backend_targets += llvmjit # Define a few bits and pieces used here and elsewhere to generate bitcode -llvm_irgen_args = [ - '-c', '-o', '@OUTPUT@', '@INPUT@', +llvm_irgen_command = [] +if ccache.found() + llvm_irgen_command += ccache +endif + +llvm_irgen_command += [ + clang, + '-c', '-o', '@OUTPUT0@', '@INPUT0@', '-flto=thin', '-emit-llvm', - '-MD', '-MQ', '@OUTPUT@', '-MF', '@DEPFILE@', '-O2', '-Wno-ignored-attributes', '-Wno-empty-body', + '-Wno-unknown-warning-option', + '-Wno-compound-token-split-by-macro', ] - -if ccache.found() - llvm_irgen_command = ccache - llvm_irgen_args = [clang.full_path()] + llvm_irgen_args -else - llvm_irgen_command = clang -endif +llvm_irgen_dep_args = ['-MD', '-MQ', '@OUTPUT0@', '-MF', '@DEPFILE@'] # XXX: Need to determine proper version of the function cflags for clang @@ -73,7 +74,7 @@ bitcode_cflags += '-I@SOURCE_ROOT@/src/include' # Note this is intentionally not installed to bitcodedir, as it's not for # inlining llvmjit_types = custom_target('llvmjit_types.bc', - command: [llvm_irgen_command] + llvm_irgen_args + bitcode_cflags, + command: llvm_irgen_command + llvm_irgen_dep_args + bitcode_cflags, input: 'llvmjit_types.c', output: 'llvmjit_types.bc', depends: [postgres], @@ -82,3 +83,11 @@ llvmjit_types = custom_target('llvmjit_types.bc', depfile: '@[email protected]', ) backend_targets += llvmjit_types + +# Figure out -I's needed to build all postgres code, including all its +# dependencies +pkg_config = find_program(['pkg-config', 'pkgconf'], required: true) +r = run_command(pkg_config, + ['--cflags-only-I', meson.build_root() / 'meson-uninstalled/postgresql-extension-uninstalled.pc'], + check: true) +bitcode_cflags += r.stdout().split() diff --git a/src/backend/meson.build b/src/backend/meson.build index 9d79d4d058c..5fb33660d3d 100644 --- a/src/backend/meson.build +++ b/src/backend/meson.build @@ -140,6 +140,12 @@ postgres = executable('postgres', backend_targets += postgres +bitcode_modules += { + 'name': 'postgres', + 'target': postgres_lib, + 'srcfiles': backend_sources, +} + pg_mod_c_args = cflags_mod pg_mod_cpp_args = cxxflags_mod pg_mod_link_args = ldflags_sl + ldflags_mod diff --git a/meson.build b/meson.build index 455f4715bf3..c65ddd7ebe2 100644 --- a/meson.build +++ b/meson.build @@ -820,6 +820,8 @@ if add_languages('cpp', required: llvmopt, native: false) # Some distros put LLVM and clang in different paths, so fallback to # find via PATH, too. clang = find_program(llvm_binpath / 'clang', 'clang', required: true) + llvm_lto = find_program(llvm_binpath / 'llvm-lto', required: true) + irlink = find_program('src/tools/irlink', native: true) endif elif llvmopt.auto() message('llvm requires a C++ compiler') @@ -3050,6 +3052,11 @@ test_deps = [] tests = [] meson_extension_tests = [] +# List of object files + source files to generated LLVM IR for inlining. +# Each element is a hash of: +# {'target': target, 'srcfiles': ..., 'additional_flags': ...}. +bitcode_modules = [] + # Default options for targets @@ -3372,6 +3379,11 @@ subdir('src/interfaces/ecpg/test') subdir('doc/src/sgml') +# generate bitcode for JIT inlining after giving contrib modules etc a chance +# to add themselves to bitcode_modules[] +subdir('src/backend/jit/llvm/bitcode', if_found: llvm) + + generated_sources_ac += {'': ['GNUmakefile']} # After processing src/test, add test_install_libs to the testprep_targets @@ -3994,6 +4006,15 @@ summary( section: 'Programs', ) +if llvm.found() + summary( + { + 'clang': clang, + }, + section: 'Programs', + ) +endif + summary( { 'bonjour': bonjour, diff --git a/src/tools/irlink b/src/tools/irlink new file mode 100644 index 00000000000..793c0abf91a --- /dev/null +++ b/src/tools/irlink @@ -0,0 +1,25 @@ +#!/usr/bin/env python3 + +import argparse +import os +import shutil +import subprocess +import sys + +parser = argparse.ArgumentParser( + description='generate PostgreSQL JIT IR module') + +parser.add_argument('--index', type=str, required=True) +parser.add_argument('--lto', type=str, required=True) +parser.add_argument('--outdir', type=str, required=True) +parser.add_argument('INPUT', type=str, nargs='+') + +args = parser.parse_args() + +file_names = [os.path.basename(f) for f in args.INPUT] +command = [args.lto, + '-thinlto', '-thinlto-action=thinlink', + '-o', args.index] + file_names +res = subprocess.run(command, cwd=args.outdir) + +exit(res.returncode) -- 2.49.0 [text/x-patch] v6-0006-meson-Add-LLVM-bitcode-emissions-for-contrib-libr.patch (22.7K, 7-v6-0006-meson-Add-LLVM-bitcode-emissions-for-contrib-libr.patch) download | inline diff: From 2a8222598d280944e3239402613cc38ca6ee9bf4 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Wed, 12 Mar 2025 10:49:18 +0300 Subject: [PATCH v6 6/7] meson: Add LLVM bitcode emissions for contrib libraries The libraries which the bitcode files will be generated in are selected manually. Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Author: Diego Fronza <[email protected]> Reviewed-by: Diego Fronza <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/interfaces/libpq/meson.build | 3 +++ contrib/bloom/meson.build | 5 +++++ contrib/bool_plperl/meson.build | 9 +++++++++ contrib/btree_gin/meson.build | 5 +++++ contrib/btree_gist/meson.build | 5 +++++ contrib/citext/meson.build | 5 +++++ contrib/cube/meson.build | 13 +++++++++++++ contrib/dict_int/meson.build | 5 +++++ contrib/dict_xsyn/meson.build | 5 +++++ contrib/earthdistance/meson.build | 6 ++++++ contrib/fuzzystrmatch/meson.build | 9 +++++++++ contrib/hstore/meson.build | 7 +++++++ contrib/hstore_plperl/meson.build | 10 ++++++++++ contrib/hstore_plpython/meson.build | 12 ++++++++++++ contrib/intarray/meson.build | 5 +++++ contrib/isn/meson.build | 5 +++++ contrib/jsonb_plperl/meson.build | 8 ++++++++ contrib/jsonb_plpython/meson.build | 10 ++++++++++ contrib/lo/meson.build | 5 +++++ contrib/ltree/meson.build | 10 ++++++++++ contrib/ltree_plpython/meson.build | 11 +++++++++++ contrib/pg_buffercache/meson.build | 5 +++++ contrib/pg_freespacemap/meson.build | 5 +++++ contrib/pg_logicalinspect/meson.build | 5 +++++ contrib/pg_surgery/meson.build | 4 ++++ contrib/pg_trgm/meson.build | 5 +++++ contrib/pgcrypto/meson.build | 4 ++++ contrib/seg/meson.build | 12 ++++++++++++ contrib/spi/meson.build | 20 ++++++++++++++++++++ contrib/sslinfo/meson.build | 5 +++++ contrib/tablefunc/meson.build | 5 +++++ contrib/tcn/meson.build | 5 +++++ contrib/tsm_system_rows/meson.build | 5 +++++ contrib/tsm_system_time/meson.build | 5 +++++ contrib/unaccent/meson.build | 5 +++++ contrib/uuid-ossp/meson.build | 5 +++++ contrib/xml2/meson.build | 5 +++++ src/pl/plperl/meson.build | 1 + src/pl/plpython/meson.build | 1 + meson.build | 1 + 40 files changed, 256 insertions(+) diff --git a/src/interfaces/libpq/meson.build b/src/interfaces/libpq/meson.build index a74e885b169..c3ceab1dd70 100644 --- a/src/interfaces/libpq/meson.build +++ b/src/interfaces/libpq/meson.build @@ -49,6 +49,9 @@ libpq_c_args = ['-DSO_MAJOR_VERSION=5'] # The OAuth implementation differs depending on the type of library being built. libpq_so_c_args = ['-DUSE_DYNAMIC_OAUTH'] +# libpq-fe.h is needed to generate bitcode for some extensions +libpq_dir = meson.current_source_dir() + # Not using both_libraries() here as # 1) resource files should only be in the shared library # 2) we want the .pc file to include a dependency to {pgport,common}_static for diff --git a/contrib/bloom/meson.build b/contrib/bloom/meson.build index 695712a455e..1090c281532 100644 --- a/contrib/bloom/meson.build +++ b/contrib/bloom/meson.build @@ -28,6 +28,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': bloom, + 'srcfiles': bloom_sources, +} + tests += { 'name': 'bloom', 'sd': meson.current_source_dir(), diff --git a/contrib/bool_plperl/meson.build b/contrib/bool_plperl/meson.build index f489d99044c..1758adb7489 100644 --- a/contrib/bool_plperl/meson.build +++ b/contrib/bool_plperl/meson.build @@ -37,6 +37,15 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': bool_plperl, + 'srcfiles': bool_plperl_sources, + 'additional_flags': [ + '-I@0@'.format(plperl_dir), + perl_ccflags + ] +} + tests += { 'name': 'bool_plperl', 'sd': meson.current_source_dir(), diff --git a/contrib/btree_gin/meson.build b/contrib/btree_gin/meson.build index ece0a716973..e92e52822b0 100644 --- a/contrib/btree_gin/meson.build +++ b/contrib/btree_gin/meson.build @@ -26,6 +26,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': btree_gin, + 'srcfiles': btree_gin_sources, +} + tests += { 'name': 'btree_gin', 'sd': meson.current_source_dir(), diff --git a/contrib/btree_gist/meson.build b/contrib/btree_gist/meson.build index 89932dd3844..6a276ec02f4 100644 --- a/contrib/btree_gist/meson.build +++ b/contrib/btree_gist/meson.build @@ -55,6 +55,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': btree_gist, + 'srcfiles': btree_gist_sources, +} + tests += { 'name': 'btree_gist', 'sd': meson.current_source_dir(), diff --git a/contrib/citext/meson.build b/contrib/citext/meson.build index 7fff34f2368..fa07ff72be4 100644 --- a/contrib/citext/meson.build +++ b/contrib/citext/meson.build @@ -30,6 +30,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': citext, + 'srcfiles': citext_sources, +} + tests += { 'name': 'citext', 'sd': meson.current_source_dir(), diff --git a/contrib/cube/meson.build b/contrib/cube/meson.build index fd3c057f469..a7649a74a1f 100644 --- a/contrib/cube/meson.build +++ b/contrib/cube/meson.build @@ -3,6 +3,7 @@ cube_sources = files( 'cube.c', ) +bc_cube_sources = cube_sources cube_scan = custom_target('cubescan', input: 'cubescan.l', @@ -11,6 +12,7 @@ cube_scan = custom_target('cubescan', ) generated_sources += cube_scan cube_sources += cube_scan +bc_cube_gen_sources = [{'srcfiles': [cube_scan]}] cube_parse = custom_target('cubeparse', input: 'cubeparse.y', @@ -18,6 +20,7 @@ cube_parse = custom_target('cubeparse', ) generated_sources += cube_parse.to_list() cube_sources += cube_parse +bc_cube_gen_sources += {'srcfiles': [cube_parse[0]]} if host_system == 'windows' cube_sources += rc_lib_gen.process(win32ver_rc, extra_args: [ @@ -48,6 +51,16 @@ install_headers( install_dir: dir_include_extension / 'cube', ) +cube_dir = meson.current_source_dir() +bitcode_modules += { + 'target': cube, + 'srcfiles': bc_cube_sources, + 'gen_sources': bc_cube_gen_sources, + 'additional_flags': [ + '-I@0@'.format(cube_dir), + ] +} + tests += { 'name': 'cube', 'sd': meson.current_source_dir(), diff --git a/contrib/dict_int/meson.build b/contrib/dict_int/meson.build index ab41588547b..894c8a9c94d 100644 --- a/contrib/dict_int/meson.build +++ b/contrib/dict_int/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': dict_int, + 'srcfiles': dict_int_sources, +} + tests += { 'name': 'dict_int', 'sd': meson.current_source_dir(), diff --git a/contrib/dict_xsyn/meson.build b/contrib/dict_xsyn/meson.build index 93f41b1f963..d4bd4a6ef5a 100644 --- a/contrib/dict_xsyn/meson.build +++ b/contrib/dict_xsyn/meson.build @@ -29,6 +29,11 @@ install_data( } ) +bitcode_modules += { + 'target': dict_xsyn, + 'srcfiles': dict_xsyn_sources, +} + tests += { 'name': 'dict_xsyn', 'sd': meson.current_source_dir(), diff --git a/contrib/earthdistance/meson.build b/contrib/earthdistance/meson.build index a5f8d1a3ffa..c101359b118 100644 --- a/contrib/earthdistance/meson.build +++ b/contrib/earthdistance/meson.build @@ -24,6 +24,12 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': earthdistance, + 'srcfiles': earthdistance_sources, +} + + tests += { 'name': 'earthdistance', 'sd': meson.current_source_dir(), diff --git a/contrib/fuzzystrmatch/meson.build b/contrib/fuzzystrmatch/meson.build index f52daa4ea1c..fc5e88d0c92 100644 --- a/contrib/fuzzystrmatch/meson.build +++ b/contrib/fuzzystrmatch/meson.build @@ -5,6 +5,7 @@ fuzzystrmatch_sources = files( 'dmetaphone.c', 'fuzzystrmatch.c', ) +bc_fuzzystrmatch_sources = fuzzystrmatch_sources daitch_mokotoff_h = custom_target('daitch_mokotoff', input: 'daitch_mokotoff_header.pl', @@ -35,6 +36,14 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': fuzzystrmatch, + 'srcfiles': bc_fuzzystrmatch_sources, + 'additional_flags': [ + '-I@0@'.format(meson.current_build_dir()) + ] +} + tests += { 'name': 'fuzzystrmatch', 'sd': meson.current_source_dir(), diff --git a/contrib/hstore/meson.build b/contrib/hstore/meson.build index 39622d1024d..344a620273f 100644 --- a/contrib/hstore/meson.build +++ b/contrib/hstore/meson.build @@ -43,6 +43,13 @@ install_headers( install_dir: dir_include_extension / 'hstore', ) +# some libraries include "hstore/hstore.h" instead of "hstore.h" +hstore_dir_up = join_paths(meson.current_source_dir(), '..') +bitcode_modules += { + 'target': hstore, + 'srcfiles': hstore_sources, +} + tests += { 'name': 'hstore', 'sd': meson.current_source_dir(), diff --git a/contrib/hstore_plperl/meson.build b/contrib/hstore_plperl/meson.build index 60b8ad23569..b689b55e1c9 100644 --- a/contrib/hstore_plperl/meson.build +++ b/contrib/hstore_plperl/meson.build @@ -37,6 +37,16 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': hstore_plperl, + 'srcfiles': hstore_plperl_sources, + 'additional_flags': [ + '-I@0@'.format(hstore_dir_up), + '-I@0@'.format(plperl_dir), + perl_ccflags, + ] +} + tests += { 'name': 'hstore_plperl', 'sd': meson.current_source_dir(), diff --git a/contrib/hstore_plpython/meson.build b/contrib/hstore_plpython/meson.build index ea8e20a377b..d68de3dc46b 100644 --- a/contrib/hstore_plpython/meson.build +++ b/contrib/hstore_plpython/meson.build @@ -30,6 +30,18 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': hstore_plpython, + 'srcfiles': hstore_plpython_sources, + 'additional_flags': [ + '-I@0@'.format(hstore_dir_up), + '-DPLPYTHON_LIBNAME="plpython3"', + '-I@0@'.format(python3_inc_dir), + '-I@0@'.format(plpython_dir), + perl_ccflags, + ] +} + hstore_plpython_regress = [ 'hstore_plpython' ] diff --git a/contrib/intarray/meson.build b/contrib/intarray/meson.build index fae9add981d..3d7da0c9c9a 100644 --- a/contrib/intarray/meson.build +++ b/contrib/intarray/meson.build @@ -33,6 +33,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': intarray, + 'srcfiles': intarray_sources, +} + tests += { 'name': 'intarray', 'sd': meson.current_source_dir(), diff --git a/contrib/isn/meson.build b/contrib/isn/meson.build index 39cf781684e..71dc8cf1a76 100644 --- a/contrib/isn/meson.build +++ b/contrib/isn/meson.build @@ -30,6 +30,11 @@ install_headers( install_dir: dir_include_extension / 'isn', ) +bitcode_modules += { + 'target': isn, + 'srcfiles': isn_sources, +} + tests += { 'name': 'isn', 'sd': meson.current_source_dir(), diff --git a/contrib/jsonb_plperl/meson.build b/contrib/jsonb_plperl/meson.build index 95a9a7bc082..febafc73a99 100644 --- a/contrib/jsonb_plperl/meson.build +++ b/contrib/jsonb_plperl/meson.build @@ -37,6 +37,14 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': jsonb_plperl, + 'srcfiles': jsonb_plperl_sources, + 'additional_flags': [ + '-I@0@'.format(plperl_dir), + perl_ccflags, + ] +} tests += { 'name': 'jsonb_plperl', diff --git a/contrib/jsonb_plpython/meson.build b/contrib/jsonb_plpython/meson.build index 5fe80483e58..6be92221272 100644 --- a/contrib/jsonb_plpython/meson.build +++ b/contrib/jsonb_plpython/meson.build @@ -30,6 +30,16 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': jsonb_plpython, + 'srcfiles': jsonb_plpython_sources, + 'additional_flags': [ + '-DPLPYTHON_LIBNAME="plpython3"', + '-I@0@'.format(python3_inc_dir), + '-I@0@'.format(plpython_dir), + ] +} + jsonb_plpython_regress = [ 'jsonb_plpython' ] diff --git a/contrib/lo/meson.build b/contrib/lo/meson.build index 2d78907ba12..1222faa9169 100644 --- a/contrib/lo/meson.build +++ b/contrib/lo/meson.build @@ -24,6 +24,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': lo, + 'srcfiles': lo_sources, +} + tests += { 'name': 'lo', 'sd': meson.current_source_dir(), diff --git a/contrib/ltree/meson.build b/contrib/ltree/meson.build index f9b06302839..cda86935544 100644 --- a/contrib/ltree/meson.build +++ b/contrib/ltree/meson.build @@ -41,6 +41,16 @@ install_headers( install_dir: dir_include_extension / 'ltree', ) +ltree_dir = meson.current_source_dir() +ltree_dir_up = join_paths(ltree_dir, '..') +bitcode_modules += { + 'target': ltree, + 'srcfiles': ltree_sources, + 'additional_flags': [ + '-I@0@'.format(ltree_dir) + ] +} + tests += { 'name': 'ltree', 'sd': meson.current_source_dir(), diff --git a/contrib/ltree_plpython/meson.build b/contrib/ltree_plpython/meson.build index a37732c486b..74b285da05c 100644 --- a/contrib/ltree_plpython/meson.build +++ b/contrib/ltree_plpython/meson.build @@ -30,6 +30,17 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': ltree_plpython, + 'srcfiles': ltree_plpython_sources, + 'additional_flags': [ + '-I@0@'.format(ltree_dir_up), + '-DPLPYTHON_LIBNAME="plpython3"', + '-I@0@'.format(python3_inc_dir), + '-I@0@'.format(plpython_dir), + ] +} + ltree_plpython_regress = [ 'ltree_plpython' ] diff --git a/contrib/pg_buffercache/meson.build b/contrib/pg_buffercache/meson.build index 7cd039a1df9..65dcc690601 100644 --- a/contrib/pg_buffercache/meson.build +++ b/contrib/pg_buffercache/meson.build @@ -28,6 +28,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pg_buffercache, + 'srcfiles': pg_buffercache_sources, +} + tests += { 'name': 'pg_buffercache', 'sd': meson.current_source_dir(), diff --git a/contrib/pg_freespacemap/meson.build b/contrib/pg_freespacemap/meson.build index ff8eda3580e..120def1ad2e 100644 --- a/contrib/pg_freespacemap/meson.build +++ b/contrib/pg_freespacemap/meson.build @@ -25,6 +25,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pg_freespacemap, + 'srcfiles': pg_freespacemap_sources, +} + tests += { 'name': 'pg_freespacemap', 'sd': meson.current_source_dir(), diff --git a/contrib/pg_logicalinspect/meson.build b/contrib/pg_logicalinspect/meson.build index 5c0528a8c63..1b4f46277ed 100644 --- a/contrib/pg_logicalinspect/meson.build +++ b/contrib/pg_logicalinspect/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pg_logicalinspect, + 'srcfiles': pg_logicalinspect_sources, +} + tests += { 'name': 'pg_logicalinspect', 'sd': meson.current_source_dir(), diff --git a/contrib/pg_surgery/meson.build b/contrib/pg_surgery/meson.build index c6cfa9c4694..9d7199392c7 100644 --- a/contrib/pg_surgery/meson.build +++ b/contrib/pg_surgery/meson.build @@ -22,6 +22,10 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pg_surgery, + 'srcfiles': pg_surgery_sources, +} tests += { 'name': 'pg_surgery', diff --git a/contrib/pg_trgm/meson.build b/contrib/pg_trgm/meson.build index a31aa5c574d..408e287172e 100644 --- a/contrib/pg_trgm/meson.build +++ b/contrib/pg_trgm/meson.build @@ -32,6 +32,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pg_trgm, + 'srcfiles': pg_trgm_sources, +} + tests += { 'name': 'pg_trgm', 'sd': meson.current_source_dir(), diff --git a/contrib/pgcrypto/meson.build b/contrib/pgcrypto/meson.build index 7d5ef9b6d32..65d94174f7e 100644 --- a/contrib/pgcrypto/meson.build +++ b/contrib/pgcrypto/meson.build @@ -100,6 +100,10 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pgcrypto, + 'srcfiles': pgcrypto_sources, +} tests += { 'name': 'pgcrypto', diff --git a/contrib/seg/meson.build b/contrib/seg/meson.build index e331e097230..371cd272a06 100644 --- a/contrib/seg/meson.build +++ b/contrib/seg/meson.build @@ -3,6 +3,7 @@ seg_sources = files( 'seg.c', ) +bc_seg_sources = seg_sources seg_scan = custom_target('segscan', input: 'segscan.l', @@ -11,6 +12,7 @@ seg_scan = custom_target('segscan', ) generated_sources += seg_scan seg_sources += seg_scan +bc_seg_gen_sources = [{'srcfiles': [seg_scan]}] seg_parse = custom_target('segparse', input: 'segparse.y', @@ -18,6 +20,7 @@ seg_parse = custom_target('segparse', ) generated_sources += seg_parse.to_list() seg_sources += seg_parse +bc_seg_gen_sources += {'srcfiles': [seg_parse[0]]} if host_system == 'windows' seg_sources += rc_lib_gen.process(win32ver_rc, extra_args: [ @@ -47,6 +50,15 @@ install_headers( install_dir: dir_include_extension / 'seg', ) +bitcode_modules += { + 'target': seg, + 'srcfiles': bc_seg_sources, + 'gen_sources': bc_seg_gen_sources, + 'additional_flags': [ + '-I@0@'.format(meson.current_source_dir()), + ] +} + tests += { 'name': 'seg', 'sd': meson.current_source_dir(), diff --git a/contrib/spi/meson.build b/contrib/spi/meson.build index 3832a91019a..c8216fcbcd0 100644 --- a/contrib/spi/meson.build +++ b/contrib/spi/meson.build @@ -24,6 +24,11 @@ install_data('autoinc.example', kwargs: contrib_doc_args, ) +bitcode_modules += { + 'target': autoinc, + 'srcfiles': autoinc_sources, +} + insert_username_sources = files( 'insert_username.c', @@ -51,6 +56,11 @@ install_data('insert_username.example', kwargs: contrib_doc_args, ) +bitcode_modules += { + 'target': insert_username, + 'srcfiles': insert_username_sources, +} + moddatetime_sources = files( 'moddatetime.c', @@ -78,6 +88,11 @@ install_data('moddatetime.example', kwargs: contrib_doc_args, ) +bitcode_modules += { + 'target': moddatetime, + 'srcfiles': moddatetime_sources, +} + # this is needed for the regression tests; # comment out if you want a quieter refint package for other uses @@ -108,6 +123,11 @@ install_data('refint.example', kwargs: contrib_doc_args, ) +bitcode_modules += { + 'target': refint, + 'srcfiles': refint_sources, +} + tests += { 'name': 'spi', 'sd': meson.current_source_dir(), diff --git a/contrib/sslinfo/meson.build b/contrib/sslinfo/meson.build index 4c513759200..5750d783b92 100644 --- a/contrib/sslinfo/meson.build +++ b/contrib/sslinfo/meson.build @@ -29,3 +29,8 @@ install_data( 'sslinfo.control', kwargs: contrib_data_args, ) + +bitcode_modules += { + 'target': sslinfo, + 'srcfiles': sslinfo_sources, +} diff --git a/contrib/tablefunc/meson.build b/contrib/tablefunc/meson.build index ee67272ec0a..6c26ffafddf 100644 --- a/contrib/tablefunc/meson.build +++ b/contrib/tablefunc/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': tablefunc, + 'srcfiles': tablefunc_sources, +} + tests += { 'name': 'tablefunc', 'sd': meson.current_source_dir(), diff --git a/contrib/tcn/meson.build b/contrib/tcn/meson.build index 6ffb136af90..df71e8a91f9 100644 --- a/contrib/tcn/meson.build +++ b/contrib/tcn/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': tcn, + 'srcfiles': tcn_sources, +} + tests += { 'name': 'tcn', 'sd': meson.current_source_dir(), diff --git a/contrib/tsm_system_rows/meson.build b/contrib/tsm_system_rows/meson.build index b8cece3d80f..1ea74cce949 100644 --- a/contrib/tsm_system_rows/meson.build +++ b/contrib/tsm_system_rows/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': tsm_system_rows, + 'srcfiles': tsm_system_rows_sources, +} + tests += { 'name': 'tsm_system_rows', 'sd': meson.current_source_dir(), diff --git a/contrib/tsm_system_time/meson.build b/contrib/tsm_system_time/meson.build index 8a143a8f8e6..1f6b98d3ec4 100644 --- a/contrib/tsm_system_time/meson.build +++ b/contrib/tsm_system_time/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': tsm_system_time, + 'srcfiles': tsm_system_time_sources, +} + tests += { 'name': 'tsm_system_time', 'sd': meson.current_source_dir(), diff --git a/contrib/unaccent/meson.build b/contrib/unaccent/meson.build index 33d4649bae1..1f5d120e3d8 100644 --- a/contrib/unaccent/meson.build +++ b/contrib/unaccent/meson.build @@ -28,6 +28,11 @@ install_data( install_dir: dir_data / 'tsearch_data' ) +bitcode_modules += { + 'target': unaccent, + 'srcfiles': unaccent_sources, +} + # XXX: Implement downlo tests += { 'name': 'unaccent', diff --git a/contrib/uuid-ossp/meson.build b/contrib/uuid-ossp/meson.build index 982f27c085f..8d21bb0a996 100644 --- a/contrib/uuid-ossp/meson.build +++ b/contrib/uuid-ossp/meson.build @@ -29,6 +29,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': uuid_ossp, + 'srcfiles': uuid_ossp_sources, +} + tests += { 'name': 'uuid-ossp', 'sd': meson.current_source_dir(), diff --git a/contrib/xml2/meson.build b/contrib/xml2/meson.build index 08d3c3b8e30..b23a8fe0cd7 100644 --- a/contrib/xml2/meson.build +++ b/contrib/xml2/meson.build @@ -32,6 +32,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': xml2, + 'srcfiles': xml2_sources, +} + tests += { 'name': 'xml2', 'sd': meson.current_source_dir(), diff --git a/src/pl/plperl/meson.build b/src/pl/plperl/meson.build index 7c4081c3460..06e853c1ac7 100644 --- a/src/pl/plperl/meson.build +++ b/src/pl/plperl/meson.build @@ -37,6 +37,7 @@ foreach n : ['SPI', 'Util'] plperl_sources += xs_c endforeach +plperl_dir = meson.current_source_dir() plperl_inc = include_directories('.') if host_system == 'windows' diff --git a/src/pl/plpython/meson.build b/src/pl/plpython/meson.build index 709e5932a93..e2c19771d85 100644 --- a/src/pl/plpython/meson.build +++ b/src/pl/plpython/meson.build @@ -29,6 +29,7 @@ plpython_sources += custom_target('spiexceptions.h', # FIXME: need to duplicate import library ugliness? plpython_inc = include_directories('.') +plpython_dir = meson.current_source_dir() if host_system == 'windows' plpython_sources += rc_lib_gen.process(win32ver_rc, extra_args: [ diff --git a/meson.build b/meson.build index c65ddd7ebe2..4c86275a528 100644 --- a/meson.build +++ b/meson.build @@ -1294,6 +1294,7 @@ if not pyopt.disabled() python3_inst = pm.find_installation(python.full_path(), required: pyopt) if python3_inst.found() python3_dep = python3_inst.dependency(embed: true, required: pyopt) + python3_inc_dir = python3_inst.get_variable('INCLUDEPY') # Remove this check after we depend on Meson >= 1.1.0 if not cc.check_header('Python.h', dependencies: python3_dep, required: pyopt, include_directories: postgres_inc) python3_dep = not_found_dep -- 2.49.0 [text/x-patch] v6-0007-meson-Add-LLVM-bitcode-emission-for-backend-sourc.patch (5.2K, 8-v6-0007-meson-Add-LLVM-bitcode-emission-for-backend-sourc.patch) download | inline diff: From a080e22ca017759532122b8f22b0e5955c9e0d3d Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Wed, 12 Mar 2025 10:44:46 +0300 Subject: [PATCH v6 7/7] meson: Add LLVM bitcode emission for backend sources Since generated backend sources may have their own compilation flags and must also be included in the postgres.index.bc, the way to make it work with current code was to create a new variable, called `bc_generated_backend_sources`, which is a list of dictionaries, each one having an optional 'additional_flags' and a `srclist` pointing to the list of custom_target generated sources. An example of a possible structure of bitcode_modules which is processed by the main meson llvm bitcode emission file src/backend/jit/llvm/bitcode/meson.build: ``` bitcode_modules = [ { 'name': 'postgres', 'target': postgres_lib, 'src_file': backend_sources, 'gen_srcfiles': [ { 'additional_flags': [ '-I/path/postgresl/src/backend/parser', '-I/path/postgresl/build/src/backend/parser', ], 'srcfiles': [ <custom_target for scan.c>, <custom_target for gram.c> ] } ] } ] ``` Author: Diego Fronza <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/backend/bootstrap/meson.build | 2 ++ src/backend/meson.build | 2 ++ src/backend/parser/meson.build | 8 ++++++++ src/backend/replication/meson.build | 4 ++++ src/backend/utils/fmgr/meson.build | 1 + 5 files changed, 17 insertions(+) diff --git a/src/backend/bootstrap/meson.build b/src/backend/bootstrap/meson.build index 29726c1ab4f..389c75f8081 100644 --- a/src/backend/bootstrap/meson.build +++ b/src/backend/bootstrap/meson.build @@ -13,6 +13,7 @@ bootscanner = custom_target('bootscanner', ) generated_sources += bootscanner boot_parser_sources += bootscanner +bc_generated_backend_sources += {'srcfiles': [bootscanner]} bootparse = custom_target('bootparse', input: 'bootparse.y', @@ -20,6 +21,7 @@ bootparse = custom_target('bootparse', ) generated_sources += bootparse.to_list() boot_parser_sources += bootparse +bc_generated_backend_sources += {'srcfiles': [bootparse[0]]} boot_parser = static_library('boot_parser', boot_parser_sources, diff --git a/src/backend/meson.build b/src/backend/meson.build index 5fb33660d3d..85453de1928 100644 --- a/src/backend/meson.build +++ b/src/backend/meson.build @@ -5,6 +5,7 @@ backend_sources = [] backend_link_with = [pgport_srv, common_srv] generated_backend_sources = [] +bc_generated_backend_sources = [] post_export_backend_sources = [] subdir('access') @@ -144,6 +145,7 @@ bitcode_modules += { 'name': 'postgres', 'target': postgres_lib, 'srcfiles': backend_sources, + 'gen_sources': bc_generated_backend_sources, } pg_mod_c_args = cflags_mod diff --git a/src/backend/parser/meson.build b/src/backend/parser/meson.build index 874aa749aa6..add472a0cd8 100644 --- a/src/backend/parser/meson.build +++ b/src/backend/parser/meson.build @@ -42,6 +42,14 @@ backend_parser = custom_target('gram', generated_sources += backend_parser.to_list() parser_sources += backend_parser +bc_generated_backend_sources += { + 'additional_flags': [ + '-I@0@'.format(meson.current_build_dir()), + '-I@0@'.format(meson.current_source_dir()), + ], + 'srcfiles': [backend_scanner, backend_parser[0]], +} + parser = static_library('parser', parser_sources, dependencies: [backend_code], diff --git a/src/backend/replication/meson.build b/src/backend/replication/meson.build index b0601498865..71ab1164960 100644 --- a/src/backend/replication/meson.build +++ b/src/backend/replication/meson.build @@ -19,6 +19,7 @@ repl_scanner = custom_target('repl_scanner', ) generated_sources += repl_scanner repl_parser_sources += repl_scanner +bc_generated_backend_sources += {'srcfiles': [repl_scanner]} repl_gram = custom_target('repl_gram', input: 'repl_gram.y', @@ -26,6 +27,7 @@ repl_gram = custom_target('repl_gram', ) generated_sources += repl_gram.to_list() repl_parser_sources += repl_gram +bc_generated_backend_sources += {'srcfiles': [repl_gram[0]]} syncrep_scanner = custom_target('syncrep_scanner', input: 'syncrep_scanner.l', @@ -34,6 +36,7 @@ syncrep_scanner = custom_target('syncrep_scanner', ) generated_sources += syncrep_scanner repl_parser_sources += syncrep_scanner +bc_generated_backend_sources += {'srcfiles': [syncrep_scanner]} syncrep_gram = custom_target('syncrep_gram', input: 'syncrep_gram.y', @@ -41,6 +44,7 @@ syncrep_gram = custom_target('syncrep_gram', ) generated_sources += syncrep_gram.to_list() repl_parser_sources += syncrep_gram +bc_generated_backend_sources += {'srcfiles': [syncrep_gram[0]]} repl_parser = static_library('repl_parser', repl_parser_sources, diff --git a/src/backend/utils/fmgr/meson.build b/src/backend/utils/fmgr/meson.build index b1dcab93e70..080f59988e9 100644 --- a/src/backend/utils/fmgr/meson.build +++ b/src/backend/utils/fmgr/meson.build @@ -8,3 +8,4 @@ backend_sources += files( # fmgrtab.c generated_backend_sources += fmgrtab_target[2] +bc_generated_backend_sources += {'srcfiles': [fmgrtab_target[2]]} -- 2.49.0 ^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: meson vs. llvm bitcode files @ 2025-08-13 13:25 Nazir Bilal Yavuz <[email protected]> parent: Nazir Bilal Yavuz <[email protected]> 0 siblings, 1 reply; 10+ messages in thread From: Nazir Bilal Yavuz @ 2025-08-13 13:25 UTC (permalink / raw) To: Diego Fronza <[email protected]>; +Cc: Peter Eisentraut <[email protected]>; pgsql-hackers; Andres Freund <[email protected]> Hi, On Mon, 7 Jul 2025 at 11:45, Nazir Bilal Yavuz <[email protected]> wrote: > > Mandatory rebase, v6 is attached. Rebase is needed due to 01d6832c10, v7 is attached. -- Regards, Nazir Bilal Yavuz Microsoft Attachments: [text/x-patch] v7-0001-meson-Add-postgresql-extension.pc-for-building-ex.patch (4.5K, 2-v7-0001-meson-Add-postgresql-extension.pc-for-building-ex.patch) download | inline diff: From d8bba14adf6e7ade5440c01f95390e701112d07d Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Sat, 27 Aug 2022 09:52:03 -0700 Subject: [PATCH v7 1/6] meson: Add postgresql-extension.pc for building extension libraries This should work with several other buildsystems. TODO: Docs Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/backend/meson.build | 110 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 110 insertions(+) diff --git a/src/backend/meson.build b/src/backend/meson.build index b831a541652..dd903591e34 100644 --- a/src/backend/meson.build +++ b/src/backend/meson.build @@ -188,6 +188,116 @@ pg_test_mod_args = pg_mod_args + { +############################################################### +# Define a .pc file that can be used to build server extensions +############################################################### + +pg_ext_vars = [] +pg_ext_vars_inst = [] +pg_ext_vars_uninst = [] + +pg_ext_cflags = pg_mod_c_args + cppflags +pg_ext_libs = [backend_mod_deps, thread_dep, ldflags, ldflags_mod] +pg_ext_subdirs = [''] + +# Compute directories to add include directories to the .pc files for. +# This is a bit more complicated due to port/win32 etc. +i = 0 +foreach incdir : postgres_inc_d + if fs.is_absolute(incdir) + # an absolute path from -Dextra_include_dirs + pg_ext_cflags += '-I@0@'.format(incdir) + continue + elif incdir.startswith('src/include') + subincdir = dir_include_pkg_rel / 'server' / incdir.split('src/include/').get(1, '') + else + subincdir = '' + endif + pg_ext_subdirs += subincdir + + # Add directories in source / build dir containing headers to cflags for the + # -uninstalled.pc. Older versions of pkg-config complain if a referenced + # variable is not defined, so we emit an empty one for the installed .pc + # file. + pg_ext_vars += [ + 'build_inc@0@=""'.format(i), + 'src_inc@0@=""'.format(i), + ] + pg_ext_vars_uninst += [ + 'build_inc@0@=-I${prefix}/@1@'.format(i, incdir), + 'src_inc@0@=-I${srcdir}/@1@'.format(i, incdir), + ] + pg_ext_cflags += [ + '${build_inc@0@}'.format(i), + '${src_inc@0@}'.format(i) + ] + + i += 1 +endforeach + + +# Extension modules should likely also use -fwrapv etc. But it it's a bit odd +# to expose it to a .pc file? +pg_ext_cflags_warn = pg_ext_cflags + cflags_warn +pg_ext_cflags += cflags + +# Directories for extensions to install into +# XXX: more needed +pg_ext_vars += 'pkglibdir=${prefix}/@0@'.format(dir_lib_pkg) +pg_ext_vars += 'dir_mod=${pkglibdir}' +pg_ext_vars += 'dir_data=${prefix}/@0@'.format(dir_data_extension) +pg_ext_vars += 'dir_include=${prefix}/@0@'.format(dir_include_extension) +pg_ext_vars += 'dir_doc=${prefix}/@0@'.format(dir_doc_extension) +pg_ext_vars += 'dir_bitcode=${prefix}/@0@'.format(dir_bitcode) +# referenced on some platforms, via mod_link_with_dir +pg_ext_vars += 'bindir=${prefix}/@0@'.format(dir_bin) + +# XXX: Define variables making it easy to define tests, too + +# Some platforms need linker flags to link with binary, they are the same +# between building with meson and .pc file, except that we have have to +# reference a variable to make it work for both normal and -uninstalled .pc +# files. +if mod_link_args_fmt.length() != 0 + assert(link_with_inst != '') + assert(link_with_uninst != '') + + pg_ext_vars_inst += 'mod_link_with=@0@'.format(link_with_inst) + pg_ext_vars_uninst += 'mod_link_with=@0@'.format(link_with_uninst) + + foreach el : mod_link_args_fmt + pg_ext_libs += el.format('${mod_link_with}') + endforeach +endif + +# main .pc to build extensions +pkgconfig.generate( + name: 'postgresql-extension', + description: 'PostgreSQL Extension Support', + url: pg_url, + + subdirs: pg_ext_subdirs, + libraries: pg_ext_libs, + extra_cflags: pg_ext_cflags, + + variables: pg_ext_vars + pg_ext_vars_inst, + uninstalled_variables: pg_ext_vars + pg_ext_vars_uninst, +) + +# a .pc depending on the above, but with all our warnings enabled +pkgconfig.generate( + name: 'postgresql-extension-warnings', + description: 'PostgreSQL Extension Support - Compiler Warnings', + requires: 'postgresql-extension', + url: pg_url, + extra_cflags: pg_ext_cflags_warn, + + variables: pg_ext_vars + pg_ext_vars_inst, + uninstalled_variables: pg_ext_vars + pg_ext_vars_uninst, +) + + + # Shared modules that, on some system, link against the server binary. Only # enter these after we defined the server build. -- 2.50.1 [text/x-patch] v7-0002-meson-Test-building-extensions-by-using-postgresq.patch (10.7K, 3-v7-0002-meson-Test-building-extensions-by-using-postgresq.patch) download | inline diff: From f1975f4b47d6da0334525062aaadd16940f6a34e Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Thu, 27 Feb 2025 17:45:31 +0300 Subject: [PATCH v7 2/6] meson: Test building extensions by using postgresql-extension.pc The 'test_meson_extensions' pyton wrapper is added to run these tests. It compiles and builds extensions at ${build}/testrun/meson_extensions/${extension_name} path. The tests for building amcheck, auth_delay and postgres_fdw extensions are added. These are also examples of how to build extensions by using postgresql-extension.pc. Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/test/modules/meson.build | 1 + .../modules/test_meson_extensions/meson.build | 3 + .../amcheck/meson.build | 28 ++++++++ .../auth_delay/meson.build | 17 +++++ .../test_pkg_config_extensions/meson.build | 24 +++++++ .../postgres_fdw/meson.build | 30 ++++++++ meson.build | 32 +++++++++ src/tools/ci/test_meson_extensions | 70 +++++++++++++++++++ 8 files changed, 205 insertions(+) create mode 100644 src/test/modules/test_meson_extensions/meson.build create mode 100644 src/test/modules/test_meson_extensions/test_pkg_config_extensions/amcheck/meson.build create mode 100644 src/test/modules/test_meson_extensions/test_pkg_config_extensions/auth_delay/meson.build create mode 100644 src/test/modules/test_meson_extensions/test_pkg_config_extensions/meson.build create mode 100644 src/test/modules/test_meson_extensions/test_pkg_config_extensions/postgres_fdw/meson.build create mode 100644 src/tools/ci/test_meson_extensions diff --git a/src/test/modules/meson.build b/src/test/modules/meson.build index 93be0f57289..4e63da3abfe 100644 --- a/src/test/modules/meson.build +++ b/src/test/modules/meson.build @@ -28,6 +28,7 @@ subdir('test_int128') subdir('test_integerset') subdir('test_json_parser') subdir('test_lfind') +subdir('test_meson_extensions') subdir('test_misc') subdir('test_oat_hooks') subdir('test_parser') diff --git a/src/test/modules/test_meson_extensions/meson.build b/src/test/modules/test_meson_extensions/meson.build new file mode 100644 index 00000000000..06cf5d555df --- /dev/null +++ b/src/test/modules/test_meson_extensions/meson.build @@ -0,0 +1,3 @@ +# Copyright (c) 2022-2025, PostgreSQL Global Development Group + +subdir('test_pkg_config_extensions') diff --git a/src/test/modules/test_meson_extensions/test_pkg_config_extensions/amcheck/meson.build b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/amcheck/meson.build new file mode 100644 index 00000000000..482a543eb86 --- /dev/null +++ b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/amcheck/meson.build @@ -0,0 +1,28 @@ +# Copyright (c) 2022-2025, PostgreSQL Global Development Group + +project('amcheck', 'c') + +amcheck_path = '../../../../../../contrib/amcheck/' + +amcheck_sources = files( + amcheck_path / 'verify_heapam.c', + amcheck_path / 'verify_nbtree.c', +) + +pg_ext = dependency('postgresql-extension-warnings') + +amcheck = shared_module('amcheck', + amcheck_sources, + dependencies: pg_ext, + install_dir: pg_ext.get_variable(pkgconfig: 'dir_mod'), +) + +install_data( + amcheck_path / 'amcheck.control', + amcheck_path / 'amcheck--1.0.sql', + amcheck_path / 'amcheck--1.0--1.1.sql', + amcheck_path / 'amcheck--1.1--1.2.sql', + amcheck_path / 'amcheck--1.2--1.3.sql', + amcheck_path / 'amcheck--1.3--1.4.sql', + install_dir: pg_ext.get_variable(pkgconfig: 'dir_data'), +) diff --git a/src/test/modules/test_meson_extensions/test_pkg_config_extensions/auth_delay/meson.build b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/auth_delay/meson.build new file mode 100644 index 00000000000..98ad24cc183 --- /dev/null +++ b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/auth_delay/meson.build @@ -0,0 +1,17 @@ +# Copyright (c) 2022-2025, PostgreSQL Global Development Group + +project('auth_delay', 'c') + +auth_delay_path = '../../../../../../contrib/auth_delay/' + +auth_delay_sources = files( + auth_delay_path / 'auth_delay.c', +) + +pg_ext = dependency('postgresql-extension-warnings') + +auth_delay = shared_module('auth_delay', + auth_delay_sources, + dependencies: pg_ext, + install_dir: pg_ext.get_variable(pkgconfig: 'dir_mod'), +) diff --git a/src/test/modules/test_meson_extensions/test_pkg_config_extensions/meson.build b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/meson.build new file mode 100644 index 00000000000..dae94a384a1 --- /dev/null +++ b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/meson.build @@ -0,0 +1,24 @@ +# Copyright (c) 2022-2025, PostgreSQL Global Development Group + +# pkgconfig is not available on Windows, so skip it. +if host_machine.system() == 'windows' + subdir_done() +endif + +meson_extension_tests += { + 'name': 'amcheck', + 'kind': 'pkg_config', + 'sd': meson.current_source_dir() / 'amcheck', +} + +meson_extension_tests += { + 'name': 'auth_delay', + 'kind': 'pkg_config', + 'sd': meson.current_source_dir() / 'auth_delay', +} + +meson_extension_tests += { + 'name': 'postgres_fdw', + 'kind': 'pkg_config', + 'sd': meson.current_source_dir() / 'postgres_fdw', +} diff --git a/src/test/modules/test_meson_extensions/test_pkg_config_extensions/postgres_fdw/meson.build b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/postgres_fdw/meson.build new file mode 100644 index 00000000000..6d561e3789b --- /dev/null +++ b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/postgres_fdw/meson.build @@ -0,0 +1,30 @@ +# Copyright (c) 2022-2025, PostgreSQL Global Development Group + +project('auth_delay', 'c') + +postgres_fdw_path = '../../../../../../contrib/postgres_fdw/' + +postgres_fdw_sources = files( + postgres_fdw_path / 'connection.c', + postgres_fdw_path / 'deparse.c', + postgres_fdw_path / 'option.c', + postgres_fdw_path / 'postgres_fdw.c', + postgres_fdw_path / 'shippable.c', +) + +pg_ext = dependency('postgresql-extension-warnings') +libpq = dependency('libpq') + +postgres_fdw = shared_module('postgres_fdw', + postgres_fdw_sources, + dependencies: [pg_ext, libpq], + install_dir: pg_ext.get_variable(pkgconfig: 'dir_mod'), +) + +install_data( + postgres_fdw_path / 'postgres_fdw.control', + postgres_fdw_path / 'postgres_fdw--1.0.sql', + postgres_fdw_path / 'postgres_fdw--1.0--1.1.sql', + postgres_fdw_path / 'postgres_fdw--1.1--1.2.sql', + install_dir: pg_ext.get_variable(pkgconfig: 'dir_data'), +) diff --git a/meson.build b/meson.build index 30e0edda3e7..a1142db800c 100644 --- a/meson.build +++ b/meson.build @@ -3051,6 +3051,7 @@ nls_targets = [] # Define the tests to distribute them to the correct test styles later test_deps = [] tests = [] +meson_extension_tests = [] # Default options for targets @@ -3613,6 +3614,37 @@ sys.exit(sp.returncode) suite: ['setup']) +# it seems freebsd doesn't use libdir for pkgconfig path +if host_system == 'freebsd' + pkgconf_installdir = dir_prefix / 'libdata' / 'pkgconfig' +else + pkgconf_installdir = dir_prefix / dir_lib / 'pkgconfig' +endif +test_pkg_conf_file = files('src/tools/ci/test_meson_extensions') + +foreach test : meson_extension_tests + if test['kind'] not in ['pkg_config'] + error('unknown kind @0@ of test in @1@'.format(test['kind'], test['sd'])) + endif + + test_group = 'meson_@0@_extensions'.format(test['kind']) + + test(test_group / test['name'], + test_pkg_conf_file, + args: [ + '--meson', meson_bin.path(), + '--meson_args', meson_args, + '--test_dir', test['sd'], + '--test_out_dir', test_result_dir / 'meson_extensions' / test['name'], + '--builddir', meson.build_root(), + '--pkg_conf_path', get_option('pkg_config_path'), + '--', + cc.cmd_array(), + ], + suite: test_group, + ) + +endforeach ############################################################### # Test Generation diff --git a/src/tools/ci/test_meson_extensions b/src/tools/ci/test_meson_extensions new file mode 100644 index 00000000000..50358121f49 --- /dev/null +++ b/src/tools/ci/test_meson_extensions @@ -0,0 +1,70 @@ +#!/usr/bin/env python3 + +import argparse +import os +import shutil +import subprocess + +parser = argparse.ArgumentParser() + +parser.add_argument('--meson', help='path to meson binary', + type=str, required=True) +parser.add_argument('--meson_args', help='args of meson binary', + type=str, nargs='*', required=False) +parser.add_argument('--test_dir', help='test source directory', + type=str, required=True) +parser.add_argument('--test_out_dir', help='test output directory', + type=str, required=True) +parser.add_argument('--builddir', help='meson build directory', + type=str, required=True) +parser.add_argument('--pkg_conf_path', + help='PKG_CONF_PATH from surrounding meson build', + type=str, nargs='?', const='', required=False) +parser.add_argument('c_args', help='c_args from surrounding meson build', + nargs='*') + +args = parser.parse_args() + +meson_bin = args.meson +meson_args = ' '.join(args.meson_args) +test_source_dir = args.test_dir +test_out_dir = args.test_out_dir +build_dir = args.builddir +pkg_conf_path = args.pkg_conf_path +c_args = ' '.join(args.c_args) + +exit_code = 0 + +def remove_duplicates(duplicate_str): + words = duplicate_str.split() + return ' '.join(sorted(set(words), key=words.index)) + + +def run_tests(pkg_conf_path_local, message): + print('\n{}\n{}\n'.format('#' * 60, message), flush=True) + + env = {**os.environ, } + env['PKG_CONFIG_PATH'] = '{}:{}:{}'.format( + pkg_conf_path_local, pkg_conf_path, env.get('PKG_CONFIG_PATH', ''), + ).strip(': ') + env['CC'] = '{} {}'.format( + c_args, env.get('CC', ''), + ) + env['CC'] = remove_duplicates(env['CC']) + + # Clear the build directory beforehand. + if os.path.exists(test_out_dir): + shutil.rmtree(test_out_dir) + + if meson_args: + meson_setup_command = [meson_bin, meson_args, 'setup', test_out_dir] + else: + meson_setup_command = [meson_bin, 'setup', test_out_dir] + + meson_compile_command = ['meson', 'compile', '-C', test_out_dir, '-v'] + + subprocess.run(meson_setup_command, env=env, cwd=test_source_dir, check=True) + subprocess.run(meson_compile_command, cwd=test_source_dir, check=True) + +run_tests(os.path.join(build_dir, 'meson-uninstalled'), + message='Testing postgresql-extension-warnings-uninstalled') -- 2.50.1 [text/x-patch] v7-0003-meson-WIP-Add-docs-for-postgresql-extension.pc.patch (12.3K, 4-v7-0003-meson-WIP-Add-docs-for-postgresql-extension.pc.patch) download | inline diff: From 0f98e0fb67bcd57c3460fda77377ad3af6f1c074 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Thu, 6 Mar 2025 17:46:57 +0300 Subject: [PATCH v7 3/6] meson: [WIP] Add docs for postgresql-extension.pc Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- doc/src/sgml/acronyms.sgml | 2 +- doc/src/sgml/extend.sgml | 101 +++++++++++++++++++++++-------------- doc/src/sgml/jit.sgml | 2 +- 3 files changed, 66 insertions(+), 39 deletions(-) diff --git a/doc/src/sgml/acronyms.sgml b/doc/src/sgml/acronyms.sgml index 2f906e9f018..57f49c06a19 100644 --- a/doc/src/sgml/acronyms.sgml +++ b/doc/src/sgml/acronyms.sgml @@ -579,7 +579,7 @@ <term><acronym>PGXS</acronym></term> <listitem> <para> - <link linkend="extend-pgxs"><productname>PostgreSQL</productname> Extension System</link> + <link linkend="extend-postgres-pgxs"><productname>PostgreSQL</productname> Extension System</link> </para> </listitem> </varlistentry> diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml index 63c5ec6d1eb..6d9fe5e96ad 100644 --- a/doc/src/sgml/extend.sgml +++ b/doc/src/sgml/extend.sgml @@ -1426,7 +1426,7 @@ include $(PGXS) </programlisting> This makefile relies on <acronym>PGXS</acronym>, which is described - in <xref linkend="extend-pgxs"/>. The command <literal>make install</literal> + in <xref linkend="extend-postgres-pgxs"/>. The command <literal>make install</literal> will install the control and script files into the correct directory as reported by <application>pg_config</application>. </para> @@ -1439,21 +1439,26 @@ include $(PGXS) </sect2> </sect1> - <sect1 id="extend-pgxs"> + <sect1 id="extend-postgres"> <title>Extension Building Infrastructure</title> - <indexterm zone="extend-pgxs"> - <primary>pgxs</primary> - </indexterm> - <para> If you are thinking about distributing your <productname>PostgreSQL</productname> extension modules, setting up a portable build system for them can be fairly difficult. Therefore the <productname>PostgreSQL</productname> installation provides a build - infrastructure for extensions, called <acronym>PGXS</acronym>, so - that simple extension modules can be built simply against an - already installed server. <acronym>PGXS</acronym> is mainly intended + infrastructure for extensions, called <literal>PGXS</literal> + (<xref linkend="extend-postgres-pgxs"/>) and + its meson counterpart <literal>postgresql-extension.pc</literal> + (<xref linkend="extend-postgres-meson"/>). + </para> + + </sect1> + + <sect1 id="extend-postgres-pgxs"> + <title>PGXS</title> + + <para> <acronym>PGXS</acronym> is mainly intended for extensions that include C code, although it can be used for pure-SQL extensions too. Note that <acronym>PGXS</acronym> is not intended to be a universal build system framework that can be used @@ -1493,7 +1498,7 @@ include $(PGXS) Set one of these three variables to specify what is built: <variablelist> - <varlistentry id="extend-pgxs-modules"> + <varlistentry id="extend-postgres-pgxs-modules"> <term><varname>MODULES</varname></term> <listitem> <para> @@ -1503,7 +1508,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-module-big"> + <varlistentry id="extend-postgres-pgxs-module-big"> <term><varname>MODULE_big</varname></term> <listitem> <para> @@ -1513,7 +1518,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-program"> + <varlistentry id="extend-postgres-pgxs-program"> <term><varname>PROGRAM</varname></term> <listitem> <para> @@ -1527,7 +1532,7 @@ include $(PGXS) The following variables can also be set: <variablelist> - <varlistentry id="extend-pgxs-extension"> + <varlistentry id="extend-postgres-pgxs-extension"> <term><varname>EXTENSION</varname></term> <listitem> <para> @@ -1539,7 +1544,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-moduledir"> + <varlistentry id="extend-postgres-pgxs-moduledir"> <term><varname>MODULEDIR</varname></term> <listitem> <para> @@ -1552,7 +1557,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-data"> + <varlistentry id="extend-postgres-pgxs-data"> <term><varname>DATA</varname></term> <listitem> <para> @@ -1561,7 +1566,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-data-built"> + <varlistentry id="extend-postgres-pgxs-data-built"> <term><varname>DATA_built</varname></term> <listitem> <para> @@ -1572,7 +1577,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-data-tsearch"> + <varlistentry id="extend-postgres-pgxs-data-tsearch"> <term><varname>DATA_TSEARCH</varname></term> <listitem> <para> @@ -1582,7 +1587,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-docs"> + <varlistentry id="extend-postgres-pgxs-docs"> <term><varname>DOCS</varname></term> <listitem> <para> @@ -1592,7 +1597,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-headers"> + <varlistentry id="extend-postgres-pgxs-headers"> <term><varname>HEADERS</varname></term> <term><varname>HEADERS_built</varname></term> <listitem> @@ -1608,7 +1613,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-headers-module"> + <varlistentry id="extend-postgres-pgxs-headers-module"> <term><varname>HEADERS_$MODULE</varname></term> <term><varname>HEADERS_built_$MODULE</varname></term> <listitem> @@ -1634,7 +1639,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-scripts"> + <varlistentry id="extend-postgres-pgxs-scripts"> <term><varname>SCRIPTS</varname></term> <listitem> <para> @@ -1644,7 +1649,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-scripts-built"> + <varlistentry id="extend-postgres-pgxs-scripts-built"> <term><varname>SCRIPTS_built</varname></term> <listitem> <para> @@ -1655,7 +1660,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-regress"> + <varlistentry id="extend-postgres-pgxs-regress"> <term><varname>REGRESS</varname></term> <listitem> <para> @@ -1664,7 +1669,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-regress-opts"> + <varlistentry id="extend-postgres-pgxs-regress-opts"> <term><varname>REGRESS_OPTS</varname></term> <listitem> <para> @@ -1673,7 +1678,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-isolation"> + <varlistentry id="extend-postgres-pgxs-isolation"> <term><varname>ISOLATION</varname></term> <listitem> <para> @@ -1682,7 +1687,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-isolation-opts"> + <varlistentry id="extend-postgres-pgxs-isolation-opts"> <term><varname>ISOLATION_OPTS</varname></term> <listitem> <para> @@ -1692,7 +1697,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-tap-tests"> + <varlistentry id="extend-postgres-pgxs-tap-tests"> <term><varname>TAP_TESTS</varname></term> <listitem> <para> @@ -1701,7 +1706,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-no-install"> + <varlistentry id="extend-postgres-pgxs-no-install"> <term><varname>NO_INSTALL</varname></term> <listitem> <para> @@ -1711,7 +1716,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-no-installcheck"> + <varlistentry id="extend-postgres-pgxs-no-installcheck"> <term><varname>NO_INSTALLCHECK</varname></term> <listitem> <para> @@ -1720,7 +1725,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-extra-clean"> + <varlistentry id="extend-postgres-pgxs-extra-clean"> <term><varname>EXTRA_CLEAN</varname></term> <listitem> <para> @@ -1729,7 +1734,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-cppflags"> + <varlistentry id="extend-postgres-pgxs-pg-cppflags"> <term><varname>PG_CPPFLAGS</varname></term> <listitem> <para> @@ -1738,7 +1743,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-cflags"> + <varlistentry id="extend-postgres-pgxs-pg-cflags"> <term><varname>PG_CFLAGS</varname></term> <listitem> <para> @@ -1747,7 +1752,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-cxxflags"> + <varlistentry id="extend-postgres-pgxs-pg-cxxflags"> <term><varname>PG_CXXFLAGS</varname></term> <listitem> <para> @@ -1756,7 +1761,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-ldflags"> + <varlistentry id="extend-postgres-pgxs-pg-ldflags"> <term><varname>PG_LDFLAGS</varname></term> <listitem> <para> @@ -1765,7 +1770,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-libs"> + <varlistentry id="extend-postgres-pgxs-pg-libs"> <term><varname>PG_LIBS</varname></term> <listitem> <para> @@ -1774,7 +1779,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-shlib-link"> + <varlistentry id="extend-postgres-pgxs-shlib-link"> <term><varname>SHLIB_LINK</varname></term> <listitem> <para> @@ -1783,7 +1788,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-config"> + <varlistentry id="extend-postgres-pgxs-pg-config"> <term><varname>PG_CONFIG</varname></term> <listitem> <para> @@ -1929,4 +1934,26 @@ make VPATH=/path/to/extension/source/tree install </tip> </sect1> + <sect1 id="extend-postgres-meson"> + <title>postgresql-extension.pc</title> + + <para> + When Postgres is built by using meson, it generates + <literal>postgresql-extension.pc</literal> pkg-config file. Extension + libraries can use this file like <literal>PGXS</literal> + (<xref linkend="extend-postgres-pgxs"/>). + + To use the <literal>postgresql-extension.pc</literal> infrastructure for + your extension, you must write a simple meson.build file. In the + meson.build file, you need to include the + <literal>postgresql-extension.pc</literal> pkg-config file. Here is an + example that builds an extension module named isbn_issn, consisting of a + shared library containing some C code, an extension control file, an SQL + script, an include file (only needed if other modules might need to access + the extension functions without going via SQL), and a documentation text + file: + </para> + + </sect1> + </chapter> diff --git a/doc/src/sgml/jit.sgml b/doc/src/sgml/jit.sgml index 44e18bf1a6f..81a4644a97d 100644 --- a/doc/src/sgml/jit.sgml +++ b/doc/src/sgml/jit.sgml @@ -223,7 +223,7 @@ SET of types <literal>C</literal> and <literal>internal</literal>, as well as operators based on such functions. To do so for functions in extensions, the definitions of those functions need to be made available. - When using <link linkend="extend-pgxs">PGXS</link> to build an extension + When using <link linkend="extend-postgres-pgxs">PGXS</link> to build an extension against a server that has been compiled with LLVM JIT support, the relevant files will be built and installed automatically. </para> -- 2.50.1 [text/x-patch] v7-0004-meson-Add-architecture-for-LLVM-bitcode-emission.patch (9.0K, 5-v7-0004-meson-Add-architecture-for-LLVM-bitcode-emission.patch) download | inline diff: From 146a3dc852f04216b8aa805e4e7739cbcf58b117 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Fri, 7 Mar 2025 12:10:58 +0300 Subject: [PATCH v7 4/6] meson: Add architecture for LLVM bitcode emission This commit adds suport for bitcode emission for both normal and generated source files (processed by bison, flex, etc). These bitcode files are installed into $pkglibdir/bitcode/ directory if the LLVM is found. New variable `bitcode_modules` is introduced to generate bitcode files. All required information is gathered in this variable. Then, this variable is processed by the main meson LLVM bitcode emission scripts: src/backend/jit/llvm/bitcode/meson.build -> src/tools/irlink. An example of a possible structure of bitcode_modules is: ``` bitcode_modules = [ { 'name': '...', 'target': ..., 'srcfiles': [ '...', '...', ], 'additional_flags': [ '-I...', '-I...', ], 'gen_srcfiles': [ { 'srcfiles': [ <custom_target for ...>, <custom_target for ...>, ], 'additional_flags': [ '-I...', '-I...', ] } ] } ] ``` Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Author: Diego Fronza <[email protected]> Reviewed-by: Diego Fronza <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/backend/jit/llvm/bitcode/meson.build | 69 ++++++++++++++++++++++++ src/backend/jit/llvm/meson.build | 31 +++++++---- src/backend/meson.build | 6 +++ meson.build | 21 ++++++++ src/tools/irlink | 25 +++++++++ 5 files changed, 141 insertions(+), 11 deletions(-) create mode 100644 src/backend/jit/llvm/bitcode/meson.build create mode 100644 src/tools/irlink diff --git a/src/backend/jit/llvm/bitcode/meson.build b/src/backend/jit/llvm/bitcode/meson.build new file mode 100644 index 00000000000..546e4d8b898 --- /dev/null +++ b/src/backend/jit/llvm/bitcode/meson.build @@ -0,0 +1,69 @@ +# Copyright (c) 2022-2024, PostgreSQL Global Development Group +# +# emit LLVM bitcode for JIT inlining + +assert(llvm.found()) + +foreach bitcode_module : bitcode_modules + bitcode_targets = [] + bitcode_obj = bitcode_module['target'] + bitcode_cflags_local = bitcode_cflags + bitcode_module.get('additional_flags', []) + bitcode_name = bitcode_module.get('name', bitcode_obj.name()) + + foreach srcfile : bitcode_module['srcfiles'] + if meson.version().version_compare('>=0.59') + srcfilename = fs.parent(srcfile) / fs.name(srcfile) + else + srcfilename = '@0@'.format(srcfile) + endif + + targetname = '@0@_@[email protected]'.format( + bitcode_name, + srcfilename.underscorify(), + ) + bitcode_targets += custom_target( + targetname, + depends: [bitcode_obj], + input: [srcfile], + output: targetname, + command: [llvm_irgen_command, bitcode_cflags_local], + install: true, + install_dir: dir_bitcode, + ) + endforeach + + # Process generated sources, which may include custom compilation flags. + foreach gen_sources: bitcode_module.get('gen_sources', []) + bitcode_cflags_gen_local = bitcode_cflags_local + gen_sources.get('additional_flags', []) + + foreach srcfile: gen_sources['srcfiles'] + # Generated sources are stored in some folder under meson.build_root()/**, + # remove the build prefix from the string. + srcfilename = srcfile.full_path().split(meson.build_root() + '/')[1] + + targetname = '@0@_@[email protected]'.format( + bitcode_name, + srcfilename.underscorify(), + ) + bitcode_targets += custom_target( + targetname, + depends: [bitcode_obj], + input: [srcfile], + output: targetname, + command: [llvm_irgen_command, bitcode_cflags_gen_local], + install: true, + install_dir: dir_bitcode, + ) + endforeach + endforeach + + index_name = '@[email protected]'.format(bitcode_name) + bitcode_index = custom_target('@0@'.format(bitcode_name), + output: index_name, + input: bitcode_targets, + command: [irlink, '--lto', llvm_lto, '--outdir', '@OUTDIR@', '--index', index_name, '@INPUT@'], + install: true, + install_dir: dir_bitcode, + ) + backend_targets += bitcode_index +endforeach diff --git a/src/backend/jit/llvm/meson.build b/src/backend/jit/llvm/meson.build index 805fbd69006..2dd922e573b 100644 --- a/src/backend/jit/llvm/meson.build +++ b/src/backend/jit/llvm/meson.build @@ -42,21 +42,22 @@ backend_targets += llvmjit # Define a few bits and pieces used here and elsewhere to generate bitcode -llvm_irgen_args = [ - '-c', '-o', '@OUTPUT@', '@INPUT@', +llvm_irgen_command = [] +if ccache.found() + llvm_irgen_command += ccache +endif + +llvm_irgen_command += [ + clang, + '-c', '-o', '@OUTPUT0@', '@INPUT0@', '-flto=thin', '-emit-llvm', - '-MD', '-MQ', '@OUTPUT@', '-MF', '@DEPFILE@', '-O2', '-Wno-ignored-attributes', '-Wno-empty-body', + '-Wno-unknown-warning-option', + '-Wno-compound-token-split-by-macro', ] - -if ccache.found() - llvm_irgen_command = ccache - llvm_irgen_args = [clang.full_path()] + llvm_irgen_args -else - llvm_irgen_command = clang -endif +llvm_irgen_dep_args = ['-MD', '-MQ', '@OUTPUT0@', '-MF', '@DEPFILE@'] # XXX: Need to determine proper version of the function cflags for clang @@ -73,7 +74,7 @@ bitcode_cflags += '-I@SOURCE_ROOT@/src/include' # Note this is intentionally not installed to bitcodedir, as it's not for # inlining llvmjit_types = custom_target('llvmjit_types.bc', - command: [llvm_irgen_command] + llvm_irgen_args + bitcode_cflags, + command: llvm_irgen_command + llvm_irgen_dep_args + bitcode_cflags, input: 'llvmjit_types.c', output: 'llvmjit_types.bc', depends: [postgres], @@ -82,3 +83,11 @@ llvmjit_types = custom_target('llvmjit_types.bc', depfile: '@[email protected]', ) backend_targets += llvmjit_types + +# Figure out -I's needed to build all postgres code, including all its +# dependencies +pkg_config = find_program(['pkg-config', 'pkgconf'], required: true) +r = run_command(pkg_config, + ['--cflags-only-I', meson.build_root() / 'meson-uninstalled/postgresql-extension-uninstalled.pc'], + check: true) +bitcode_cflags += r.stdout().split() diff --git a/src/backend/meson.build b/src/backend/meson.build index dd903591e34..276ecbd6d74 100644 --- a/src/backend/meson.build +++ b/src/backend/meson.build @@ -140,6 +140,12 @@ postgres = executable('postgres', backend_targets += postgres +bitcode_modules += { + 'name': 'postgres', + 'target': postgres_lib, + 'srcfiles': backend_sources, +} + pg_mod_c_args = cflags_mod pg_mod_cpp_args = cxxflags_mod pg_mod_link_args = ldflags_sl + ldflags_mod diff --git a/meson.build b/meson.build index a1142db800c..646148e8373 100644 --- a/meson.build +++ b/meson.build @@ -820,6 +820,8 @@ if add_languages('cpp', required: llvmopt, native: false) # Some distros put LLVM and clang in different paths, so fallback to # find via PATH, too. clang = find_program(llvm_binpath / 'clang', 'clang', required: true) + llvm_lto = find_program(llvm_binpath / 'llvm-lto', required: true) + irlink = find_program('src/tools/irlink', native: true) endif elif llvmopt.auto() message('llvm requires a C++ compiler') @@ -3053,6 +3055,11 @@ test_deps = [] tests = [] meson_extension_tests = [] +# List of object files + source files to generated LLVM IR for inlining. +# Each element is a hash of: +# {'target': target, 'srcfiles': ..., 'additional_flags': ...}. +bitcode_modules = [] + # Default options for targets @@ -3375,6 +3382,11 @@ subdir('src/interfaces/ecpg/test') subdir('doc/src/sgml') +# generate bitcode for JIT inlining after giving contrib modules etc a chance +# to add themselves to bitcode_modules[] +subdir('src/backend/jit/llvm/bitcode', if_found: llvm) + + generated_sources_ac += {'': ['GNUmakefile']} # After processing src/test, add test_install_libs to the testprep_targets @@ -4004,6 +4016,15 @@ summary( section: 'Programs', ) +if llvm.found() + summary( + { + 'clang': clang, + }, + section: 'Programs', + ) +endif + summary( { 'bonjour': bonjour, diff --git a/src/tools/irlink b/src/tools/irlink new file mode 100644 index 00000000000..793c0abf91a --- /dev/null +++ b/src/tools/irlink @@ -0,0 +1,25 @@ +#!/usr/bin/env python3 + +import argparse +import os +import shutil +import subprocess +import sys + +parser = argparse.ArgumentParser( + description='generate PostgreSQL JIT IR module') + +parser.add_argument('--index', type=str, required=True) +parser.add_argument('--lto', type=str, required=True) +parser.add_argument('--outdir', type=str, required=True) +parser.add_argument('INPUT', type=str, nargs='+') + +args = parser.parse_args() + +file_names = [os.path.basename(f) for f in args.INPUT] +command = [args.lto, + '-thinlto', '-thinlto-action=thinlink', + '-o', args.index] + file_names +res = subprocess.run(command, cwd=args.outdir) + +exit(res.returncode) -- 2.50.1 [text/x-patch] v7-0005-meson-Add-LLVM-bitcode-emissions-for-contrib-libr.patch (22.7K, 6-v7-0005-meson-Add-LLVM-bitcode-emissions-for-contrib-libr.patch) download | inline diff: From 501a49d8633e7edb7f53584b4fe8c46af7098abc Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Wed, 12 Mar 2025 10:49:18 +0300 Subject: [PATCH v7 5/6] meson: Add LLVM bitcode emissions for contrib libraries The libraries which the bitcode files will be generated in are selected manually. Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Author: Diego Fronza <[email protected]> Reviewed-by: Diego Fronza <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/interfaces/libpq/meson.build | 3 +++ contrib/bloom/meson.build | 5 +++++ contrib/bool_plperl/meson.build | 9 +++++++++ contrib/btree_gin/meson.build | 5 +++++ contrib/btree_gist/meson.build | 5 +++++ contrib/citext/meson.build | 5 +++++ contrib/cube/meson.build | 13 +++++++++++++ contrib/dict_int/meson.build | 5 +++++ contrib/dict_xsyn/meson.build | 5 +++++ contrib/earthdistance/meson.build | 6 ++++++ contrib/fuzzystrmatch/meson.build | 9 +++++++++ contrib/hstore/meson.build | 7 +++++++ contrib/hstore_plperl/meson.build | 10 ++++++++++ contrib/hstore_plpython/meson.build | 12 ++++++++++++ contrib/intarray/meson.build | 5 +++++ contrib/isn/meson.build | 5 +++++ contrib/jsonb_plperl/meson.build | 8 ++++++++ contrib/jsonb_plpython/meson.build | 10 ++++++++++ contrib/lo/meson.build | 5 +++++ contrib/ltree/meson.build | 10 ++++++++++ contrib/ltree_plpython/meson.build | 11 +++++++++++ contrib/pg_buffercache/meson.build | 5 +++++ contrib/pg_freespacemap/meson.build | 5 +++++ contrib/pg_logicalinspect/meson.build | 5 +++++ contrib/pg_surgery/meson.build | 4 ++++ contrib/pg_trgm/meson.build | 5 +++++ contrib/pgcrypto/meson.build | 4 ++++ contrib/seg/meson.build | 12 ++++++++++++ contrib/spi/meson.build | 20 ++++++++++++++++++++ contrib/sslinfo/meson.build | 5 +++++ contrib/tablefunc/meson.build | 5 +++++ contrib/tcn/meson.build | 5 +++++ contrib/tsm_system_rows/meson.build | 5 +++++ contrib/tsm_system_time/meson.build | 5 +++++ contrib/unaccent/meson.build | 5 +++++ contrib/uuid-ossp/meson.build | 5 +++++ contrib/xml2/meson.build | 5 +++++ src/pl/plperl/meson.build | 1 + src/pl/plpython/meson.build | 1 + meson.build | 1 + 40 files changed, 256 insertions(+) diff --git a/src/interfaces/libpq/meson.build b/src/interfaces/libpq/meson.build index a74e885b169..c3ceab1dd70 100644 --- a/src/interfaces/libpq/meson.build +++ b/src/interfaces/libpq/meson.build @@ -49,6 +49,9 @@ libpq_c_args = ['-DSO_MAJOR_VERSION=5'] # The OAuth implementation differs depending on the type of library being built. libpq_so_c_args = ['-DUSE_DYNAMIC_OAUTH'] +# libpq-fe.h is needed to generate bitcode for some extensions +libpq_dir = meson.current_source_dir() + # Not using both_libraries() here as # 1) resource files should only be in the shared library # 2) we want the .pc file to include a dependency to {pgport,common}_static for diff --git a/contrib/bloom/meson.build b/contrib/bloom/meson.build index 695712a455e..1090c281532 100644 --- a/contrib/bloom/meson.build +++ b/contrib/bloom/meson.build @@ -28,6 +28,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': bloom, + 'srcfiles': bloom_sources, +} + tests += { 'name': 'bloom', 'sd': meson.current_source_dir(), diff --git a/contrib/bool_plperl/meson.build b/contrib/bool_plperl/meson.build index f489d99044c..1758adb7489 100644 --- a/contrib/bool_plperl/meson.build +++ b/contrib/bool_plperl/meson.build @@ -37,6 +37,15 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': bool_plperl, + 'srcfiles': bool_plperl_sources, + 'additional_flags': [ + '-I@0@'.format(plperl_dir), + perl_ccflags + ] +} + tests += { 'name': 'bool_plperl', 'sd': meson.current_source_dir(), diff --git a/contrib/btree_gin/meson.build b/contrib/btree_gin/meson.build index ece0a716973..e92e52822b0 100644 --- a/contrib/btree_gin/meson.build +++ b/contrib/btree_gin/meson.build @@ -26,6 +26,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': btree_gin, + 'srcfiles': btree_gin_sources, +} + tests += { 'name': 'btree_gin', 'sd': meson.current_source_dir(), diff --git a/contrib/btree_gist/meson.build b/contrib/btree_gist/meson.build index f4fa9574f1f..2649d5a7ffb 100644 --- a/contrib/btree_gist/meson.build +++ b/contrib/btree_gist/meson.build @@ -54,6 +54,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': btree_gist, + 'srcfiles': btree_gist_sources, +} + tests += { 'name': 'btree_gist', 'sd': meson.current_source_dir(), diff --git a/contrib/citext/meson.build b/contrib/citext/meson.build index 7fff34f2368..fa07ff72be4 100644 --- a/contrib/citext/meson.build +++ b/contrib/citext/meson.build @@ -30,6 +30,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': citext, + 'srcfiles': citext_sources, +} + tests += { 'name': 'citext', 'sd': meson.current_source_dir(), diff --git a/contrib/cube/meson.build b/contrib/cube/meson.build index fd3c057f469..a7649a74a1f 100644 --- a/contrib/cube/meson.build +++ b/contrib/cube/meson.build @@ -3,6 +3,7 @@ cube_sources = files( 'cube.c', ) +bc_cube_sources = cube_sources cube_scan = custom_target('cubescan', input: 'cubescan.l', @@ -11,6 +12,7 @@ cube_scan = custom_target('cubescan', ) generated_sources += cube_scan cube_sources += cube_scan +bc_cube_gen_sources = [{'srcfiles': [cube_scan]}] cube_parse = custom_target('cubeparse', input: 'cubeparse.y', @@ -18,6 +20,7 @@ cube_parse = custom_target('cubeparse', ) generated_sources += cube_parse.to_list() cube_sources += cube_parse +bc_cube_gen_sources += {'srcfiles': [cube_parse[0]]} if host_system == 'windows' cube_sources += rc_lib_gen.process(win32ver_rc, extra_args: [ @@ -48,6 +51,16 @@ install_headers( install_dir: dir_include_extension / 'cube', ) +cube_dir = meson.current_source_dir() +bitcode_modules += { + 'target': cube, + 'srcfiles': bc_cube_sources, + 'gen_sources': bc_cube_gen_sources, + 'additional_flags': [ + '-I@0@'.format(cube_dir), + ] +} + tests += { 'name': 'cube', 'sd': meson.current_source_dir(), diff --git a/contrib/dict_int/meson.build b/contrib/dict_int/meson.build index ab41588547b..894c8a9c94d 100644 --- a/contrib/dict_int/meson.build +++ b/contrib/dict_int/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': dict_int, + 'srcfiles': dict_int_sources, +} + tests += { 'name': 'dict_int', 'sd': meson.current_source_dir(), diff --git a/contrib/dict_xsyn/meson.build b/contrib/dict_xsyn/meson.build index 93f41b1f963..d4bd4a6ef5a 100644 --- a/contrib/dict_xsyn/meson.build +++ b/contrib/dict_xsyn/meson.build @@ -29,6 +29,11 @@ install_data( } ) +bitcode_modules += { + 'target': dict_xsyn, + 'srcfiles': dict_xsyn_sources, +} + tests += { 'name': 'dict_xsyn', 'sd': meson.current_source_dir(), diff --git a/contrib/earthdistance/meson.build b/contrib/earthdistance/meson.build index a5f8d1a3ffa..c101359b118 100644 --- a/contrib/earthdistance/meson.build +++ b/contrib/earthdistance/meson.build @@ -24,6 +24,12 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': earthdistance, + 'srcfiles': earthdistance_sources, +} + + tests += { 'name': 'earthdistance', 'sd': meson.current_source_dir(), diff --git a/contrib/fuzzystrmatch/meson.build b/contrib/fuzzystrmatch/meson.build index f52daa4ea1c..fc5e88d0c92 100644 --- a/contrib/fuzzystrmatch/meson.build +++ b/contrib/fuzzystrmatch/meson.build @@ -5,6 +5,7 @@ fuzzystrmatch_sources = files( 'dmetaphone.c', 'fuzzystrmatch.c', ) +bc_fuzzystrmatch_sources = fuzzystrmatch_sources daitch_mokotoff_h = custom_target('daitch_mokotoff', input: 'daitch_mokotoff_header.pl', @@ -35,6 +36,14 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': fuzzystrmatch, + 'srcfiles': bc_fuzzystrmatch_sources, + 'additional_flags': [ + '-I@0@'.format(meson.current_build_dir()) + ] +} + tests += { 'name': 'fuzzystrmatch', 'sd': meson.current_source_dir(), diff --git a/contrib/hstore/meson.build b/contrib/hstore/meson.build index 39622d1024d..344a620273f 100644 --- a/contrib/hstore/meson.build +++ b/contrib/hstore/meson.build @@ -43,6 +43,13 @@ install_headers( install_dir: dir_include_extension / 'hstore', ) +# some libraries include "hstore/hstore.h" instead of "hstore.h" +hstore_dir_up = join_paths(meson.current_source_dir(), '..') +bitcode_modules += { + 'target': hstore, + 'srcfiles': hstore_sources, +} + tests += { 'name': 'hstore', 'sd': meson.current_source_dir(), diff --git a/contrib/hstore_plperl/meson.build b/contrib/hstore_plperl/meson.build index 60b8ad23569..b689b55e1c9 100644 --- a/contrib/hstore_plperl/meson.build +++ b/contrib/hstore_plperl/meson.build @@ -37,6 +37,16 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': hstore_plperl, + 'srcfiles': hstore_plperl_sources, + 'additional_flags': [ + '-I@0@'.format(hstore_dir_up), + '-I@0@'.format(plperl_dir), + perl_ccflags, + ] +} + tests += { 'name': 'hstore_plperl', 'sd': meson.current_source_dir(), diff --git a/contrib/hstore_plpython/meson.build b/contrib/hstore_plpython/meson.build index ea8e20a377b..d68de3dc46b 100644 --- a/contrib/hstore_plpython/meson.build +++ b/contrib/hstore_plpython/meson.build @@ -30,6 +30,18 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': hstore_plpython, + 'srcfiles': hstore_plpython_sources, + 'additional_flags': [ + '-I@0@'.format(hstore_dir_up), + '-DPLPYTHON_LIBNAME="plpython3"', + '-I@0@'.format(python3_inc_dir), + '-I@0@'.format(plpython_dir), + perl_ccflags, + ] +} + hstore_plpython_regress = [ 'hstore_plpython' ] diff --git a/contrib/intarray/meson.build b/contrib/intarray/meson.build index fae9add981d..3d7da0c9c9a 100644 --- a/contrib/intarray/meson.build +++ b/contrib/intarray/meson.build @@ -33,6 +33,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': intarray, + 'srcfiles': intarray_sources, +} + tests += { 'name': 'intarray', 'sd': meson.current_source_dir(), diff --git a/contrib/isn/meson.build b/contrib/isn/meson.build index 39cf781684e..71dc8cf1a76 100644 --- a/contrib/isn/meson.build +++ b/contrib/isn/meson.build @@ -30,6 +30,11 @@ install_headers( install_dir: dir_include_extension / 'isn', ) +bitcode_modules += { + 'target': isn, + 'srcfiles': isn_sources, +} + tests += { 'name': 'isn', 'sd': meson.current_source_dir(), diff --git a/contrib/jsonb_plperl/meson.build b/contrib/jsonb_plperl/meson.build index 95a9a7bc082..febafc73a99 100644 --- a/contrib/jsonb_plperl/meson.build +++ b/contrib/jsonb_plperl/meson.build @@ -37,6 +37,14 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': jsonb_plperl, + 'srcfiles': jsonb_plperl_sources, + 'additional_flags': [ + '-I@0@'.format(plperl_dir), + perl_ccflags, + ] +} tests += { 'name': 'jsonb_plperl', diff --git a/contrib/jsonb_plpython/meson.build b/contrib/jsonb_plpython/meson.build index 5fe80483e58..6be92221272 100644 --- a/contrib/jsonb_plpython/meson.build +++ b/contrib/jsonb_plpython/meson.build @@ -30,6 +30,16 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': jsonb_plpython, + 'srcfiles': jsonb_plpython_sources, + 'additional_flags': [ + '-DPLPYTHON_LIBNAME="plpython3"', + '-I@0@'.format(python3_inc_dir), + '-I@0@'.format(plpython_dir), + ] +} + jsonb_plpython_regress = [ 'jsonb_plpython' ] diff --git a/contrib/lo/meson.build b/contrib/lo/meson.build index 2d78907ba12..1222faa9169 100644 --- a/contrib/lo/meson.build +++ b/contrib/lo/meson.build @@ -24,6 +24,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': lo, + 'srcfiles': lo_sources, +} + tests += { 'name': 'lo', 'sd': meson.current_source_dir(), diff --git a/contrib/ltree/meson.build b/contrib/ltree/meson.build index f9b06302839..cda86935544 100644 --- a/contrib/ltree/meson.build +++ b/contrib/ltree/meson.build @@ -41,6 +41,16 @@ install_headers( install_dir: dir_include_extension / 'ltree', ) +ltree_dir = meson.current_source_dir() +ltree_dir_up = join_paths(ltree_dir, '..') +bitcode_modules += { + 'target': ltree, + 'srcfiles': ltree_sources, + 'additional_flags': [ + '-I@0@'.format(ltree_dir) + ] +} + tests += { 'name': 'ltree', 'sd': meson.current_source_dir(), diff --git a/contrib/ltree_plpython/meson.build b/contrib/ltree_plpython/meson.build index a37732c486b..74b285da05c 100644 --- a/contrib/ltree_plpython/meson.build +++ b/contrib/ltree_plpython/meson.build @@ -30,6 +30,17 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': ltree_plpython, + 'srcfiles': ltree_plpython_sources, + 'additional_flags': [ + '-I@0@'.format(ltree_dir_up), + '-DPLPYTHON_LIBNAME="plpython3"', + '-I@0@'.format(python3_inc_dir), + '-I@0@'.format(plpython_dir), + ] +} + ltree_plpython_regress = [ 'ltree_plpython' ] diff --git a/contrib/pg_buffercache/meson.build b/contrib/pg_buffercache/meson.build index 7cd039a1df9..65dcc690601 100644 --- a/contrib/pg_buffercache/meson.build +++ b/contrib/pg_buffercache/meson.build @@ -28,6 +28,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pg_buffercache, + 'srcfiles': pg_buffercache_sources, +} + tests += { 'name': 'pg_buffercache', 'sd': meson.current_source_dir(), diff --git a/contrib/pg_freespacemap/meson.build b/contrib/pg_freespacemap/meson.build index ff8eda3580e..120def1ad2e 100644 --- a/contrib/pg_freespacemap/meson.build +++ b/contrib/pg_freespacemap/meson.build @@ -25,6 +25,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pg_freespacemap, + 'srcfiles': pg_freespacemap_sources, +} + tests += { 'name': 'pg_freespacemap', 'sd': meson.current_source_dir(), diff --git a/contrib/pg_logicalinspect/meson.build b/contrib/pg_logicalinspect/meson.build index 5c0528a8c63..1b4f46277ed 100644 --- a/contrib/pg_logicalinspect/meson.build +++ b/contrib/pg_logicalinspect/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pg_logicalinspect, + 'srcfiles': pg_logicalinspect_sources, +} + tests += { 'name': 'pg_logicalinspect', 'sd': meson.current_source_dir(), diff --git a/contrib/pg_surgery/meson.build b/contrib/pg_surgery/meson.build index c6cfa9c4694..9d7199392c7 100644 --- a/contrib/pg_surgery/meson.build +++ b/contrib/pg_surgery/meson.build @@ -22,6 +22,10 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pg_surgery, + 'srcfiles': pg_surgery_sources, +} tests += { 'name': 'pg_surgery', diff --git a/contrib/pg_trgm/meson.build b/contrib/pg_trgm/meson.build index a31aa5c574d..408e287172e 100644 --- a/contrib/pg_trgm/meson.build +++ b/contrib/pg_trgm/meson.build @@ -32,6 +32,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pg_trgm, + 'srcfiles': pg_trgm_sources, +} + tests += { 'name': 'pg_trgm', 'sd': meson.current_source_dir(), diff --git a/contrib/pgcrypto/meson.build b/contrib/pgcrypto/meson.build index 7d5ef9b6d32..65d94174f7e 100644 --- a/contrib/pgcrypto/meson.build +++ b/contrib/pgcrypto/meson.build @@ -100,6 +100,10 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pgcrypto, + 'srcfiles': pgcrypto_sources, +} tests += { 'name': 'pgcrypto', diff --git a/contrib/seg/meson.build b/contrib/seg/meson.build index e331e097230..371cd272a06 100644 --- a/contrib/seg/meson.build +++ b/contrib/seg/meson.build @@ -3,6 +3,7 @@ seg_sources = files( 'seg.c', ) +bc_seg_sources = seg_sources seg_scan = custom_target('segscan', input: 'segscan.l', @@ -11,6 +12,7 @@ seg_scan = custom_target('segscan', ) generated_sources += seg_scan seg_sources += seg_scan +bc_seg_gen_sources = [{'srcfiles': [seg_scan]}] seg_parse = custom_target('segparse', input: 'segparse.y', @@ -18,6 +20,7 @@ seg_parse = custom_target('segparse', ) generated_sources += seg_parse.to_list() seg_sources += seg_parse +bc_seg_gen_sources += {'srcfiles': [seg_parse[0]]} if host_system == 'windows' seg_sources += rc_lib_gen.process(win32ver_rc, extra_args: [ @@ -47,6 +50,15 @@ install_headers( install_dir: dir_include_extension / 'seg', ) +bitcode_modules += { + 'target': seg, + 'srcfiles': bc_seg_sources, + 'gen_sources': bc_seg_gen_sources, + 'additional_flags': [ + '-I@0@'.format(meson.current_source_dir()), + ] +} + tests += { 'name': 'seg', 'sd': meson.current_source_dir(), diff --git a/contrib/spi/meson.build b/contrib/spi/meson.build index 3832a91019a..c8216fcbcd0 100644 --- a/contrib/spi/meson.build +++ b/contrib/spi/meson.build @@ -24,6 +24,11 @@ install_data('autoinc.example', kwargs: contrib_doc_args, ) +bitcode_modules += { + 'target': autoinc, + 'srcfiles': autoinc_sources, +} + insert_username_sources = files( 'insert_username.c', @@ -51,6 +56,11 @@ install_data('insert_username.example', kwargs: contrib_doc_args, ) +bitcode_modules += { + 'target': insert_username, + 'srcfiles': insert_username_sources, +} + moddatetime_sources = files( 'moddatetime.c', @@ -78,6 +88,11 @@ install_data('moddatetime.example', kwargs: contrib_doc_args, ) +bitcode_modules += { + 'target': moddatetime, + 'srcfiles': moddatetime_sources, +} + # this is needed for the regression tests; # comment out if you want a quieter refint package for other uses @@ -108,6 +123,11 @@ install_data('refint.example', kwargs: contrib_doc_args, ) +bitcode_modules += { + 'target': refint, + 'srcfiles': refint_sources, +} + tests += { 'name': 'spi', 'sd': meson.current_source_dir(), diff --git a/contrib/sslinfo/meson.build b/contrib/sslinfo/meson.build index 4c513759200..5750d783b92 100644 --- a/contrib/sslinfo/meson.build +++ b/contrib/sslinfo/meson.build @@ -29,3 +29,8 @@ install_data( 'sslinfo.control', kwargs: contrib_data_args, ) + +bitcode_modules += { + 'target': sslinfo, + 'srcfiles': sslinfo_sources, +} diff --git a/contrib/tablefunc/meson.build b/contrib/tablefunc/meson.build index ee67272ec0a..6c26ffafddf 100644 --- a/contrib/tablefunc/meson.build +++ b/contrib/tablefunc/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': tablefunc, + 'srcfiles': tablefunc_sources, +} + tests += { 'name': 'tablefunc', 'sd': meson.current_source_dir(), diff --git a/contrib/tcn/meson.build b/contrib/tcn/meson.build index 6ffb136af90..df71e8a91f9 100644 --- a/contrib/tcn/meson.build +++ b/contrib/tcn/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': tcn, + 'srcfiles': tcn_sources, +} + tests += { 'name': 'tcn', 'sd': meson.current_source_dir(), diff --git a/contrib/tsm_system_rows/meson.build b/contrib/tsm_system_rows/meson.build index b8cece3d80f..1ea74cce949 100644 --- a/contrib/tsm_system_rows/meson.build +++ b/contrib/tsm_system_rows/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': tsm_system_rows, + 'srcfiles': tsm_system_rows_sources, +} + tests += { 'name': 'tsm_system_rows', 'sd': meson.current_source_dir(), diff --git a/contrib/tsm_system_time/meson.build b/contrib/tsm_system_time/meson.build index 8a143a8f8e6..1f6b98d3ec4 100644 --- a/contrib/tsm_system_time/meson.build +++ b/contrib/tsm_system_time/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': tsm_system_time, + 'srcfiles': tsm_system_time_sources, +} + tests += { 'name': 'tsm_system_time', 'sd': meson.current_source_dir(), diff --git a/contrib/unaccent/meson.build b/contrib/unaccent/meson.build index 33d4649bae1..1f5d120e3d8 100644 --- a/contrib/unaccent/meson.build +++ b/contrib/unaccent/meson.build @@ -28,6 +28,11 @@ install_data( install_dir: dir_data / 'tsearch_data' ) +bitcode_modules += { + 'target': unaccent, + 'srcfiles': unaccent_sources, +} + # XXX: Implement downlo tests += { 'name': 'unaccent', diff --git a/contrib/uuid-ossp/meson.build b/contrib/uuid-ossp/meson.build index 982f27c085f..8d21bb0a996 100644 --- a/contrib/uuid-ossp/meson.build +++ b/contrib/uuid-ossp/meson.build @@ -29,6 +29,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': uuid_ossp, + 'srcfiles': uuid_ossp_sources, +} + tests += { 'name': 'uuid-ossp', 'sd': meson.current_source_dir(), diff --git a/contrib/xml2/meson.build b/contrib/xml2/meson.build index 08d3c3b8e30..b23a8fe0cd7 100644 --- a/contrib/xml2/meson.build +++ b/contrib/xml2/meson.build @@ -32,6 +32,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': xml2, + 'srcfiles': xml2_sources, +} + tests += { 'name': 'xml2', 'sd': meson.current_source_dir(), diff --git a/src/pl/plperl/meson.build b/src/pl/plperl/meson.build index 7c4081c3460..06e853c1ac7 100644 --- a/src/pl/plperl/meson.build +++ b/src/pl/plperl/meson.build @@ -37,6 +37,7 @@ foreach n : ['SPI', 'Util'] plperl_sources += xs_c endforeach +plperl_dir = meson.current_source_dir() plperl_inc = include_directories('.') if host_system == 'windows' diff --git a/src/pl/plpython/meson.build b/src/pl/plpython/meson.build index 709e5932a93..e2c19771d85 100644 --- a/src/pl/plpython/meson.build +++ b/src/pl/plpython/meson.build @@ -29,6 +29,7 @@ plpython_sources += custom_target('spiexceptions.h', # FIXME: need to duplicate import library ugliness? plpython_inc = include_directories('.') +plpython_dir = meson.current_source_dir() if host_system == 'windows' plpython_sources += rc_lib_gen.process(win32ver_rc, extra_args: [ diff --git a/meson.build b/meson.build index 646148e8373..8e6aed17a85 100644 --- a/meson.build +++ b/meson.build @@ -1300,6 +1300,7 @@ if not pyopt.disabled() python3_inst = pm.find_installation(python.full_path(), required: pyopt) if python3_inst.found() python3_dep = python3_inst.dependency(embed: true, required: pyopt) + python3_inc_dir = python3_inst.get_variable('INCLUDEPY') # Remove this check after we depend on Meson >= 1.1.0 if not cc.check_header('Python.h', dependencies: python3_dep, required: pyopt, include_directories: postgres_inc) python3_dep = not_found_dep -- 2.50.1 [text/x-patch] v7-0006-meson-Add-LLVM-bitcode-emission-for-backend-sourc.patch (5.2K, 7-v7-0006-meson-Add-LLVM-bitcode-emission-for-backend-sourc.patch) download | inline diff: From 0bf3e5ff87631940642fed0f71b89b547b83d642 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Wed, 12 Mar 2025 10:44:46 +0300 Subject: [PATCH v7 6/6] meson: Add LLVM bitcode emission for backend sources Since generated backend sources may have their own compilation flags and must also be included in the postgres.index.bc, the way to make it work with current code was to create a new variable, called `bc_generated_backend_sources`, which is a list of dictionaries, each one having an optional 'additional_flags' and a `srclist` pointing to the list of custom_target generated sources. An example of a possible structure of bitcode_modules which is processed by the main meson llvm bitcode emission file src/backend/jit/llvm/bitcode/meson.build: ``` bitcode_modules = [ { 'name': 'postgres', 'target': postgres_lib, 'src_file': backend_sources, 'gen_srcfiles': [ { 'additional_flags': [ '-I/path/postgresl/src/backend/parser', '-I/path/postgresl/build/src/backend/parser', ], 'srcfiles': [ <custom_target for scan.c>, <custom_target for gram.c> ] } ] } ] ``` Author: Diego Fronza <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/backend/bootstrap/meson.build | 2 ++ src/backend/meson.build | 2 ++ src/backend/parser/meson.build | 8 ++++++++ src/backend/replication/meson.build | 4 ++++ src/backend/utils/fmgr/meson.build | 1 + 5 files changed, 17 insertions(+) diff --git a/src/backend/bootstrap/meson.build b/src/backend/bootstrap/meson.build index 29726c1ab4f..389c75f8081 100644 --- a/src/backend/bootstrap/meson.build +++ b/src/backend/bootstrap/meson.build @@ -13,6 +13,7 @@ bootscanner = custom_target('bootscanner', ) generated_sources += bootscanner boot_parser_sources += bootscanner +bc_generated_backend_sources += {'srcfiles': [bootscanner]} bootparse = custom_target('bootparse', input: 'bootparse.y', @@ -20,6 +21,7 @@ bootparse = custom_target('bootparse', ) generated_sources += bootparse.to_list() boot_parser_sources += bootparse +bc_generated_backend_sources += {'srcfiles': [bootparse[0]]} boot_parser = static_library('boot_parser', boot_parser_sources, diff --git a/src/backend/meson.build b/src/backend/meson.build index 276ecbd6d74..aec0276f308 100644 --- a/src/backend/meson.build +++ b/src/backend/meson.build @@ -5,6 +5,7 @@ backend_sources = [] backend_link_with = [pgport_srv, common_srv] generated_backend_sources = [] +bc_generated_backend_sources = [] post_export_backend_sources = [] subdir('access') @@ -144,6 +145,7 @@ bitcode_modules += { 'name': 'postgres', 'target': postgres_lib, 'srcfiles': backend_sources, + 'gen_sources': bc_generated_backend_sources, } pg_mod_c_args = cflags_mod diff --git a/src/backend/parser/meson.build b/src/backend/parser/meson.build index 874aa749aa6..add472a0cd8 100644 --- a/src/backend/parser/meson.build +++ b/src/backend/parser/meson.build @@ -42,6 +42,14 @@ backend_parser = custom_target('gram', generated_sources += backend_parser.to_list() parser_sources += backend_parser +bc_generated_backend_sources += { + 'additional_flags': [ + '-I@0@'.format(meson.current_build_dir()), + '-I@0@'.format(meson.current_source_dir()), + ], + 'srcfiles': [backend_scanner, backend_parser[0]], +} + parser = static_library('parser', parser_sources, dependencies: [backend_code], diff --git a/src/backend/replication/meson.build b/src/backend/replication/meson.build index b0601498865..71ab1164960 100644 --- a/src/backend/replication/meson.build +++ b/src/backend/replication/meson.build @@ -19,6 +19,7 @@ repl_scanner = custom_target('repl_scanner', ) generated_sources += repl_scanner repl_parser_sources += repl_scanner +bc_generated_backend_sources += {'srcfiles': [repl_scanner]} repl_gram = custom_target('repl_gram', input: 'repl_gram.y', @@ -26,6 +27,7 @@ repl_gram = custom_target('repl_gram', ) generated_sources += repl_gram.to_list() repl_parser_sources += repl_gram +bc_generated_backend_sources += {'srcfiles': [repl_gram[0]]} syncrep_scanner = custom_target('syncrep_scanner', input: 'syncrep_scanner.l', @@ -34,6 +36,7 @@ syncrep_scanner = custom_target('syncrep_scanner', ) generated_sources += syncrep_scanner repl_parser_sources += syncrep_scanner +bc_generated_backend_sources += {'srcfiles': [syncrep_scanner]} syncrep_gram = custom_target('syncrep_gram', input: 'syncrep_gram.y', @@ -41,6 +44,7 @@ syncrep_gram = custom_target('syncrep_gram', ) generated_sources += syncrep_gram.to_list() repl_parser_sources += syncrep_gram +bc_generated_backend_sources += {'srcfiles': [syncrep_gram[0]]} repl_parser = static_library('repl_parser', repl_parser_sources, diff --git a/src/backend/utils/fmgr/meson.build b/src/backend/utils/fmgr/meson.build index b1dcab93e70..080f59988e9 100644 --- a/src/backend/utils/fmgr/meson.build +++ b/src/backend/utils/fmgr/meson.build @@ -8,3 +8,4 @@ backend_sources += files( # fmgrtab.c generated_backend_sources += fmgrtab_target[2] +bc_generated_backend_sources += {'srcfiles': [fmgrtab_target[2]]} -- 2.50.1 ^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: meson vs. llvm bitcode files @ 2025-10-31 12:13 Nazir Bilal Yavuz <[email protected]> parent: Nazir Bilal Yavuz <[email protected]> 0 siblings, 0 replies; 10+ messages in thread From: Nazir Bilal Yavuz @ 2025-10-31 12:13 UTC (permalink / raw) To: Andres Freund <[email protected]>; +Cc: Peter Eisentraut <[email protected]>; pgsql-hackers Hi, On Wed, 13 Aug 2025 at 16:25, Nazir Bilal Yavuz <[email protected]> wrote: > > Hi, > > On Mon, 7 Jul 2025 at 11:45, Nazir Bilal Yavuz <[email protected]> wrote: > > > > Mandatory rebase, v6 is attached. > > Rebase is needed due to 01d6832c10, v7 is attached. Rebase is needed due to 16607718c0, v8 is attached. -- Regards, Nazir Bilal Yavuz Microsoft Attachments: [text/x-patch] v8-0001-meson-Add-postgresql-extension.pc-for-building-ex.patch (4.5K, 2-v8-0001-meson-Add-postgresql-extension.pc-for-building-ex.patch) download | inline diff: From d4d5eb326b0c5abe4f2ef2596258d3b6c7d8497b Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Sat, 27 Aug 2022 09:52:03 -0700 Subject: [PATCH v8 1/6] meson: Add postgresql-extension.pc for building extension libraries This should work with several other buildsystems. TODO: Docs Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/backend/meson.build | 110 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 110 insertions(+) diff --git a/src/backend/meson.build b/src/backend/meson.build index b831a541652..dd903591e34 100644 --- a/src/backend/meson.build +++ b/src/backend/meson.build @@ -188,6 +188,116 @@ pg_test_mod_args = pg_mod_args + { +############################################################### +# Define a .pc file that can be used to build server extensions +############################################################### + +pg_ext_vars = [] +pg_ext_vars_inst = [] +pg_ext_vars_uninst = [] + +pg_ext_cflags = pg_mod_c_args + cppflags +pg_ext_libs = [backend_mod_deps, thread_dep, ldflags, ldflags_mod] +pg_ext_subdirs = [''] + +# Compute directories to add include directories to the .pc files for. +# This is a bit more complicated due to port/win32 etc. +i = 0 +foreach incdir : postgres_inc_d + if fs.is_absolute(incdir) + # an absolute path from -Dextra_include_dirs + pg_ext_cflags += '-I@0@'.format(incdir) + continue + elif incdir.startswith('src/include') + subincdir = dir_include_pkg_rel / 'server' / incdir.split('src/include/').get(1, '') + else + subincdir = '' + endif + pg_ext_subdirs += subincdir + + # Add directories in source / build dir containing headers to cflags for the + # -uninstalled.pc. Older versions of pkg-config complain if a referenced + # variable is not defined, so we emit an empty one for the installed .pc + # file. + pg_ext_vars += [ + 'build_inc@0@=""'.format(i), + 'src_inc@0@=""'.format(i), + ] + pg_ext_vars_uninst += [ + 'build_inc@0@=-I${prefix}/@1@'.format(i, incdir), + 'src_inc@0@=-I${srcdir}/@1@'.format(i, incdir), + ] + pg_ext_cflags += [ + '${build_inc@0@}'.format(i), + '${src_inc@0@}'.format(i) + ] + + i += 1 +endforeach + + +# Extension modules should likely also use -fwrapv etc. But it it's a bit odd +# to expose it to a .pc file? +pg_ext_cflags_warn = pg_ext_cflags + cflags_warn +pg_ext_cflags += cflags + +# Directories for extensions to install into +# XXX: more needed +pg_ext_vars += 'pkglibdir=${prefix}/@0@'.format(dir_lib_pkg) +pg_ext_vars += 'dir_mod=${pkglibdir}' +pg_ext_vars += 'dir_data=${prefix}/@0@'.format(dir_data_extension) +pg_ext_vars += 'dir_include=${prefix}/@0@'.format(dir_include_extension) +pg_ext_vars += 'dir_doc=${prefix}/@0@'.format(dir_doc_extension) +pg_ext_vars += 'dir_bitcode=${prefix}/@0@'.format(dir_bitcode) +# referenced on some platforms, via mod_link_with_dir +pg_ext_vars += 'bindir=${prefix}/@0@'.format(dir_bin) + +# XXX: Define variables making it easy to define tests, too + +# Some platforms need linker flags to link with binary, they are the same +# between building with meson and .pc file, except that we have have to +# reference a variable to make it work for both normal and -uninstalled .pc +# files. +if mod_link_args_fmt.length() != 0 + assert(link_with_inst != '') + assert(link_with_uninst != '') + + pg_ext_vars_inst += 'mod_link_with=@0@'.format(link_with_inst) + pg_ext_vars_uninst += 'mod_link_with=@0@'.format(link_with_uninst) + + foreach el : mod_link_args_fmt + pg_ext_libs += el.format('${mod_link_with}') + endforeach +endif + +# main .pc to build extensions +pkgconfig.generate( + name: 'postgresql-extension', + description: 'PostgreSQL Extension Support', + url: pg_url, + + subdirs: pg_ext_subdirs, + libraries: pg_ext_libs, + extra_cflags: pg_ext_cflags, + + variables: pg_ext_vars + pg_ext_vars_inst, + uninstalled_variables: pg_ext_vars + pg_ext_vars_uninst, +) + +# a .pc depending on the above, but with all our warnings enabled +pkgconfig.generate( + name: 'postgresql-extension-warnings', + description: 'PostgreSQL Extension Support - Compiler Warnings', + requires: 'postgresql-extension', + url: pg_url, + extra_cflags: pg_ext_cflags_warn, + + variables: pg_ext_vars + pg_ext_vars_inst, + uninstalled_variables: pg_ext_vars + pg_ext_vars_uninst, +) + + + # Shared modules that, on some system, link against the server binary. Only # enter these after we defined the server build. -- 2.51.0 [text/x-patch] v8-0002-meson-Test-building-extensions-by-using-postgresq.patch (10.7K, 3-v8-0002-meson-Test-building-extensions-by-using-postgresq.patch) download | inline diff: From ca2a2a30ea935ad58cc58b56faef43c4fd827544 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Thu, 27 Feb 2025 17:45:31 +0300 Subject: [PATCH v8 2/6] meson: Test building extensions by using postgresql-extension.pc The 'test_meson_extensions' pyton wrapper is added to run these tests. It compiles and builds extensions at ${build}/testrun/meson_extensions/${extension_name} path. The tests for building amcheck, auth_delay and postgres_fdw extensions are added. These are also examples of how to build extensions by using postgresql-extension.pc. Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/test/modules/meson.build | 1 + .../modules/test_meson_extensions/meson.build | 3 + .../amcheck/meson.build | 28 ++++++++ .../auth_delay/meson.build | 17 +++++ .../test_pkg_config_extensions/meson.build | 24 +++++++ .../postgres_fdw/meson.build | 30 ++++++++ meson.build | 32 +++++++++ src/tools/ci/test_meson_extensions | 70 +++++++++++++++++++ 8 files changed, 205 insertions(+) create mode 100644 src/test/modules/test_meson_extensions/meson.build create mode 100644 src/test/modules/test_meson_extensions/test_pkg_config_extensions/amcheck/meson.build create mode 100644 src/test/modules/test_meson_extensions/test_pkg_config_extensions/auth_delay/meson.build create mode 100644 src/test/modules/test_meson_extensions/test_pkg_config_extensions/meson.build create mode 100644 src/test/modules/test_meson_extensions/test_pkg_config_extensions/postgres_fdw/meson.build create mode 100644 src/tools/ci/test_meson_extensions diff --git a/src/test/modules/meson.build b/src/test/modules/meson.build index 14fc761c4cf..5c8049aab1f 100644 --- a/src/test/modules/meson.build +++ b/src/test/modules/meson.build @@ -30,6 +30,7 @@ subdir('test_integerset') subdir('test_json_parser') subdir('test_lfind') subdir('test_lwlock_tranches') +subdir('test_meson_extensions') subdir('test_misc') subdir('test_oat_hooks') subdir('test_parser') diff --git a/src/test/modules/test_meson_extensions/meson.build b/src/test/modules/test_meson_extensions/meson.build new file mode 100644 index 00000000000..06cf5d555df --- /dev/null +++ b/src/test/modules/test_meson_extensions/meson.build @@ -0,0 +1,3 @@ +# Copyright (c) 2022-2025, PostgreSQL Global Development Group + +subdir('test_pkg_config_extensions') diff --git a/src/test/modules/test_meson_extensions/test_pkg_config_extensions/amcheck/meson.build b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/amcheck/meson.build new file mode 100644 index 00000000000..482a543eb86 --- /dev/null +++ b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/amcheck/meson.build @@ -0,0 +1,28 @@ +# Copyright (c) 2022-2025, PostgreSQL Global Development Group + +project('amcheck', 'c') + +amcheck_path = '../../../../../../contrib/amcheck/' + +amcheck_sources = files( + amcheck_path / 'verify_heapam.c', + amcheck_path / 'verify_nbtree.c', +) + +pg_ext = dependency('postgresql-extension-warnings') + +amcheck = shared_module('amcheck', + amcheck_sources, + dependencies: pg_ext, + install_dir: pg_ext.get_variable(pkgconfig: 'dir_mod'), +) + +install_data( + amcheck_path / 'amcheck.control', + amcheck_path / 'amcheck--1.0.sql', + amcheck_path / 'amcheck--1.0--1.1.sql', + amcheck_path / 'amcheck--1.1--1.2.sql', + amcheck_path / 'amcheck--1.2--1.3.sql', + amcheck_path / 'amcheck--1.3--1.4.sql', + install_dir: pg_ext.get_variable(pkgconfig: 'dir_data'), +) diff --git a/src/test/modules/test_meson_extensions/test_pkg_config_extensions/auth_delay/meson.build b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/auth_delay/meson.build new file mode 100644 index 00000000000..98ad24cc183 --- /dev/null +++ b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/auth_delay/meson.build @@ -0,0 +1,17 @@ +# Copyright (c) 2022-2025, PostgreSQL Global Development Group + +project('auth_delay', 'c') + +auth_delay_path = '../../../../../../contrib/auth_delay/' + +auth_delay_sources = files( + auth_delay_path / 'auth_delay.c', +) + +pg_ext = dependency('postgresql-extension-warnings') + +auth_delay = shared_module('auth_delay', + auth_delay_sources, + dependencies: pg_ext, + install_dir: pg_ext.get_variable(pkgconfig: 'dir_mod'), +) diff --git a/src/test/modules/test_meson_extensions/test_pkg_config_extensions/meson.build b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/meson.build new file mode 100644 index 00000000000..dae94a384a1 --- /dev/null +++ b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/meson.build @@ -0,0 +1,24 @@ +# Copyright (c) 2022-2025, PostgreSQL Global Development Group + +# pkgconfig is not available on Windows, so skip it. +if host_machine.system() == 'windows' + subdir_done() +endif + +meson_extension_tests += { + 'name': 'amcheck', + 'kind': 'pkg_config', + 'sd': meson.current_source_dir() / 'amcheck', +} + +meson_extension_tests += { + 'name': 'auth_delay', + 'kind': 'pkg_config', + 'sd': meson.current_source_dir() / 'auth_delay', +} + +meson_extension_tests += { + 'name': 'postgres_fdw', + 'kind': 'pkg_config', + 'sd': meson.current_source_dir() / 'postgres_fdw', +} diff --git a/src/test/modules/test_meson_extensions/test_pkg_config_extensions/postgres_fdw/meson.build b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/postgres_fdw/meson.build new file mode 100644 index 00000000000..6d561e3789b --- /dev/null +++ b/src/test/modules/test_meson_extensions/test_pkg_config_extensions/postgres_fdw/meson.build @@ -0,0 +1,30 @@ +# Copyright (c) 2022-2025, PostgreSQL Global Development Group + +project('auth_delay', 'c') + +postgres_fdw_path = '../../../../../../contrib/postgres_fdw/' + +postgres_fdw_sources = files( + postgres_fdw_path / 'connection.c', + postgres_fdw_path / 'deparse.c', + postgres_fdw_path / 'option.c', + postgres_fdw_path / 'postgres_fdw.c', + postgres_fdw_path / 'shippable.c', +) + +pg_ext = dependency('postgresql-extension-warnings') +libpq = dependency('libpq') + +postgres_fdw = shared_module('postgres_fdw', + postgres_fdw_sources, + dependencies: [pg_ext, libpq], + install_dir: pg_ext.get_variable(pkgconfig: 'dir_mod'), +) + +install_data( + postgres_fdw_path / 'postgres_fdw.control', + postgres_fdw_path / 'postgres_fdw--1.0.sql', + postgres_fdw_path / 'postgres_fdw--1.0--1.1.sql', + postgres_fdw_path / 'postgres_fdw--1.1--1.2.sql', + install_dir: pg_ext.get_variable(pkgconfig: 'dir_data'), +) diff --git a/meson.build b/meson.build index 0f61ff6a700..6011eba8c4d 100644 --- a/meson.build +++ b/meson.build @@ -3034,6 +3034,7 @@ nls_targets = [] # Define the tests to distribute them to the correct test styles later test_deps = [] tests = [] +meson_extension_tests = [] # Default options for targets @@ -3597,6 +3598,37 @@ sys.exit(sp.returncode) suite: ['setup']) +# it seems freebsd doesn't use libdir for pkgconfig path +if host_system == 'freebsd' + pkgconf_installdir = dir_prefix / 'libdata' / 'pkgconfig' +else + pkgconf_installdir = dir_prefix / dir_lib / 'pkgconfig' +endif +test_pkg_conf_file = files('src/tools/ci/test_meson_extensions') + +foreach test : meson_extension_tests + if test['kind'] not in ['pkg_config'] + error('unknown kind @0@ of test in @1@'.format(test['kind'], test['sd'])) + endif + + test_group = 'meson_@0@_extensions'.format(test['kind']) + + test(test_group / test['name'], + test_pkg_conf_file, + args: [ + '--meson', meson_bin.path(), + '--meson_args', meson_args, + '--test_dir', test['sd'], + '--test_out_dir', test_result_dir / 'meson_extensions' / test['name'], + '--builddir', meson.build_root(), + '--pkg_conf_path', get_option('pkg_config_path'), + '--', + cc.cmd_array(), + ], + suite: test_group, + ) + +endforeach ############################################################### # Test Generation diff --git a/src/tools/ci/test_meson_extensions b/src/tools/ci/test_meson_extensions new file mode 100644 index 00000000000..50358121f49 --- /dev/null +++ b/src/tools/ci/test_meson_extensions @@ -0,0 +1,70 @@ +#!/usr/bin/env python3 + +import argparse +import os +import shutil +import subprocess + +parser = argparse.ArgumentParser() + +parser.add_argument('--meson', help='path to meson binary', + type=str, required=True) +parser.add_argument('--meson_args', help='args of meson binary', + type=str, nargs='*', required=False) +parser.add_argument('--test_dir', help='test source directory', + type=str, required=True) +parser.add_argument('--test_out_dir', help='test output directory', + type=str, required=True) +parser.add_argument('--builddir', help='meson build directory', + type=str, required=True) +parser.add_argument('--pkg_conf_path', + help='PKG_CONF_PATH from surrounding meson build', + type=str, nargs='?', const='', required=False) +parser.add_argument('c_args', help='c_args from surrounding meson build', + nargs='*') + +args = parser.parse_args() + +meson_bin = args.meson +meson_args = ' '.join(args.meson_args) +test_source_dir = args.test_dir +test_out_dir = args.test_out_dir +build_dir = args.builddir +pkg_conf_path = args.pkg_conf_path +c_args = ' '.join(args.c_args) + +exit_code = 0 + +def remove_duplicates(duplicate_str): + words = duplicate_str.split() + return ' '.join(sorted(set(words), key=words.index)) + + +def run_tests(pkg_conf_path_local, message): + print('\n{}\n{}\n'.format('#' * 60, message), flush=True) + + env = {**os.environ, } + env['PKG_CONFIG_PATH'] = '{}:{}:{}'.format( + pkg_conf_path_local, pkg_conf_path, env.get('PKG_CONFIG_PATH', ''), + ).strip(': ') + env['CC'] = '{} {}'.format( + c_args, env.get('CC', ''), + ) + env['CC'] = remove_duplicates(env['CC']) + + # Clear the build directory beforehand. + if os.path.exists(test_out_dir): + shutil.rmtree(test_out_dir) + + if meson_args: + meson_setup_command = [meson_bin, meson_args, 'setup', test_out_dir] + else: + meson_setup_command = [meson_bin, 'setup', test_out_dir] + + meson_compile_command = ['meson', 'compile', '-C', test_out_dir, '-v'] + + subprocess.run(meson_setup_command, env=env, cwd=test_source_dir, check=True) + subprocess.run(meson_compile_command, cwd=test_source_dir, check=True) + +run_tests(os.path.join(build_dir, 'meson-uninstalled'), + message='Testing postgresql-extension-warnings-uninstalled') -- 2.51.0 [text/x-patch] v8-0003-meson-WIP-Add-docs-for-postgresql-extension.pc.patch (12.3K, 4-v8-0003-meson-WIP-Add-docs-for-postgresql-extension.pc.patch) download | inline diff: From eef59f6917b5af5ed876eca40837b5c6642b025d Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Thu, 6 Mar 2025 17:46:57 +0300 Subject: [PATCH v8 3/6] meson: [WIP] Add docs for postgresql-extension.pc Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- doc/src/sgml/acronyms.sgml | 2 +- doc/src/sgml/extend.sgml | 101 +++++++++++++++++++++++-------------- doc/src/sgml/jit.sgml | 2 +- 3 files changed, 66 insertions(+), 39 deletions(-) diff --git a/doc/src/sgml/acronyms.sgml b/doc/src/sgml/acronyms.sgml index 2f906e9f018..57f49c06a19 100644 --- a/doc/src/sgml/acronyms.sgml +++ b/doc/src/sgml/acronyms.sgml @@ -579,7 +579,7 @@ <term><acronym>PGXS</acronym></term> <listitem> <para> - <link linkend="extend-pgxs"><productname>PostgreSQL</productname> Extension System</link> + <link linkend="extend-postgres-pgxs"><productname>PostgreSQL</productname> Extension System</link> </para> </listitem> </varlistentry> diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml index 63c5ec6d1eb..6d9fe5e96ad 100644 --- a/doc/src/sgml/extend.sgml +++ b/doc/src/sgml/extend.sgml @@ -1426,7 +1426,7 @@ include $(PGXS) </programlisting> This makefile relies on <acronym>PGXS</acronym>, which is described - in <xref linkend="extend-pgxs"/>. The command <literal>make install</literal> + in <xref linkend="extend-postgres-pgxs"/>. The command <literal>make install</literal> will install the control and script files into the correct directory as reported by <application>pg_config</application>. </para> @@ -1439,21 +1439,26 @@ include $(PGXS) </sect2> </sect1> - <sect1 id="extend-pgxs"> + <sect1 id="extend-postgres"> <title>Extension Building Infrastructure</title> - <indexterm zone="extend-pgxs"> - <primary>pgxs</primary> - </indexterm> - <para> If you are thinking about distributing your <productname>PostgreSQL</productname> extension modules, setting up a portable build system for them can be fairly difficult. Therefore the <productname>PostgreSQL</productname> installation provides a build - infrastructure for extensions, called <acronym>PGXS</acronym>, so - that simple extension modules can be built simply against an - already installed server. <acronym>PGXS</acronym> is mainly intended + infrastructure for extensions, called <literal>PGXS</literal> + (<xref linkend="extend-postgres-pgxs"/>) and + its meson counterpart <literal>postgresql-extension.pc</literal> + (<xref linkend="extend-postgres-meson"/>). + </para> + + </sect1> + + <sect1 id="extend-postgres-pgxs"> + <title>PGXS</title> + + <para> <acronym>PGXS</acronym> is mainly intended for extensions that include C code, although it can be used for pure-SQL extensions too. Note that <acronym>PGXS</acronym> is not intended to be a universal build system framework that can be used @@ -1493,7 +1498,7 @@ include $(PGXS) Set one of these three variables to specify what is built: <variablelist> - <varlistentry id="extend-pgxs-modules"> + <varlistentry id="extend-postgres-pgxs-modules"> <term><varname>MODULES</varname></term> <listitem> <para> @@ -1503,7 +1508,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-module-big"> + <varlistentry id="extend-postgres-pgxs-module-big"> <term><varname>MODULE_big</varname></term> <listitem> <para> @@ -1513,7 +1518,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-program"> + <varlistentry id="extend-postgres-pgxs-program"> <term><varname>PROGRAM</varname></term> <listitem> <para> @@ -1527,7 +1532,7 @@ include $(PGXS) The following variables can also be set: <variablelist> - <varlistentry id="extend-pgxs-extension"> + <varlistentry id="extend-postgres-pgxs-extension"> <term><varname>EXTENSION</varname></term> <listitem> <para> @@ -1539,7 +1544,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-moduledir"> + <varlistentry id="extend-postgres-pgxs-moduledir"> <term><varname>MODULEDIR</varname></term> <listitem> <para> @@ -1552,7 +1557,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-data"> + <varlistentry id="extend-postgres-pgxs-data"> <term><varname>DATA</varname></term> <listitem> <para> @@ -1561,7 +1566,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-data-built"> + <varlistentry id="extend-postgres-pgxs-data-built"> <term><varname>DATA_built</varname></term> <listitem> <para> @@ -1572,7 +1577,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-data-tsearch"> + <varlistentry id="extend-postgres-pgxs-data-tsearch"> <term><varname>DATA_TSEARCH</varname></term> <listitem> <para> @@ -1582,7 +1587,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-docs"> + <varlistentry id="extend-postgres-pgxs-docs"> <term><varname>DOCS</varname></term> <listitem> <para> @@ -1592,7 +1597,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-headers"> + <varlistentry id="extend-postgres-pgxs-headers"> <term><varname>HEADERS</varname></term> <term><varname>HEADERS_built</varname></term> <listitem> @@ -1608,7 +1613,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-headers-module"> + <varlistentry id="extend-postgres-pgxs-headers-module"> <term><varname>HEADERS_$MODULE</varname></term> <term><varname>HEADERS_built_$MODULE</varname></term> <listitem> @@ -1634,7 +1639,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-scripts"> + <varlistentry id="extend-postgres-pgxs-scripts"> <term><varname>SCRIPTS</varname></term> <listitem> <para> @@ -1644,7 +1649,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-scripts-built"> + <varlistentry id="extend-postgres-pgxs-scripts-built"> <term><varname>SCRIPTS_built</varname></term> <listitem> <para> @@ -1655,7 +1660,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-regress"> + <varlistentry id="extend-postgres-pgxs-regress"> <term><varname>REGRESS</varname></term> <listitem> <para> @@ -1664,7 +1669,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-regress-opts"> + <varlistentry id="extend-postgres-pgxs-regress-opts"> <term><varname>REGRESS_OPTS</varname></term> <listitem> <para> @@ -1673,7 +1678,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-isolation"> + <varlistentry id="extend-postgres-pgxs-isolation"> <term><varname>ISOLATION</varname></term> <listitem> <para> @@ -1682,7 +1687,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-isolation-opts"> + <varlistentry id="extend-postgres-pgxs-isolation-opts"> <term><varname>ISOLATION_OPTS</varname></term> <listitem> <para> @@ -1692,7 +1697,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-tap-tests"> + <varlistentry id="extend-postgres-pgxs-tap-tests"> <term><varname>TAP_TESTS</varname></term> <listitem> <para> @@ -1701,7 +1706,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-no-install"> + <varlistentry id="extend-postgres-pgxs-no-install"> <term><varname>NO_INSTALL</varname></term> <listitem> <para> @@ -1711,7 +1716,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-no-installcheck"> + <varlistentry id="extend-postgres-pgxs-no-installcheck"> <term><varname>NO_INSTALLCHECK</varname></term> <listitem> <para> @@ -1720,7 +1725,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-extra-clean"> + <varlistentry id="extend-postgres-pgxs-extra-clean"> <term><varname>EXTRA_CLEAN</varname></term> <listitem> <para> @@ -1729,7 +1734,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-cppflags"> + <varlistentry id="extend-postgres-pgxs-pg-cppflags"> <term><varname>PG_CPPFLAGS</varname></term> <listitem> <para> @@ -1738,7 +1743,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-cflags"> + <varlistentry id="extend-postgres-pgxs-pg-cflags"> <term><varname>PG_CFLAGS</varname></term> <listitem> <para> @@ -1747,7 +1752,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-cxxflags"> + <varlistentry id="extend-postgres-pgxs-pg-cxxflags"> <term><varname>PG_CXXFLAGS</varname></term> <listitem> <para> @@ -1756,7 +1761,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-ldflags"> + <varlistentry id="extend-postgres-pgxs-pg-ldflags"> <term><varname>PG_LDFLAGS</varname></term> <listitem> <para> @@ -1765,7 +1770,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-libs"> + <varlistentry id="extend-postgres-pgxs-pg-libs"> <term><varname>PG_LIBS</varname></term> <listitem> <para> @@ -1774,7 +1779,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-shlib-link"> + <varlistentry id="extend-postgres-pgxs-shlib-link"> <term><varname>SHLIB_LINK</varname></term> <listitem> <para> @@ -1783,7 +1788,7 @@ include $(PGXS) </listitem> </varlistentry> - <varlistentry id="extend-pgxs-pg-config"> + <varlistentry id="extend-postgres-pgxs-pg-config"> <term><varname>PG_CONFIG</varname></term> <listitem> <para> @@ -1929,4 +1934,26 @@ make VPATH=/path/to/extension/source/tree install </tip> </sect1> + <sect1 id="extend-postgres-meson"> + <title>postgresql-extension.pc</title> + + <para> + When Postgres is built by using meson, it generates + <literal>postgresql-extension.pc</literal> pkg-config file. Extension + libraries can use this file like <literal>PGXS</literal> + (<xref linkend="extend-postgres-pgxs"/>). + + To use the <literal>postgresql-extension.pc</literal> infrastructure for + your extension, you must write a simple meson.build file. In the + meson.build file, you need to include the + <literal>postgresql-extension.pc</literal> pkg-config file. Here is an + example that builds an extension module named isbn_issn, consisting of a + shared library containing some C code, an extension control file, an SQL + script, an include file (only needed if other modules might need to access + the extension functions without going via SQL), and a documentation text + file: + </para> + + </sect1> + </chapter> diff --git a/doc/src/sgml/jit.sgml b/doc/src/sgml/jit.sgml index 44e18bf1a6f..81a4644a97d 100644 --- a/doc/src/sgml/jit.sgml +++ b/doc/src/sgml/jit.sgml @@ -223,7 +223,7 @@ SET of types <literal>C</literal> and <literal>internal</literal>, as well as operators based on such functions. To do so for functions in extensions, the definitions of those functions need to be made available. - When using <link linkend="extend-pgxs">PGXS</link> to build an extension + When using <link linkend="extend-postgres-pgxs">PGXS</link> to build an extension against a server that has been compiled with LLVM JIT support, the relevant files will be built and installed automatically. </para> -- 2.51.0 [text/x-patch] v8-0004-meson-Add-architecture-for-LLVM-bitcode-emission.patch (9.0K, 5-v8-0004-meson-Add-architecture-for-LLVM-bitcode-emission.patch) download | inline diff: From 338d6fb33620cd54cc40ee2c4b52d1d14fe8cfbf Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Fri, 7 Mar 2025 12:10:58 +0300 Subject: [PATCH v8 4/6] meson: Add architecture for LLVM bitcode emission This commit adds suport for bitcode emission for both normal and generated source files (processed by bison, flex, etc). These bitcode files are installed into $pkglibdir/bitcode/ directory if the LLVM is found. New variable `bitcode_modules` is introduced to generate bitcode files. All required information is gathered in this variable. Then, this variable is processed by the main meson LLVM bitcode emission scripts: src/backend/jit/llvm/bitcode/meson.build -> src/tools/irlink. An example of a possible structure of bitcode_modules is: ``` bitcode_modules = [ { 'name': '...', 'target': ..., 'srcfiles': [ '...', '...', ], 'additional_flags': [ '-I...', '-I...', ], 'gen_srcfiles': [ { 'srcfiles': [ <custom_target for ...>, <custom_target for ...>, ], 'additional_flags': [ '-I...', '-I...', ] } ] } ] ``` Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Author: Diego Fronza <[email protected]> Reviewed-by: Diego Fronza <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/backend/jit/llvm/bitcode/meson.build | 69 ++++++++++++++++++++++++ src/backend/jit/llvm/meson.build | 31 +++++++---- src/backend/meson.build | 6 +++ meson.build | 21 ++++++++ src/tools/irlink | 25 +++++++++ 5 files changed, 141 insertions(+), 11 deletions(-) create mode 100644 src/backend/jit/llvm/bitcode/meson.build create mode 100644 src/tools/irlink diff --git a/src/backend/jit/llvm/bitcode/meson.build b/src/backend/jit/llvm/bitcode/meson.build new file mode 100644 index 00000000000..546e4d8b898 --- /dev/null +++ b/src/backend/jit/llvm/bitcode/meson.build @@ -0,0 +1,69 @@ +# Copyright (c) 2022-2024, PostgreSQL Global Development Group +# +# emit LLVM bitcode for JIT inlining + +assert(llvm.found()) + +foreach bitcode_module : bitcode_modules + bitcode_targets = [] + bitcode_obj = bitcode_module['target'] + bitcode_cflags_local = bitcode_cflags + bitcode_module.get('additional_flags', []) + bitcode_name = bitcode_module.get('name', bitcode_obj.name()) + + foreach srcfile : bitcode_module['srcfiles'] + if meson.version().version_compare('>=0.59') + srcfilename = fs.parent(srcfile) / fs.name(srcfile) + else + srcfilename = '@0@'.format(srcfile) + endif + + targetname = '@0@_@[email protected]'.format( + bitcode_name, + srcfilename.underscorify(), + ) + bitcode_targets += custom_target( + targetname, + depends: [bitcode_obj], + input: [srcfile], + output: targetname, + command: [llvm_irgen_command, bitcode_cflags_local], + install: true, + install_dir: dir_bitcode, + ) + endforeach + + # Process generated sources, which may include custom compilation flags. + foreach gen_sources: bitcode_module.get('gen_sources', []) + bitcode_cflags_gen_local = bitcode_cflags_local + gen_sources.get('additional_flags', []) + + foreach srcfile: gen_sources['srcfiles'] + # Generated sources are stored in some folder under meson.build_root()/**, + # remove the build prefix from the string. + srcfilename = srcfile.full_path().split(meson.build_root() + '/')[1] + + targetname = '@0@_@[email protected]'.format( + bitcode_name, + srcfilename.underscorify(), + ) + bitcode_targets += custom_target( + targetname, + depends: [bitcode_obj], + input: [srcfile], + output: targetname, + command: [llvm_irgen_command, bitcode_cflags_gen_local], + install: true, + install_dir: dir_bitcode, + ) + endforeach + endforeach + + index_name = '@[email protected]'.format(bitcode_name) + bitcode_index = custom_target('@0@'.format(bitcode_name), + output: index_name, + input: bitcode_targets, + command: [irlink, '--lto', llvm_lto, '--outdir', '@OUTDIR@', '--index', index_name, '@INPUT@'], + install: true, + install_dir: dir_bitcode, + ) + backend_targets += bitcode_index +endforeach diff --git a/src/backend/jit/llvm/meson.build b/src/backend/jit/llvm/meson.build index 805fbd69006..2dd922e573b 100644 --- a/src/backend/jit/llvm/meson.build +++ b/src/backend/jit/llvm/meson.build @@ -42,21 +42,22 @@ backend_targets += llvmjit # Define a few bits and pieces used here and elsewhere to generate bitcode -llvm_irgen_args = [ - '-c', '-o', '@OUTPUT@', '@INPUT@', +llvm_irgen_command = [] +if ccache.found() + llvm_irgen_command += ccache +endif + +llvm_irgen_command += [ + clang, + '-c', '-o', '@OUTPUT0@', '@INPUT0@', '-flto=thin', '-emit-llvm', - '-MD', '-MQ', '@OUTPUT@', '-MF', '@DEPFILE@', '-O2', '-Wno-ignored-attributes', '-Wno-empty-body', + '-Wno-unknown-warning-option', + '-Wno-compound-token-split-by-macro', ] - -if ccache.found() - llvm_irgen_command = ccache - llvm_irgen_args = [clang.full_path()] + llvm_irgen_args -else - llvm_irgen_command = clang -endif +llvm_irgen_dep_args = ['-MD', '-MQ', '@OUTPUT0@', '-MF', '@DEPFILE@'] # XXX: Need to determine proper version of the function cflags for clang @@ -73,7 +74,7 @@ bitcode_cflags += '-I@SOURCE_ROOT@/src/include' # Note this is intentionally not installed to bitcodedir, as it's not for # inlining llvmjit_types = custom_target('llvmjit_types.bc', - command: [llvm_irgen_command] + llvm_irgen_args + bitcode_cflags, + command: llvm_irgen_command + llvm_irgen_dep_args + bitcode_cflags, input: 'llvmjit_types.c', output: 'llvmjit_types.bc', depends: [postgres], @@ -82,3 +83,11 @@ llvmjit_types = custom_target('llvmjit_types.bc', depfile: '@[email protected]', ) backend_targets += llvmjit_types + +# Figure out -I's needed to build all postgres code, including all its +# dependencies +pkg_config = find_program(['pkg-config', 'pkgconf'], required: true) +r = run_command(pkg_config, + ['--cflags-only-I', meson.build_root() / 'meson-uninstalled/postgresql-extension-uninstalled.pc'], + check: true) +bitcode_cflags += r.stdout().split() diff --git a/src/backend/meson.build b/src/backend/meson.build index dd903591e34..276ecbd6d74 100644 --- a/src/backend/meson.build +++ b/src/backend/meson.build @@ -140,6 +140,12 @@ postgres = executable('postgres', backend_targets += postgres +bitcode_modules += { + 'name': 'postgres', + 'target': postgres_lib, + 'srcfiles': backend_sources, +} + pg_mod_c_args = cflags_mod pg_mod_cpp_args = cxxflags_mod pg_mod_link_args = ldflags_sl + ldflags_mod diff --git a/meson.build b/meson.build index 6011eba8c4d..d5c326928b6 100644 --- a/meson.build +++ b/meson.build @@ -843,6 +843,8 @@ if add_languages('cpp', required: llvmopt, native: false) # Some distros put LLVM and clang in different paths, so fallback to # find via PATH, too. clang = find_program(llvm_binpath / 'clang', 'clang', required: true) + llvm_lto = find_program(llvm_binpath / 'llvm-lto', required: true) + irlink = find_program('src/tools/irlink', native: true) endif elif llvmopt.auto() message('llvm requires a C++ compiler') @@ -3036,6 +3038,11 @@ test_deps = [] tests = [] meson_extension_tests = [] +# List of object files + source files to generated LLVM IR for inlining. +# Each element is a hash of: +# {'target': target, 'srcfiles': ..., 'additional_flags': ...}. +bitcode_modules = [] + # Default options for targets @@ -3359,6 +3366,11 @@ subdir('src/interfaces/ecpg/test') subdir('doc/src/sgml') +# generate bitcode for JIT inlining after giving contrib modules etc a chance +# to add themselves to bitcode_modules[] +subdir('src/backend/jit/llvm/bitcode', if_found: llvm) + + generated_sources_ac += {'': ['GNUmakefile']} # After processing src/test, add test_install_libs to the testprep_targets @@ -3988,6 +4000,15 @@ summary( section: 'Programs', ) +if llvm.found() + summary( + { + 'clang': clang, + }, + section: 'Programs', + ) +endif + summary( { 'bonjour': bonjour, diff --git a/src/tools/irlink b/src/tools/irlink new file mode 100644 index 00000000000..793c0abf91a --- /dev/null +++ b/src/tools/irlink @@ -0,0 +1,25 @@ +#!/usr/bin/env python3 + +import argparse +import os +import shutil +import subprocess +import sys + +parser = argparse.ArgumentParser( + description='generate PostgreSQL JIT IR module') + +parser.add_argument('--index', type=str, required=True) +parser.add_argument('--lto', type=str, required=True) +parser.add_argument('--outdir', type=str, required=True) +parser.add_argument('INPUT', type=str, nargs='+') + +args = parser.parse_args() + +file_names = [os.path.basename(f) for f in args.INPUT] +command = [args.lto, + '-thinlto', '-thinlto-action=thinlink', + '-o', args.index] + file_names +res = subprocess.run(command, cwd=args.outdir) + +exit(res.returncode) -- 2.51.0 [text/x-patch] v8-0005-meson-Add-LLVM-bitcode-emissions-for-contrib-libr.patch (22.7K, 6-v8-0005-meson-Add-LLVM-bitcode-emissions-for-contrib-libr.patch) download | inline diff: From 2737c8a599871d260e4ef8af488399f6096e4bdd Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Wed, 12 Mar 2025 10:49:18 +0300 Subject: [PATCH v8 5/6] meson: Add LLVM bitcode emissions for contrib libraries The libraries which the bitcode files will be generated in are selected manually. Author: Andres Freund <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Author: Diego Fronza <[email protected]> Reviewed-by: Diego Fronza <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/interfaces/libpq/meson.build | 3 +++ contrib/bloom/meson.build | 5 +++++ contrib/bool_plperl/meson.build | 9 +++++++++ contrib/btree_gin/meson.build | 5 +++++ contrib/btree_gist/meson.build | 5 +++++ contrib/citext/meson.build | 5 +++++ contrib/cube/meson.build | 13 +++++++++++++ contrib/dict_int/meson.build | 5 +++++ contrib/dict_xsyn/meson.build | 5 +++++ contrib/earthdistance/meson.build | 6 ++++++ contrib/fuzzystrmatch/meson.build | 9 +++++++++ contrib/hstore/meson.build | 7 +++++++ contrib/hstore_plperl/meson.build | 10 ++++++++++ contrib/hstore_plpython/meson.build | 12 ++++++++++++ contrib/intarray/meson.build | 5 +++++ contrib/isn/meson.build | 5 +++++ contrib/jsonb_plperl/meson.build | 8 ++++++++ contrib/jsonb_plpython/meson.build | 10 ++++++++++ contrib/lo/meson.build | 5 +++++ contrib/ltree/meson.build | 10 ++++++++++ contrib/ltree_plpython/meson.build | 11 +++++++++++ contrib/pg_buffercache/meson.build | 5 +++++ contrib/pg_freespacemap/meson.build | 5 +++++ contrib/pg_logicalinspect/meson.build | 5 +++++ contrib/pg_surgery/meson.build | 4 ++++ contrib/pg_trgm/meson.build | 5 +++++ contrib/pgcrypto/meson.build | 4 ++++ contrib/seg/meson.build | 12 ++++++++++++ contrib/spi/meson.build | 20 ++++++++++++++++++++ contrib/sslinfo/meson.build | 5 +++++ contrib/tablefunc/meson.build | 5 +++++ contrib/tcn/meson.build | 5 +++++ contrib/tsm_system_rows/meson.build | 5 +++++ contrib/tsm_system_time/meson.build | 5 +++++ contrib/unaccent/meson.build | 5 +++++ contrib/uuid-ossp/meson.build | 5 +++++ contrib/xml2/meson.build | 5 +++++ src/pl/plperl/meson.build | 1 + src/pl/plpython/meson.build | 1 + meson.build | 1 + 40 files changed, 256 insertions(+) diff --git a/src/interfaces/libpq/meson.build b/src/interfaces/libpq/meson.build index a74e885b169..c3ceab1dd70 100644 --- a/src/interfaces/libpq/meson.build +++ b/src/interfaces/libpq/meson.build @@ -49,6 +49,9 @@ libpq_c_args = ['-DSO_MAJOR_VERSION=5'] # The OAuth implementation differs depending on the type of library being built. libpq_so_c_args = ['-DUSE_DYNAMIC_OAUTH'] +# libpq-fe.h is needed to generate bitcode for some extensions +libpq_dir = meson.current_source_dir() + # Not using both_libraries() here as # 1) resource files should only be in the shared library # 2) we want the .pc file to include a dependency to {pgport,common}_static for diff --git a/contrib/bloom/meson.build b/contrib/bloom/meson.build index 695712a455e..1090c281532 100644 --- a/contrib/bloom/meson.build +++ b/contrib/bloom/meson.build @@ -28,6 +28,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': bloom, + 'srcfiles': bloom_sources, +} + tests += { 'name': 'bloom', 'sd': meson.current_source_dir(), diff --git a/contrib/bool_plperl/meson.build b/contrib/bool_plperl/meson.build index f489d99044c..1758adb7489 100644 --- a/contrib/bool_plperl/meson.build +++ b/contrib/bool_plperl/meson.build @@ -37,6 +37,15 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': bool_plperl, + 'srcfiles': bool_plperl_sources, + 'additional_flags': [ + '-I@0@'.format(plperl_dir), + perl_ccflags + ] +} + tests += { 'name': 'bool_plperl', 'sd': meson.current_source_dir(), diff --git a/contrib/btree_gin/meson.build b/contrib/btree_gin/meson.build index ece0a716973..e92e52822b0 100644 --- a/contrib/btree_gin/meson.build +++ b/contrib/btree_gin/meson.build @@ -26,6 +26,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': btree_gin, + 'srcfiles': btree_gin_sources, +} + tests += { 'name': 'btree_gin', 'sd': meson.current_source_dir(), diff --git a/contrib/btree_gist/meson.build b/contrib/btree_gist/meson.build index f4fa9574f1f..2649d5a7ffb 100644 --- a/contrib/btree_gist/meson.build +++ b/contrib/btree_gist/meson.build @@ -54,6 +54,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': btree_gist, + 'srcfiles': btree_gist_sources, +} + tests += { 'name': 'btree_gist', 'sd': meson.current_source_dir(), diff --git a/contrib/citext/meson.build b/contrib/citext/meson.build index 7fff34f2368..fa07ff72be4 100644 --- a/contrib/citext/meson.build +++ b/contrib/citext/meson.build @@ -30,6 +30,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': citext, + 'srcfiles': citext_sources, +} + tests += { 'name': 'citext', 'sd': meson.current_source_dir(), diff --git a/contrib/cube/meson.build b/contrib/cube/meson.build index fd3c057f469..a7649a74a1f 100644 --- a/contrib/cube/meson.build +++ b/contrib/cube/meson.build @@ -3,6 +3,7 @@ cube_sources = files( 'cube.c', ) +bc_cube_sources = cube_sources cube_scan = custom_target('cubescan', input: 'cubescan.l', @@ -11,6 +12,7 @@ cube_scan = custom_target('cubescan', ) generated_sources += cube_scan cube_sources += cube_scan +bc_cube_gen_sources = [{'srcfiles': [cube_scan]}] cube_parse = custom_target('cubeparse', input: 'cubeparse.y', @@ -18,6 +20,7 @@ cube_parse = custom_target('cubeparse', ) generated_sources += cube_parse.to_list() cube_sources += cube_parse +bc_cube_gen_sources += {'srcfiles': [cube_parse[0]]} if host_system == 'windows' cube_sources += rc_lib_gen.process(win32ver_rc, extra_args: [ @@ -48,6 +51,16 @@ install_headers( install_dir: dir_include_extension / 'cube', ) +cube_dir = meson.current_source_dir() +bitcode_modules += { + 'target': cube, + 'srcfiles': bc_cube_sources, + 'gen_sources': bc_cube_gen_sources, + 'additional_flags': [ + '-I@0@'.format(cube_dir), + ] +} + tests += { 'name': 'cube', 'sd': meson.current_source_dir(), diff --git a/contrib/dict_int/meson.build b/contrib/dict_int/meson.build index ab41588547b..894c8a9c94d 100644 --- a/contrib/dict_int/meson.build +++ b/contrib/dict_int/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': dict_int, + 'srcfiles': dict_int_sources, +} + tests += { 'name': 'dict_int', 'sd': meson.current_source_dir(), diff --git a/contrib/dict_xsyn/meson.build b/contrib/dict_xsyn/meson.build index 93f41b1f963..d4bd4a6ef5a 100644 --- a/contrib/dict_xsyn/meson.build +++ b/contrib/dict_xsyn/meson.build @@ -29,6 +29,11 @@ install_data( } ) +bitcode_modules += { + 'target': dict_xsyn, + 'srcfiles': dict_xsyn_sources, +} + tests += { 'name': 'dict_xsyn', 'sd': meson.current_source_dir(), diff --git a/contrib/earthdistance/meson.build b/contrib/earthdistance/meson.build index a5f8d1a3ffa..c101359b118 100644 --- a/contrib/earthdistance/meson.build +++ b/contrib/earthdistance/meson.build @@ -24,6 +24,12 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': earthdistance, + 'srcfiles': earthdistance_sources, +} + + tests += { 'name': 'earthdistance', 'sd': meson.current_source_dir(), diff --git a/contrib/fuzzystrmatch/meson.build b/contrib/fuzzystrmatch/meson.build index f52daa4ea1c..fc5e88d0c92 100644 --- a/contrib/fuzzystrmatch/meson.build +++ b/contrib/fuzzystrmatch/meson.build @@ -5,6 +5,7 @@ fuzzystrmatch_sources = files( 'dmetaphone.c', 'fuzzystrmatch.c', ) +bc_fuzzystrmatch_sources = fuzzystrmatch_sources daitch_mokotoff_h = custom_target('daitch_mokotoff', input: 'daitch_mokotoff_header.pl', @@ -35,6 +36,14 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': fuzzystrmatch, + 'srcfiles': bc_fuzzystrmatch_sources, + 'additional_flags': [ + '-I@0@'.format(meson.current_build_dir()) + ] +} + tests += { 'name': 'fuzzystrmatch', 'sd': meson.current_source_dir(), diff --git a/contrib/hstore/meson.build b/contrib/hstore/meson.build index 39622d1024d..344a620273f 100644 --- a/contrib/hstore/meson.build +++ b/contrib/hstore/meson.build @@ -43,6 +43,13 @@ install_headers( install_dir: dir_include_extension / 'hstore', ) +# some libraries include "hstore/hstore.h" instead of "hstore.h" +hstore_dir_up = join_paths(meson.current_source_dir(), '..') +bitcode_modules += { + 'target': hstore, + 'srcfiles': hstore_sources, +} + tests += { 'name': 'hstore', 'sd': meson.current_source_dir(), diff --git a/contrib/hstore_plperl/meson.build b/contrib/hstore_plperl/meson.build index 60b8ad23569..b689b55e1c9 100644 --- a/contrib/hstore_plperl/meson.build +++ b/contrib/hstore_plperl/meson.build @@ -37,6 +37,16 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': hstore_plperl, + 'srcfiles': hstore_plperl_sources, + 'additional_flags': [ + '-I@0@'.format(hstore_dir_up), + '-I@0@'.format(plperl_dir), + perl_ccflags, + ] +} + tests += { 'name': 'hstore_plperl', 'sd': meson.current_source_dir(), diff --git a/contrib/hstore_plpython/meson.build b/contrib/hstore_plpython/meson.build index ea8e20a377b..d68de3dc46b 100644 --- a/contrib/hstore_plpython/meson.build +++ b/contrib/hstore_plpython/meson.build @@ -30,6 +30,18 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': hstore_plpython, + 'srcfiles': hstore_plpython_sources, + 'additional_flags': [ + '-I@0@'.format(hstore_dir_up), + '-DPLPYTHON_LIBNAME="plpython3"', + '-I@0@'.format(python3_inc_dir), + '-I@0@'.format(plpython_dir), + perl_ccflags, + ] +} + hstore_plpython_regress = [ 'hstore_plpython' ] diff --git a/contrib/intarray/meson.build b/contrib/intarray/meson.build index fae9add981d..3d7da0c9c9a 100644 --- a/contrib/intarray/meson.build +++ b/contrib/intarray/meson.build @@ -33,6 +33,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': intarray, + 'srcfiles': intarray_sources, +} + tests += { 'name': 'intarray', 'sd': meson.current_source_dir(), diff --git a/contrib/isn/meson.build b/contrib/isn/meson.build index 39cf781684e..71dc8cf1a76 100644 --- a/contrib/isn/meson.build +++ b/contrib/isn/meson.build @@ -30,6 +30,11 @@ install_headers( install_dir: dir_include_extension / 'isn', ) +bitcode_modules += { + 'target': isn, + 'srcfiles': isn_sources, +} + tests += { 'name': 'isn', 'sd': meson.current_source_dir(), diff --git a/contrib/jsonb_plperl/meson.build b/contrib/jsonb_plperl/meson.build index 95a9a7bc082..febafc73a99 100644 --- a/contrib/jsonb_plperl/meson.build +++ b/contrib/jsonb_plperl/meson.build @@ -37,6 +37,14 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': jsonb_plperl, + 'srcfiles': jsonb_plperl_sources, + 'additional_flags': [ + '-I@0@'.format(plperl_dir), + perl_ccflags, + ] +} tests += { 'name': 'jsonb_plperl', diff --git a/contrib/jsonb_plpython/meson.build b/contrib/jsonb_plpython/meson.build index 5fe80483e58..6be92221272 100644 --- a/contrib/jsonb_plpython/meson.build +++ b/contrib/jsonb_plpython/meson.build @@ -30,6 +30,16 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': jsonb_plpython, + 'srcfiles': jsonb_plpython_sources, + 'additional_flags': [ + '-DPLPYTHON_LIBNAME="plpython3"', + '-I@0@'.format(python3_inc_dir), + '-I@0@'.format(plpython_dir), + ] +} + jsonb_plpython_regress = [ 'jsonb_plpython' ] diff --git a/contrib/lo/meson.build b/contrib/lo/meson.build index 2d78907ba12..1222faa9169 100644 --- a/contrib/lo/meson.build +++ b/contrib/lo/meson.build @@ -24,6 +24,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': lo, + 'srcfiles': lo_sources, +} + tests += { 'name': 'lo', 'sd': meson.current_source_dir(), diff --git a/contrib/ltree/meson.build b/contrib/ltree/meson.build index f9b06302839..cda86935544 100644 --- a/contrib/ltree/meson.build +++ b/contrib/ltree/meson.build @@ -41,6 +41,16 @@ install_headers( install_dir: dir_include_extension / 'ltree', ) +ltree_dir = meson.current_source_dir() +ltree_dir_up = join_paths(ltree_dir, '..') +bitcode_modules += { + 'target': ltree, + 'srcfiles': ltree_sources, + 'additional_flags': [ + '-I@0@'.format(ltree_dir) + ] +} + tests += { 'name': 'ltree', 'sd': meson.current_source_dir(), diff --git a/contrib/ltree_plpython/meson.build b/contrib/ltree_plpython/meson.build index a37732c486b..74b285da05c 100644 --- a/contrib/ltree_plpython/meson.build +++ b/contrib/ltree_plpython/meson.build @@ -30,6 +30,17 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': ltree_plpython, + 'srcfiles': ltree_plpython_sources, + 'additional_flags': [ + '-I@0@'.format(ltree_dir_up), + '-DPLPYTHON_LIBNAME="plpython3"', + '-I@0@'.format(python3_inc_dir), + '-I@0@'.format(plpython_dir), + ] +} + ltree_plpython_regress = [ 'ltree_plpython' ] diff --git a/contrib/pg_buffercache/meson.build b/contrib/pg_buffercache/meson.build index 7cd039a1df9..65dcc690601 100644 --- a/contrib/pg_buffercache/meson.build +++ b/contrib/pg_buffercache/meson.build @@ -28,6 +28,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pg_buffercache, + 'srcfiles': pg_buffercache_sources, +} + tests += { 'name': 'pg_buffercache', 'sd': meson.current_source_dir(), diff --git a/contrib/pg_freespacemap/meson.build b/contrib/pg_freespacemap/meson.build index ff8eda3580e..120def1ad2e 100644 --- a/contrib/pg_freespacemap/meson.build +++ b/contrib/pg_freespacemap/meson.build @@ -25,6 +25,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pg_freespacemap, + 'srcfiles': pg_freespacemap_sources, +} + tests += { 'name': 'pg_freespacemap', 'sd': meson.current_source_dir(), diff --git a/contrib/pg_logicalinspect/meson.build b/contrib/pg_logicalinspect/meson.build index 5c0528a8c63..1b4f46277ed 100644 --- a/contrib/pg_logicalinspect/meson.build +++ b/contrib/pg_logicalinspect/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pg_logicalinspect, + 'srcfiles': pg_logicalinspect_sources, +} + tests += { 'name': 'pg_logicalinspect', 'sd': meson.current_source_dir(), diff --git a/contrib/pg_surgery/meson.build b/contrib/pg_surgery/meson.build index c6cfa9c4694..9d7199392c7 100644 --- a/contrib/pg_surgery/meson.build +++ b/contrib/pg_surgery/meson.build @@ -22,6 +22,10 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pg_surgery, + 'srcfiles': pg_surgery_sources, +} tests += { 'name': 'pg_surgery', diff --git a/contrib/pg_trgm/meson.build b/contrib/pg_trgm/meson.build index a31aa5c574d..408e287172e 100644 --- a/contrib/pg_trgm/meson.build +++ b/contrib/pg_trgm/meson.build @@ -32,6 +32,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pg_trgm, + 'srcfiles': pg_trgm_sources, +} + tests += { 'name': 'pg_trgm', 'sd': meson.current_source_dir(), diff --git a/contrib/pgcrypto/meson.build b/contrib/pgcrypto/meson.build index 7d5ef9b6d32..65d94174f7e 100644 --- a/contrib/pgcrypto/meson.build +++ b/contrib/pgcrypto/meson.build @@ -100,6 +100,10 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': pgcrypto, + 'srcfiles': pgcrypto_sources, +} tests += { 'name': 'pgcrypto', diff --git a/contrib/seg/meson.build b/contrib/seg/meson.build index e331e097230..371cd272a06 100644 --- a/contrib/seg/meson.build +++ b/contrib/seg/meson.build @@ -3,6 +3,7 @@ seg_sources = files( 'seg.c', ) +bc_seg_sources = seg_sources seg_scan = custom_target('segscan', input: 'segscan.l', @@ -11,6 +12,7 @@ seg_scan = custom_target('segscan', ) generated_sources += seg_scan seg_sources += seg_scan +bc_seg_gen_sources = [{'srcfiles': [seg_scan]}] seg_parse = custom_target('segparse', input: 'segparse.y', @@ -18,6 +20,7 @@ seg_parse = custom_target('segparse', ) generated_sources += seg_parse.to_list() seg_sources += seg_parse +bc_seg_gen_sources += {'srcfiles': [seg_parse[0]]} if host_system == 'windows' seg_sources += rc_lib_gen.process(win32ver_rc, extra_args: [ @@ -47,6 +50,15 @@ install_headers( install_dir: dir_include_extension / 'seg', ) +bitcode_modules += { + 'target': seg, + 'srcfiles': bc_seg_sources, + 'gen_sources': bc_seg_gen_sources, + 'additional_flags': [ + '-I@0@'.format(meson.current_source_dir()), + ] +} + tests += { 'name': 'seg', 'sd': meson.current_source_dir(), diff --git a/contrib/spi/meson.build b/contrib/spi/meson.build index 3832a91019a..c8216fcbcd0 100644 --- a/contrib/spi/meson.build +++ b/contrib/spi/meson.build @@ -24,6 +24,11 @@ install_data('autoinc.example', kwargs: contrib_doc_args, ) +bitcode_modules += { + 'target': autoinc, + 'srcfiles': autoinc_sources, +} + insert_username_sources = files( 'insert_username.c', @@ -51,6 +56,11 @@ install_data('insert_username.example', kwargs: contrib_doc_args, ) +bitcode_modules += { + 'target': insert_username, + 'srcfiles': insert_username_sources, +} + moddatetime_sources = files( 'moddatetime.c', @@ -78,6 +88,11 @@ install_data('moddatetime.example', kwargs: contrib_doc_args, ) +bitcode_modules += { + 'target': moddatetime, + 'srcfiles': moddatetime_sources, +} + # this is needed for the regression tests; # comment out if you want a quieter refint package for other uses @@ -108,6 +123,11 @@ install_data('refint.example', kwargs: contrib_doc_args, ) +bitcode_modules += { + 'target': refint, + 'srcfiles': refint_sources, +} + tests += { 'name': 'spi', 'sd': meson.current_source_dir(), diff --git a/contrib/sslinfo/meson.build b/contrib/sslinfo/meson.build index 4c513759200..5750d783b92 100644 --- a/contrib/sslinfo/meson.build +++ b/contrib/sslinfo/meson.build @@ -29,3 +29,8 @@ install_data( 'sslinfo.control', kwargs: contrib_data_args, ) + +bitcode_modules += { + 'target': sslinfo, + 'srcfiles': sslinfo_sources, +} diff --git a/contrib/tablefunc/meson.build b/contrib/tablefunc/meson.build index ee67272ec0a..6c26ffafddf 100644 --- a/contrib/tablefunc/meson.build +++ b/contrib/tablefunc/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': tablefunc, + 'srcfiles': tablefunc_sources, +} + tests += { 'name': 'tablefunc', 'sd': meson.current_source_dir(), diff --git a/contrib/tcn/meson.build b/contrib/tcn/meson.build index 6ffb136af90..df71e8a91f9 100644 --- a/contrib/tcn/meson.build +++ b/contrib/tcn/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': tcn, + 'srcfiles': tcn_sources, +} + tests += { 'name': 'tcn', 'sd': meson.current_source_dir(), diff --git a/contrib/tsm_system_rows/meson.build b/contrib/tsm_system_rows/meson.build index b8cece3d80f..1ea74cce949 100644 --- a/contrib/tsm_system_rows/meson.build +++ b/contrib/tsm_system_rows/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': tsm_system_rows, + 'srcfiles': tsm_system_rows_sources, +} + tests += { 'name': 'tsm_system_rows', 'sd': meson.current_source_dir(), diff --git a/contrib/tsm_system_time/meson.build b/contrib/tsm_system_time/meson.build index 8a143a8f8e6..1f6b98d3ec4 100644 --- a/contrib/tsm_system_time/meson.build +++ b/contrib/tsm_system_time/meson.build @@ -22,6 +22,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': tsm_system_time, + 'srcfiles': tsm_system_time_sources, +} + tests += { 'name': 'tsm_system_time', 'sd': meson.current_source_dir(), diff --git a/contrib/unaccent/meson.build b/contrib/unaccent/meson.build index 33d4649bae1..1f5d120e3d8 100644 --- a/contrib/unaccent/meson.build +++ b/contrib/unaccent/meson.build @@ -28,6 +28,11 @@ install_data( install_dir: dir_data / 'tsearch_data' ) +bitcode_modules += { + 'target': unaccent, + 'srcfiles': unaccent_sources, +} + # XXX: Implement downlo tests += { 'name': 'unaccent', diff --git a/contrib/uuid-ossp/meson.build b/contrib/uuid-ossp/meson.build index 982f27c085f..8d21bb0a996 100644 --- a/contrib/uuid-ossp/meson.build +++ b/contrib/uuid-ossp/meson.build @@ -29,6 +29,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': uuid_ossp, + 'srcfiles': uuid_ossp_sources, +} + tests += { 'name': 'uuid-ossp', 'sd': meson.current_source_dir(), diff --git a/contrib/xml2/meson.build b/contrib/xml2/meson.build index 08d3c3b8e30..b23a8fe0cd7 100644 --- a/contrib/xml2/meson.build +++ b/contrib/xml2/meson.build @@ -32,6 +32,11 @@ install_data( kwargs: contrib_data_args, ) +bitcode_modules += { + 'target': xml2, + 'srcfiles': xml2_sources, +} + tests += { 'name': 'xml2', 'sd': meson.current_source_dir(), diff --git a/src/pl/plperl/meson.build b/src/pl/plperl/meson.build index 7c4081c3460..06e853c1ac7 100644 --- a/src/pl/plperl/meson.build +++ b/src/pl/plperl/meson.build @@ -37,6 +37,7 @@ foreach n : ['SPI', 'Util'] plperl_sources += xs_c endforeach +plperl_dir = meson.current_source_dir() plperl_inc = include_directories('.') if host_system == 'windows' diff --git a/src/pl/plpython/meson.build b/src/pl/plpython/meson.build index 709e5932a93..e2c19771d85 100644 --- a/src/pl/plpython/meson.build +++ b/src/pl/plpython/meson.build @@ -29,6 +29,7 @@ plpython_sources += custom_target('spiexceptions.h', # FIXME: need to duplicate import library ugliness? plpython_inc = include_directories('.') +plpython_dir = meson.current_source_dir() if host_system == 'windows' plpython_sources += rc_lib_gen.process(win32ver_rc, extra_args: [ diff --git a/meson.build b/meson.build index d5c326928b6..7fdcbfde57c 100644 --- a/meson.build +++ b/meson.build @@ -1323,6 +1323,7 @@ if not pyopt.disabled() python3_inst = pm.find_installation(python.full_path(), required: pyopt) if python3_inst.found() python3_dep = python3_inst.dependency(embed: true, required: pyopt) + python3_inc_dir = python3_inst.get_variable('INCLUDEPY') # Remove this check after we depend on Meson >= 1.1.0 if not cc.check_header('Python.h', dependencies: python3_dep, required: pyopt, include_directories: postgres_inc) python3_dep = not_found_dep -- 2.51.0 [text/x-patch] v8-0006-meson-Add-LLVM-bitcode-emission-for-backend-sourc.patch (5.2K, 7-v8-0006-meson-Add-LLVM-bitcode-emission-for-backend-sourc.patch) download | inline diff: From c360f354b165c46fb23681dd6c90c8e35f402cf9 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Wed, 12 Mar 2025 10:44:46 +0300 Subject: [PATCH v8 6/6] meson: Add LLVM bitcode emission for backend sources Since generated backend sources may have their own compilation flags and must also be included in the postgres.index.bc, the way to make it work with current code was to create a new variable, called `bc_generated_backend_sources`, which is a list of dictionaries, each one having an optional 'additional_flags' and a `srclist` pointing to the list of custom_target generated sources. An example of a possible structure of bitcode_modules which is processed by the main meson llvm bitcode emission file src/backend/jit/llvm/bitcode/meson.build: ``` bitcode_modules = [ { 'name': 'postgres', 'target': postgres_lib, 'src_file': backend_sources, 'gen_srcfiles': [ { 'additional_flags': [ '-I/path/postgresl/src/backend/parser', '-I/path/postgresl/build/src/backend/parser', ], 'srcfiles': [ <custom_target for scan.c>, <custom_target for gram.c> ] } ] } ] ``` Author: Diego Fronza <[email protected]> Author: Nazir Bilal Yavuz <[email protected]> Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org --- src/backend/bootstrap/meson.build | 2 ++ src/backend/meson.build | 2 ++ src/backend/parser/meson.build | 8 ++++++++ src/backend/replication/meson.build | 4 ++++ src/backend/utils/fmgr/meson.build | 1 + 5 files changed, 17 insertions(+) diff --git a/src/backend/bootstrap/meson.build b/src/backend/bootstrap/meson.build index 29726c1ab4f..389c75f8081 100644 --- a/src/backend/bootstrap/meson.build +++ b/src/backend/bootstrap/meson.build @@ -13,6 +13,7 @@ bootscanner = custom_target('bootscanner', ) generated_sources += bootscanner boot_parser_sources += bootscanner +bc_generated_backend_sources += {'srcfiles': [bootscanner]} bootparse = custom_target('bootparse', input: 'bootparse.y', @@ -20,6 +21,7 @@ bootparse = custom_target('bootparse', ) generated_sources += bootparse.to_list() boot_parser_sources += bootparse +bc_generated_backend_sources += {'srcfiles': [bootparse[0]]} boot_parser = static_library('boot_parser', boot_parser_sources, diff --git a/src/backend/meson.build b/src/backend/meson.build index 276ecbd6d74..aec0276f308 100644 --- a/src/backend/meson.build +++ b/src/backend/meson.build @@ -5,6 +5,7 @@ backend_sources = [] backend_link_with = [pgport_srv, common_srv] generated_backend_sources = [] +bc_generated_backend_sources = [] post_export_backend_sources = [] subdir('access') @@ -144,6 +145,7 @@ bitcode_modules += { 'name': 'postgres', 'target': postgres_lib, 'srcfiles': backend_sources, + 'gen_sources': bc_generated_backend_sources, } pg_mod_c_args = cflags_mod diff --git a/src/backend/parser/meson.build b/src/backend/parser/meson.build index 874aa749aa6..add472a0cd8 100644 --- a/src/backend/parser/meson.build +++ b/src/backend/parser/meson.build @@ -42,6 +42,14 @@ backend_parser = custom_target('gram', generated_sources += backend_parser.to_list() parser_sources += backend_parser +bc_generated_backend_sources += { + 'additional_flags': [ + '-I@0@'.format(meson.current_build_dir()), + '-I@0@'.format(meson.current_source_dir()), + ], + 'srcfiles': [backend_scanner, backend_parser[0]], +} + parser = static_library('parser', parser_sources, dependencies: [backend_code], diff --git a/src/backend/replication/meson.build b/src/backend/replication/meson.build index b0601498865..71ab1164960 100644 --- a/src/backend/replication/meson.build +++ b/src/backend/replication/meson.build @@ -19,6 +19,7 @@ repl_scanner = custom_target('repl_scanner', ) generated_sources += repl_scanner repl_parser_sources += repl_scanner +bc_generated_backend_sources += {'srcfiles': [repl_scanner]} repl_gram = custom_target('repl_gram', input: 'repl_gram.y', @@ -26,6 +27,7 @@ repl_gram = custom_target('repl_gram', ) generated_sources += repl_gram.to_list() repl_parser_sources += repl_gram +bc_generated_backend_sources += {'srcfiles': [repl_gram[0]]} syncrep_scanner = custom_target('syncrep_scanner', input: 'syncrep_scanner.l', @@ -34,6 +36,7 @@ syncrep_scanner = custom_target('syncrep_scanner', ) generated_sources += syncrep_scanner repl_parser_sources += syncrep_scanner +bc_generated_backend_sources += {'srcfiles': [syncrep_scanner]} syncrep_gram = custom_target('syncrep_gram', input: 'syncrep_gram.y', @@ -41,6 +44,7 @@ syncrep_gram = custom_target('syncrep_gram', ) generated_sources += syncrep_gram.to_list() repl_parser_sources += syncrep_gram +bc_generated_backend_sources += {'srcfiles': [syncrep_gram[0]]} repl_parser = static_library('repl_parser', repl_parser_sources, diff --git a/src/backend/utils/fmgr/meson.build b/src/backend/utils/fmgr/meson.build index b1dcab93e70..080f59988e9 100644 --- a/src/backend/utils/fmgr/meson.build +++ b/src/backend/utils/fmgr/meson.build @@ -8,3 +8,4 @@ backend_sources += files( # fmgrtab.c generated_backend_sources += fmgrtab_target[2] +bc_generated_backend_sources += {'srcfiles': [fmgrtab_target[2]]} -- 2.51.0 ^ permalink raw reply [nested|flat] 10+ messages in thread
end of thread, other threads:[~2025-10-31 12:13 UTC | newest] Thread overview: 10+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2024-09-05 09:24 Re: meson vs. llvm bitcode files Nazir Bilal Yavuz <[email protected]> 2025-03-07 10:52 ` Nazir Bilal Yavuz <[email protected]> 2025-03-10 22:04 ` Diego Fronza <[email protected]> 2025-03-12 10:26 ` Nazir Bilal Yavuz <[email protected]> 2025-03-12 13:39 ` Diego Fronza <[email protected]> 2025-03-13 10:11 ` Nazir Bilal Yavuz <[email protected]> 2025-04-29 08:23 ` Nazir Bilal Yavuz <[email protected]> 2025-07-07 08:45 ` Nazir Bilal Yavuz <[email protected]> 2025-08-13 13:25 ` Nazir Bilal Yavuz <[email protected]> 2025-10-31 12:13 ` Nazir Bilal Yavuz <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox