Subject: Re: FW: Building Postgres 17.0 with meson

From: Robert Haas <[email-masked]>
Date: Wed, 6 Nov 2024 16:21:21 -0500

On Wed, Nov 6, 2024 at 2:59 PM Mark Hill <[email-masked]> wrote:
> Srinath is in India I believe and not available currently.   Does anybody have any idea why meson
>
> is not finding the paths I’m specifying with the -Dextra_lib_dirs and  -Dextra_include_dirs?  See below.

I am not an expert on this topic, but do use these options on macOS
and they do work for me. A typical 'meson setup' invocation for me is:

meson setup $HOME/pgsql $HOME/pgsql-meson -Dcassert=true -Ddebug=true
-Dextra_include_dirs=/opt/local/include
-Dextra_lib_dirs=/opt/local/lib -Dprefix=$HOME/install/dev

In your example, you set
extra_lib_dirs=%openssl_directory%,%zlib_directory%,%icu4c_directory%.
Those aren't actually pathnames, so something (your shell, for
instance) would have to be substituting the actual correct pathnames.
Maybe that isn't happening, or maybe the pathnames that are being
substituted are not the right ones, or maybe they don't contain the
files that meson thinks they ought to contain.

One thing that you could do is have a look at the meson log file. On
my machine, that shows up inside the directory where I run 'meson
setup' at meson-logs/meson-logs.txt. I am not sure if it's different
on Windows. If you look in that file for strings like icu or openssl
or zlib, you may be able to understand what is going wrong.

As far as "Srinath is in India I believe and not available currently,"
while that is probably true, I'm not sure that's really the issue.
Your original email didn't give the exact command you ran, the exact
output you got, or any error messages or log files. Your follow-up
email didn't include any of that either, except material quoted from
another email that you apparently sent off-list, and even that wasn't
nearly as detailed as many posts that you will see here. Also, see
also the "Email etiquette mechanics" section of
https://wiki.postgresql.org/wiki/Mailing_Lists which suggests keeping
replies on-list and not top-posting.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

---

From: Mark Hill <[email-masked]>
Date: Wed, 13 Nov 2024 15:17:20 +0000

EXTERNAL

On Wed, Nov 6, 2024 at 4:21 PM Robert Hass <[email-masked]> wrote:
> I am not an expert on this topic, but do use these options on macOS and they do work for me. A typical 'meson setup' invocation for me is:

> meson setup $HOME/pgsql $HOME/pgsql-meson -Dcassert=true -Ddebug=true -Dextra_include_dirs=/opt/local/include

Including more information this time hoping it will help in debugging the problem.  I'm only building with openssl at the moment to keep things simple.
I've included the output of the meson setup command and also attached the log file that's generated.

I have built and installed OpenSSL 3.1.6 in the following location:
D:\Jenkins\workspace\workspace\Postgres-9.4\OpenSSL\OpenSSL-Install\OpenSSL-3.1.6-wx6

I'm setting the following env variables:
set openssl_include_dir=D:\Jenkins\workspace\workspace\Postgres-9.4\OpenSSL\OpenSSL-Install\OpenSSL-3.1.6-wx6\include\openssl
set openssl_lib_dir=D:\Jenkins\workspace\workspace\Postgres-9.4\OpenSSL\OpenSSL-Install\OpenSSL-3.1.6-wx6\lib
set prefix=D:\Jenkins\workspace\workspace\Postgres-9.4\PostgreSQL\postgres-Install\pg_17.0_wx6

meson setup command and output: 
D:\Jenkins\workspace\workspace\Postgres-9.4\PostgreSQL\postgres-Source\buildPostgreSQL_wx6_17.0\postgresql-17.0>meson setup build --prefix="%prefix%" --buildtype=release -Dssl=openssl -Dextra_lib_dirs=%openssl_lib_dir% -Dextra_include_dirs=%openssl_include_dir% -Dnls=disabled -Dplperl=disabled -Dplpython=disabled -Dpltcl=disabled -Dllvm=disabled -Dlz4=disabled -Dzstd=disabled -Dgssapi=disabled -Dldap=disabled -Dpam=disabled -Dbsd_auth=disabled -Dsystemd=disabled -Dbonjour=disabled -Duuid=none -Dlibxml=disabled -Dlibxslt=disabled -Dselinux=disabled -Dicu=disabled -Dzlib=disabled
The Meson build system
Version: 1.6.0
Source dir: D:\Jenkins\workspace\workspace\Postgres-9.4\PostgreSQL\postgres-Source\buildPostgreSQL_wx6_17.0\postgresql-17.0
Build dir: D:\Jenkins\workspace\workspace\Postgres-9.4\PostgreSQL\postgres-Source\buildPostgreSQL_wx6_17.0\postgresql-17.0\build
Build type: native build
Project name: postgresql
Project version: 17.0
C compiler for the host machine: cl (msvc 19.29.30154 "Microsoft (R) C/C++ Optimizing Compiler Version 19.29.30154 for x64")
C linker for the host machine: link link 14.29.30154.0
Host machine cpu family: x86_64
Host machine cpu: x86_64
Run-time dependency threads found: YES
Library ws2_32 found: YES
Library secur32 found: YES
Program perl found: YES (C:\Strawberry\perl\bin\perl.EXE)
Program python3 found: YES (C:\Program Files\Python38\python3.EXE)
Program flex found: YES 2.5.35 2.5.35 (C:\GnuWin32\bin\flex.EXE)
Program bison found: YES 2.4.1 2.4.1 (C:\GnuWin32\bin\bison.EXE)
Program sed found: YES (C:\GnuWin32\bin\sed.EXE)
Program prove found: YES (C:\Strawberry\perl\bin\prove.BAT)
Program tar found: YES (C:\Windows\system32\tar.EXE)
Program gzip found: NO
Program lz4 found: NO
Program openssl found: NO
Program zstd found: NO
Program dtrace skipped: feature dtrace disabled
Program config/missing found: YES (sh D:\Jenkins\workspace\workspace\Postgres-9.4\PostgreSQL\postgres-Source\buildPostgreSQL_wx6_17.0\postgresql-17.0\config/missing)
Program cp found: NO
Program xmllint found: NO
Program xsltproc found: NO
Program wget found: NO
Program C:\Python\Python310\python.exe found: YES (C:\Python\Python310\python.exe)
Check usable header "bsd_auth.h" skipped: feature bsd_auth disabled
Check usable header "dns_sd.h" skipped: feature bonjour disabled
Program fop found: NO
Compiler for language cpp skipped: feature llvm disabled
Found pkg-config 'C:\\Strawberry\\perl\\bin\\pkg-config.BAT' but it is Strawberry Perl and thus broken. Ignoring...
Found pkg-config: NO
Found CMake: C:\Program Files\CMake\bin\cmake.EXE (3.29.0)
Run-time dependency readline found: NO (tried pkgconfig and cmake)
Library readline found: NO
Run-time dependency libedit found: NO (tried pkgconfig and cmake)
Library libedit found: NO
Dependency libselinux skipped: feature selinux disabled
Dependency libsystemd skipped: feature systemd disabled
Run-time dependency openssl found: NO (tried pkgconfig, system and cmake)

meson.build:1336:17: ERROR: C header 'openssl/ssl.h' not found

A full log can be found at D:\Jenkins\workspace\workspace\Postgres-9.4\PostgreSQL\postgres-Source\buildPostgreSQL_wx6_17.0\postgresql-17.0\build\meson-logs\meson-log.txt

The error above says meson can't find openssl/ssl.h but it's there:
D:\Jenkins\workspace\workspace\Postgres-9.4\PostgreSQL\postgres-Source\buildPostgreSQL_wx6_17.0\postgresql-17.0>dir D:\Jenkins\workspace\workspace\Postgres-9.4\OpenSSL\OpenSSL-Install\OpenSSL-3.1.6-wx6\include\openssl\ssl.h
 Volume in drive D is Data
 Volume Serial Number is 58B5-7193

 Directory of D:\Jenkins\workspace\workspace\Postgres-9.4\OpenSSL\OpenSSL-Install\OpenSSL-3.1.6-wx6\include\openssl

08/12/2024  04:22 PM           127,595 ssl.h
               1 File(s)        127,595 bytes
               0 Dir(s)  210,974,670,848 bytes free

I don't know why meson cannot find the OpenSSL installation I've specified via the options:
extra_lib_dirs
extra_include_dirs

Thanks, Mark

---

From: Nazir Bilal Yavuz <[email-masked]>
Date: Wed, 13 Nov 2024 18:51:42 +0300

Hi,

> I don't know why meson cannot find the OpenSSL installation I've specified via the options:

I think that the problem is that you are setting '.../include/openssl'
as an include_dir not '.../include'. Could you please try:

set openssl_include_dir=D:\Jenkins\workspace\workspace\Postgres-9.4\OpenSSL\OpenSSL-Install\OpenSSL-3.1.6-wx6\include

-- 
Regards,
Nazir Bilal Yavuz
Microsoft

---

From: Mark Hill <[email-masked]>
Date: Thu, 14 Nov 2024 19:55:07 +0000

> set openssl_include_dir=D:\Jenkins\workspace\workspace\Postgres-9.4\OpenSSL\OpenSSL-Install\OpenSSL-3.1.6-wx6\include

Hi Nazir,

Thank you, that worked!   I had actually tried that with openssl, icu, and zlib all enabled but when either icu or zlib failed it didn't get
as far as openssl so I didn't know that it would work until I tried with just openssl by itself.  I have the setup working now but ninja
fails when I do the setup with openssl, zlib, and icu.   If I setup just openssl and zlib, the build works.

With openssl, zlib, and icu enabled in the setup, the ninja build fails trying to link src/backend/postgres.exe.  There are 40 unresolved
external symbol errors (see below.)  I checked a few of the symbols and they appear in the Postgres source without the "_72" text on
the end.   Is it getting "72" from the version of icu4c I'm using, 72.1?   Anyone know how to prevent these errors?

Thanks, Mark

[1206/2170] Linking target src/backend/postgres.exe
FAILED: src/backend/postgres.exe
"link" @src/backend/postgres.exe.rsp
   Creating library src\backend\postgres.lib
commands_collationcmds.c.obj : error LNK2019: unresolved external symbol uloc_getDisplayName_72 referenced in function get_icu_locale_comment
commands_collationcmds.c.obj : error LNK2019: unresolved external symbol uloc_getAvailable_72 referenced in function pg_import_system_collations
utils_adt_pg_locale.c.obj : error LNK2001: unresolved external symbol uloc_getAvailable_72
commands_collationcmds.c.obj : error LNK2019: unresolved external symbol uloc_countAvailable_72 referenced in function pg_import_system_collations
utils_adt_pg_locale.c.obj : error LNK2001: unresolved external symbol uloc_countAvailable_72
regex_regcomp.c.obj : error LNK2019: unresolved external symbol u_islower_72 referenced in function pg_wc_islower
regex_regcomp.c.obj : error LNK2019: unresolved external symbol u_isupper_72 referenced in function pg_wc_isupper
regex_regcomp.c.obj : error LNK2019: unresolved external symbol u_isdigit_72 referenced in function pg_wc_isdigit
regex_regcomp.c.obj : error LNK2019: unresolved external symbol u_isalpha_72 referenced in function pg_wc_isalpha
regex_regcomp.c.obj : error LNK2019: unresolved external symbol u_isalnum_72 referenced in function pg_wc_isalnum
regex_regcomp.c.obj : error LNK2019: unresolved external symbol u_ispunct_72 referenced in function pg_wc_ispunct
regex_regcomp.c.obj : error LNK2019: unresolved external symbol u_isgraph_72 referenced in function pg_wc_isgraph
regex_regcomp.c.obj : error LNK2019: unresolved external symbol u_isspace_72 referenced in function pg_wc_isspace
regex_regcomp.c.obj : error LNK2019: unresolved external symbol u_isprint_72 referenced in function pg_wc_isprint
regex_regcomp.c.obj : error LNK2019: unresolved external symbol u_tolower_72 referenced in function casecmp
regex_regcomp.c.obj : error LNK2019: unresolved external symbol u_toupper_72 referenced in function pg_wc_toupper
utils_adt_formatting.c.obj : error LNK2019: unresolved external symbol u_errorName_72 referenced in function icu_convert_case
utils_adt_pg_locale.c.obj : error LNK2001: unresolved external symbol u_errorName_72
utils_adt_formatting.c.obj : error LNK2019: unresolved external symbol u_strToUpper_72 referenced in function str_toupper
utils_adt_formatting.c.obj : error LNK2019: unresolved external symbol u_strToLower_72 referenced in function str_tolower
utils_adt_formatting.c.obj : error LNK2019: unresolved external symbol u_strToTitle_72 referenced in function u_strToTitle_default_BI
utils_adt_pg_locale.c.obj : error LNK2019: unresolved external symbol u_versionToString_72 referenced in function get_collation_actual_version
utils_adt_pg_locale.c.obj : error LNK2019: unresolved external symbol uiter_setString_72 referenced in function pg_strnxfrm_prefix_icu
utils_adt_pg_locale.c.obj : error LNK2019: unresolved external symbol uiter_setUTF8_72 referenced in function pg_strnxfrm_prefix_icu
utils_adt_pg_locale.c.obj : error LNK2019: unresolved external symbol uloc_getLanguage_72 referenced in function icu_validate_locale
utils_adt_pg_locale.c.obj : error LNK2019: unresolved external symbol uloc_canonicalize_72 referenced in function icu_set_collation_attributes
utils_adt_pg_locale.c.obj : error LNK2019: unresolved external symbol uloc_toLanguageTag_72 referenced in function icu_language_tag
utils_adt_pg_locale.c.obj : error LNK2019: unresolved external symbol ucol_open_72 referenced in function pg_ucol_open
utils_adt_pg_locale.c.obj : error LNK2019: unresolved external symbol ucol_openRules_72 referenced in function make_icu_collator
utils_adt_pg_locale.c.obj : error LNK2019: unresolved external symbol ucol_close_72 referenced in function get_collation_actual_version
utils_adt_pg_locale.c.obj : error LNK2019: unresolved external symbol ucol_strcoll_72 referenced in function pg_strncoll_icu
utils_adt_pg_locale.c.obj : error LNK2019: unresolved external symbol ucol_strcollUTF8_72 referenced in function pg_strncoll_icu
utils_adt_pg_locale.c.obj : error LNK2019: unresolved external symbol ucol_getRules_72 referenced in function make_icu_collator
utils_adt_pg_locale.c.obj : error LNK2019: unresolved external symbol ucol_getSortKey_72 referenced in function pg_strnxfrm_icu
utils_adt_pg_locale.c.obj : error LNK2019: unresolved external symbol ucol_nextSortKeyPart_72 referenced in function pg_strnxfrm_prefix_icu
utils_adt_pg_locale.c.obj : error LNK2019: unresolved external symbol ucol_getVersion_72 referenced in function get_collation_actual_version
utils_adt_pg_locale.c.obj : error LNK2019: unresolved external symbol ucol_setAttribute_72 referenced in function icu_set_collation_attributes
utils_adt_pg_locale.c.obj : error LNK2019: unresolved external symbol ucnv_open_72 referenced in function init_icu_converter
utils_adt_pg_locale.c.obj : error LNK2019: unresolved external symbol ucnv_fromUChars_72 referenced in function icu_from_uchar
utils_adt_pg_locale.c.obj : error LNK2019: unresolved external symbol ucnv_toUChars_72 referenced in function uchar_convert
utils_adt_pg_locale.c.obj : error LNK2019: unresolved external symbol u_strlen_72 referenced in function make_icu_collator
utils_adt_pg_locale.c.obj : error LNK2019: unresolved external symbol u_strcat_72 referenced in function make_icu_collator
utils_adt_pg_locale.c.obj : error LNK2019: unresolved external symbol u_strcpy_72 referenced in function make_icu_collator
src\backend\postgres.exe : fatal error LNK1120: 40 unresolved externals
[1211/2170] Compiling C object src/backend/snowball/dict_snowball.dll.p/libstemmer_stem_UTF_8_hindi.c.obj
ninja: build stopped: subcommand failed.

---

From: Mark Hill <[email-masked]>
Date: Fri, 15 Nov 2024 13:01:36 +0000

On Wed, 14 Nov 2024 at 2:55PM(EST), Mark Hill <[email-masked] > wrote:

> With openssl, zlib, and icu enabled in the setup, the ninja build fails trying to link src/backend/postgres.exe.
> There are 40 unresolved external symbol errors (see below.)  I checked a few of the symbols and they appear
> in the Postgres source without the "_72" text on the end.   Is it getting "72" from the version of icu4c I'm using,
> 72.1?   Anyone know how to prevent these errors?

I'm including the meson-log.txt file for the build I reported in my previous post for more information.