Subject: [RFC] building postgres with meson

From: Andres Freund <[email-masked]>
Date: Tue, 12 Oct 2021 01:37:21 -0700

Hi,

For the last year or so I've on and off tinkered with $subject.  I think
it's in a state worth sharing now.  First, let's look at a little
comparison.

My workstation:

non-cached configure:
current:        11.80s
meson:           6.67s

non-cached build (world-bin):
current:        40.46s
ninja:           7.31s

no-change build:
current:         1.17s
ninja:           0.06s

test world:
current:          105s
meson:             63s

What actually started to motivate me however were the long times windows
builds took to come back with testsresults. On CI, with the same machine
config:

build:
current:          202s (doesn't include genbki etc)
meson+ninja:      140s
meson+msbuild:    206s

test:
current:         1323s (many commands)
meson:            903s (single command)

(note that the test comparison isn't quite fair - there's a few tests
missing, but it's just small contrib ones afaik)

The biggest difference to me however is not the speed, but how readable
the output is.

Running the tests with meson in a terminal, shows the number of tests
that completed out of how many total, how much time has passed, how long
the currently running tests already have been running.

At the end of a testrun a count of tests is shown:

188/189 postgresql:tap+pg_basebackup / pg_basebackup/t/010_pg_basebackup.pl                         OK               39.51s   110 subtests passed
189/189 postgresql:isolation+snapshot_too_old / snapshot_too_old/isolation                          OK               62.93s

Ok:                 188
Expected Fail:      0
Fail:               1
Unexpected Pass:    0
Skipped:            0
Timeout:            0

Full log written to /tmp/meson/meson-logs/testlog.txt

The log has the output of the tests and ends with:

Summary of Failures:
120/189 postgresql:tap+recovery / recovery/t/007_sync_rep.pl                               ERROR             7.16s   (exit status 255 or signal 127 SIGinvalid)

Quite the difference to make check-world -jnn output.

So, now that the teasing is done, let me explain a bit what lead me down
this path:

Autoconf + make is not being actively developed. Especially autoconf is
*barely* in maintenance mode - despite many shortcomings and bugs. It's
also technology that very few want to use - autoconf m4 is scary, and
it's scarier for people that started more recently than a lot of us
committers for example.

Recursive make as we use it is hard to get right. One reason the clean
make build is so slow compared to meson is that we had to resort to
.NOTPARALLEL to handle dependencies in a bunch of places. And despite
that, I quite regularly see incremental build failures that can be
resolved by retrying the build.

While we have incremental build via --enable-depend, they don't work
that reliable (i.e. misses necessary rebuilds) and yet is often too
aggressive. More modern build system can keep track of the precise
command used to build a target and rebuild it when that command changes.

We also don't just have the autoconf / make buildsystem, there's also
the msvc project generator - something most of us unix-y folks do not
like to touch. I think that, combined with there being no easy way to
run all tests, and it being just different, really hurt our windows
developer appeal (and subsequently the quality of postgres on
windows). I'm not saying this to ding the project generator - that was
well before there were decent "meta" buildsystems out there (and in some
ways it is a small one itself).

The last big issue I have with the current situation is that there's no
good test integration. make check-world output is essentially unreadable
/ not automatically parseable. Which led to the buildfarm having a
separate list of things it needs to test, so that failures can be
pinpointed and paired with appropriate logs. That approach unfortunately
doesn't scale well to multi-core CPUs, slowing down the buildfarm by a
fair bit.

This all led to me to experiment with improvements. I tried a few
somewhat crazy but incremental things like converting our buildsystem to
non-recursive make (I got it to build the backend, but it's too hard to
do manually I think), or to not run tests during the recursive make
check-world, but to append commands to a list of tests, that then is run
by a helper (can kinda be made to work).  In the end I concluded that
the amount of time we'd need to invest to maintain our more-and-more
custom buildsystem going forward doesn't make sense.

Which lead me to look around and analyze which other buildsystems there
are that could make some sense for us. The halfway decent list includes,
I think:
1) cmake
2) bazel
3) meson

cmake would be a decent choice, I think. However, I just can't fully
warm up to it. Something about it just doesn't quite sit right with
me. That's not a good enough reason to prevent others from suggesting to
use it, but it's good enough to justify not investing a lot of time in
it myself.

Bazel has some nice architectural properties. But it requires a JVM to
run - I think that basically makes it insuitable for us. And the build
information seems quite arduous to maintain too.

Which left me with meson. It is a meta-buildsystem that can do the
actual work of building via ninja (the most common one, also targeted by
cmake), msbuild (visual studio project files, important for GUI work)
and xcode projects (I assume that's for a macos IDE, but I haven't tried
to use it). Meson roughly does what autoconf+automake did, in a
python-esque DSL, and outputs build-instructions for ninja / msbuild /
xcode.  One interesting bit is that meson itself is written in python (
and fairly easy to contribute too - I got a few changes in now).

I don't think meson is perfect architecturally - e.g. its insistence on
not having functions ends up making it a bit harder to not end up
duplicating code. There's some user-interface oddities that are now hard
to fix fully, due to the faily wide usage. But all-in-all it's pretty
nice to use.

Its worth calling out that a lot of large open source projects have been
/ are migrating to meson. qemu/kvm, mesa (core part of graphics stack on
linux and also widely used in other platforms), a good chunk of GNOME,
and quite a few more. Due to that it seems unlikely to be abandoned
soon.

As far as I can tell the only OS that postgres currently supports that
meson doesn't support is HPUX. It'd likely be fairly easy to add
gcc-on-hpux support, a chunk more to add support for the proprietary
ones.

The attached patch (meson support is 0016, the rest is prerequisites
that aren't that interesting at this stage) converts most of postgres to
meson. There's a few missing contrib modules, only about half the
optional library dependencies are implemented, and I've only built on
x64. It builds on freebsd, linux, macos and windows (both ninja and
msbuild) and cross builds from linux to windows. Thomas helped make the
freebsd / macos pieces a reality, thanks!

I took a number of shortcuts (although there used to be a *lot*
more). So this shouldn't be reviewed to the normal standard of the
community - it's a prototype.  But I think it's in a complete enough
shape that it allows to do a well-informed evaluation.

What doesn't yet work/ build:

- plenty optional libraries, contrib, NLS, docs build

- PGXS - and I don't yet know what to best do about it. One
  backward-compatible way would be to continue use makefiles for pgxs,
  but do the necessary replacement of Makefile.global.in via meson (and
  not use that for postgres' own build).  But that doesn't really
  provide a nicer path for building postgres extensions on windows, so
  it'd definitely not be a long-term path.

- JIT bitcode generation for anything but src/backend.

- anything but modern-ish x86. That's proably a small amount of work,
  but something that needs to be done.

- exporting all symbols for extension modules on windows (the stuff for
  postgres is implemented). Instead I marked the relevant symbols als
  declspec(dllexport). I think we should do that regardless of the
  buildsystem change. Restricting symbol visibility via gcc's
  -fvisibility=hidden for extensions results in a substantially reduced
  number of exported symbols, and even reduces object size (and I think
  improves the code too).  I'll send an email about that separately.

There's a lot more stuff to talk about, but I'll stop with a small bit
of instructions below:

Demo / instructions:
# Get code
git remote add andres [email-masked]:anarazel/postgres.git
git fetch andres
git checkout --track andres/meson

# setup build directory
meson setup build --buildtype debug
cd build

# build (uses automatically as many cores as available)
ninja

# change configuration, build again
meson configure -Dssl=openssl
ninja

# run all tests
meson test

# run just recovery tests
meson test --suite setup --suite recovery

# list tests
meson test --list

Greetings,

Andres Freund

---

From: Andres Freund <[email-masked]>
Date: Tue, 12 Oct 2021 02:08:29 -0700

Hi,

> non-cached build (world-bin):

Interestingly this is pretty close to the minimum achievable on my
machine from the buildsystem perspective.

A build with -fuse-ld=lld, which the above didn't use, takes 6.979s. The
critical path is

bison gram.y -> gram.c       4.13s
gcc gram.c -> gram.o         2.05s
gcc postgres ....            0.317

A very helpful visualization is to transform ninja's build logs into a
tracefile with https://github.com/nico/ninjatracing

I attached an example - the trace.json.gz can be uploaded as-is to
https://ui.perfetto.dev/

It's quite a bit of of fun to look at imo.

There's a few other things quickly apparent:

- genbki prevents build progress due to dependencies on the generated
  headers.
- the absolutely stupid way I implemented the python2->python3
  regression test output conversion uses up a fair bit of resources
- tablecmds.c, pg_dump.c, xlog.c and a few other files are starting to
  big enough to be problematic compile-time wise

Greetings,

Andres Freund

---

From: Peter Eisentraut <[email-masked]>
Date: Tue, 12 Oct 2021 15:30:57 +0200

> it's in a state worth sharing now.  First, let's look at a little

I played with $subject a few years ago and liked it.  I think, like you 
said, meson is the best way forward.  I support this project.

One problem I noticed back then was that some choices that we currently 
determine ourselves in configure or the makefiles are hardcoded in 
meson.  For example, at the time, gcc on macOS was not supported.  Meson 
thought, if you are on macOS, you are surely using the Apple compiler, 
and it supports these options.  Fixing that required patches deep in the 
bowels of the meson source code (and, in practice, waiting for a new 
release etc.).  I strongly suspect this isn't the only such problem. 
For example, the shared library build behavior has been carefully tuned 
in opinionated ways.  With the autotools chain, one can override 
anything with enough violence; so we have always felt free to do that. 
I haven't followed it in a while, so I don't know what the situation is 
now; but it is a concern, because we have always felt free to try new 
and unusual build tools (Sun compiler, Intel compiler, 
clang-when-it-was-new) early without waiting for anyone else.

---

From: Robert Haas <[email-masked]>
Date: Tue, 12 Oct 2021 11:00:14 -0400

> clang-when-it-was-new) early without waiting for anyone else.

I think we're going to need some solution to this problem. We have too
many people here with strong opinions about questions like this for me
to feel good about the idea that we're going to collectively be OK
with leaving these sorts of decisions up to some other project.

---

From: Andrew Dunstan <[email-masked]>
Date: Tue, 12 Oct 2021 11:08:53 -0400

> git remote add andres [email-masked]:anarazel/postgres.git

ITYM:

git remote add andres git://github.com/anarazel/postgres.git

cheers

andrew
 
--
Andrew Dunstan
EDB: https://www.enterprisedb.com

---

From: Josef Šimánek <[email-masked]>
Date: Tue, 12 Oct 2021 17:21:50 +0200

.

> # build (uses automatically as many cores as available)

I'm getting errors at this step. You can find my output at
https://pastebin.com/Ar5VqfFG. Setup went well without errors. Is that
expected for now?

---

From: Andrew Dunstan <[email-masked]>
Date: Tue, 12 Oct 2021 11:28:04 -0400

> meson setup build --buildtype debug

I took this for an outing on msys2 and it just seems to hang. If it's not hanging it's unbelievably slow.

cheers

andrew

---

From: Tom Lane <[email-masked]>
Date: Tue, 12 Oct 2021 11:47:37 -0400

> with leaving these sorts of decisions up to some other project.

Agreed.  I'm willing to put up with the costs of moving to some
other build system, but not if it dictates choices we don't want to
make about the end products.

> From my point of view, the time it takes to run configure is annoying,
> but the build time is pretty fine. On my system, configure takes about
> 33 seconds, and a full rebuild with 'make -j8' takes 14.5 seconds (I
> am using ccache). Moreover, most of the time when I run make, I'm only
> doing a partial rebuild, so it's near-instantaneous.

Read about Autoconf's --cache-file option.  That and ccache are
absolutely essential tools IMO.

			regards, tom lane

---

From: Andrew Dunstan <[email-masked]>
Date: Tue, 12 Oct 2021 11:50:03 -0400

> I took this for an outing on msys2 and it just seems to hang. If it's not hanging it's unbelievably slow.

It hung because it expected the compiler to be 'ccache cc'. Hanging in
such a case is kinda unforgivable. I remedied that by setting 'CC=gcc'
but it then errored out looking for perl libs. I think msys2 is going to
be a bit difficult here :-(

cheers

andrew

---

From: Andres Freund <[email-masked]>
Date: Tue, 12 Oct 2021 09:15:41 -0700

Hi,

> said, meson is the best way forward.  I support this project.

Cool.

> determine ourselves in configure or the makefiles are hardcoded in meson.

Yea, there's some of that. I think some degree of reduction in flexibility is
needed to realistically target multiple "backend" build-system like visual
studio project files etc. but I wish there were a bit less of that
nonetheless.

> For example, at the time, gcc on macOS was not supported.  Meson thought, if
> you are on macOS, you are surely using the Apple compiler, and it supports
> these options.

I'm pretty sure this one now can just be overridden with CC=gcc. It can on
linux and windows, but I don't have ready interactive access with a mac
(leaving cirrus asside, which now has a "start a terminal" option...).

> For example, the shared library build behavior has been carefully tuned in
> opinionated ways.  With the autotools chain, one can override anything with
> enough violence; so we have always felt free to do that. I haven't followed
> it in a while, so I don't know what the situation is now; but it is a
> concern, because we have always felt free to try new and unusual build tools
> (Sun compiler, Intel compiler, clang-when-it-was-new) early without waiting
> for anyone else.

It's possible to just take over building e.g. shared libraries ourselves with
custom targets. Although it'd be a bit annoying to do. The bigger problem is
that that e.g. wouldn't play that nicely with generating visual studio
projects, which require to generate link steps in a certain way. It'd build,
but the GUI might loose some of its options. Etc.

Greetings,

Andres Freund

---

From: Andres Freund <[email-masked]>
Date: Tue, 12 Oct 2021 09:59:26 -0700

Hi,

> be a bit difficult here :-(

Hm. Yea, the perl thing is my fault - you should be able to get past it with
-Dperl=disabled, and I'll take a look at fixing the perl detection. (*)

I can't reproduce the hanging though. I needed to install bison, flex and
ninja and disable perl as described above, but then it built just fine.

It does seems to crash somewhere in the main regression tests though, I think
I don't do the "set stack depth" dance correctly for msys.

If you repro the hanging, what's the last bit in meson-logs/meson-log.txt?

(*) I've for now made most dependencies autodetected, unless you pass
--auto-features disabled to collectively disable all the auto-detected
features. Initially I had mirrored the autoconf behaviour, but I got sick of
forgetting to turn off readline or zlib on windows. And then it was useful to
test on multiple operating systems...

For working on windows meson's wraps are quite useful. I've not added that to
the git branch, but if you manually do
  mkdir subprojects
  meson wrap install lz4
  meson wrap install zlib
building with -Dzlib=enabled -Dlz4=enabled will fall back to building lz4,
zlib as-needed.

I was wondering about adding a binary wrap for e.g. bison, flex on windows, so
that the process of getting a build going isn't as arduous.

Greetings,

Andres Freund

---

From: Andres Freund <[email-masked]>
Date: Tue, 12 Oct 2021 10:17:34 -0700

Hi,

> https://pastebin.com/Ar5VqfFG. Setup went well without errors. Is that

Thanks, that's helpful. And no, that's not expected (*), it should be fixed.

What OS / distribution / version is this?

Can you build postgres "normally" with --with-gss? Seems like we're ending up
with a version of gssapi that we're not compatible with.

You should be able to get past this by disabling gss using meson configure
-Dgssapi=disabled.

Greetings,

Andres Freund

* except kinda, in the sense that I'd expect it to be buggy, given that I've
  run it only on a few machines and it's very, uh, bleeding edge

---

From: Andres Freund <[email-masked]>
Date: Tue, 12 Oct 2021 11:09:34 -0700

Hi,

> -Dperl=disabled, and I'll take a look at fixing the perl detection. (*)

This is a weird one. I don't know much about msys, so it's probably related to
that. Perl spits out /usr/lib/perl5/core_perl/ as its archlibexp. According to
shell commands that exists, but not according to msys's own python

$ /mingw64/bin/python -c "import os; p = '/usr/lib/perl5/core_perl/CORE'; print(f'does {p} exist:', os.path.exists(p))"
does /usr/lib/perl5/core_perl/CORE exist: False

$ ls -ld /usr/lib/perl5/core_perl/CORE
drwxr-xr-x 1 anfreund anfreund 0 Oct 10 10:19 /usr/lib/perl5/core_perl/CORE

So it's not too surprising that that doesn't work out. It's easy enough to
work around, but still pretty weird.

I pushed a workaround for the config-time error, but it doesn't yet recognize
msys perl correctly. But at least it's not alone in that - configure doesn't
seem to either, so I'm probably doing something wrong :)

> I don't do the "set stack depth" dance correctly for msys.

That was it - just hadn't ported setting -Wl,--stack=... for !msvc
windows. Pushed the fix for that out.

I guess I should figure out how to commandline install msys and add it to CI.

Greetings,

Andres Freund

---

From: Andrew Dunstan <[email-masked]>
Date: Tue, 12 Oct 2021 14:11:39 -0400

> If you repro the hanging, what's the last bit in meson-logs/meson-log.txt?

Here's the entire thing

# cat 
C:/tools/msys64/home/Administrator/postgresql/build/meson-logs/meson-log.txt
Build started at 2021-10-12T18:08:34.387568
Main binary: C:/tools/msys64/mingw64/bin/python.exe
Build Options: -Dbuildtype=debug
Python system: Windows
The Meson build system
Version: 0.59.1
Source dir: C:/tools/msys64/home/Administrator/postgresql
Build dir: C:/tools/msys64/home/Administrator/postgresql/build
Build type: native build
Project name: postgresql
Project version: 15devel
Sanity testing C compiler: ccache cc
Is cross compiler: False.
Sanity check compiler command line: ccache cc sanitycheckc.c -o
sanitycheckc.exe -D_FILE_OFFSET_BITS=64
Sanity check compile stdout:

-----
Sanity check compile stderr:

-----

meson.build:1:0: ERROR: Compiler ccache cc can not compile programs.

cheers

andrew

---

From: Andres Freund <[email-masked]>
Date: Tue, 12 Oct 2021 11:23:59 -0700

Hi,

> meson.build:1:0: ERROR: Compiler ccache cc can not compile programs.

Huh, it's not a question of gcc vs cc, it's that meson automatically uses
ccache. And it looks like msys's ccache is broken at the moment (installed
yesterday):

$ ccache --version
ccache version 4.4.1
...

$ echo > test.c
$ ccache cc -c test.c
Segmentation fault (core dumped)
..

not sure how that leads to hanging, but it's not too surprising that things
don't work out after that...

Greetings,

Andres Freund

---

From: Andrew Dunstan <[email-masked]>
Date: Tue, 12 Oct 2021 14:37:04 -0400

> drwxr-xr-x 1 anfreund anfreund 0 Oct 10 10:19 /usr/lib/perl5/core_perl/CORE

Looks to me like a python issue:

# perl -e 'my $p = "/usr/lib/perl5/core_perl/CORE"; print qq(does $p
exist: ), -e $p, qq{\n};'
does /usr/lib/perl5/core_perl/CORE exist: 1

# python -c "import os; p = '/usr/lib/perl5/core_perl/CORE';
print(f'does {p} exist:', os.path.exists(p))"
does /usr/lib/perl5/core_perl/CORE exist: False

# cygpath -m /usr/lib/perl5/core_perl/CORE
C:/tools/msys64/usr/lib/perl5/core_perl/CORE

# python -c "import os; p =
'C:/tools/msys64/usr/lib/perl5/core_perl/CORE'; print(f'does {p}
exist:', os.path.exists(p))"
does C:/tools/msys64/usr/lib/perl5/core_perl/CORE exist: True

Clearly python is not understanding msys virtualized paths.

> I guess I should figure out how to commandline install msys and add it to CI.

here's what I do:

    # msys2 outputs esc-[3J which clears the screen's scroll buffer. Nasty.
    # so we redirect the output
    # find the log in c:\Windows\System32 if needed
    choco install -y --no-progress --limit-output msys2 > msys2inst.log
    c:\tools\msys64\usr\bin\bash -l
    '/c/vfiles/windows-uploads/msys2-packages.sh'

Here's what's in msys-packages.sh:

    pacman -S --needed --noconfirm \
        base-devel \
        msys/git \
        msys/ccache \
        msys/vim  \
        msys/perl-Crypt-SSLeay \
        mingw-w64-clang-x86_64-toolchain \
        mingw-w64-x86_64-toolchain

    # could do: pacman -S --needed --noconfirm development
    # this is more economical. These should cover most of the things you
    might
    # want to configure with

    pacman -S --needed --noconfirm \
           msys/gettext-devel \
           msys/icu-devel \
           msys/libiconv-devel \
           msys/libreadline-devel \
           msys/libxml2-devel \
           msys/libxslt-devel \
           msys/openssl-devel \
           msys/zlib-devel

cheers

andrew

---

From: Andrew Dunstan <[email-masked]>
Date: Tue, 12 Oct 2021 14:42:27 -0400

> don't work out after that...

Yes, I've had to disable ccache on fairywren.

cheers

andrew

---

From: Andres Freund <[email-masked]>
Date: Tue, 12 Oct 2021 12:01:22 -0700

Hi,

> (leaving cirrus asside, which now has a "start a terminal" option...).

It was a tad more complicated. But only because it took me a while to figure
out how to make gcc on macos actually work, independent of meson. Initially
gcc was always failing with errors about not finding the linker, and
installing binutils was a dead end.

Turns out just using a gcc at a specific path doesn't work, it ends up using
wrong internal binaries or something like that.

Once I got to that, the meson part was easy:

$ export PATH="/usr/local/opt/gcc/bin:$PATH"
$ CC=gcc-11 meson setup build-gcc
  ...
  C compiler for the host machine: gcc-11 (gcc 11.2.0 "gcc-11 (Homebrew GCC 11.2.0) 11.2.0")
  ...
$ cd build-gcc
$ ninja test
...

  181/181 postgresql:tap+subscription / subscription/t/100_bugs.pl                                 OK               17.83s   5 subtests passed

  Ok:                 180
  Expected Fail:      0
  Fail:               0
  Unexpected Pass:    0
  Skipped:            1
  Timeout:            0

One thing that is nice with meson's testrunner is that it can parse the output
of tap tests and recognizes the number of completed / failed subtests. I
wonder whether we could make pg_regress' output tap compliant without the
output quality suffering too much.

Greetings,

Andres Freund

---

From: Andrew Dunstan <[email-masked]>
Date: Tue, 12 Oct 2021 15:16:56 -0400

> Clearly python is not understanding msys virtualized paths.

It's a matter of which python you use. The one that understands msys
paths is msys/python. The mingw64 packages are normally pure native
windows and so don't understand msys paths. I know it's confusing :-(

# /usr/bin/python -c "import os; p = '/usr/lib/perl5/core_perl/CORE';
print(f'does {p} exist:', os.path.exists(p))"
does /usr/lib/perl5/core_perl/CORE exist: True

cheers

andrew

---

From: Andres Freund <[email-masked]>
Date: Tue, 12 Oct 2021 12:29:42 -0700

Hi,

> Clearly python is not understanding msys virtualized paths.

Ah, it's a question of the *wrong* python being used :/. I somehow ended up
with both a mingw and an msys python, with the mingw python taking preference
over the msys one. The latter one does understand such paths.

> here's what I do:

Thanks!

Does that recipe get you to a build where ./configure --with-perl succeeds?

I see this here:

checking for Perl archlibexp... /usr/lib/perl5/core_perl
checking for Perl privlibexp... /usr/share/perl5/core_perl
checking for Perl useshrplib... true
checking for CFLAGS recommended by Perl... -DPERL_USE_SAFE_PUTENV -U__STRICT_ANSI__ -D_GNU_SOURCE -march=x86-64 -mtune=generic -O2 -pipe -fwrapv -fno-strict-aliasing -fstack-protector-strong
checking for CFLAGS to compile embedded Perl... -DPERL_USE_SAFE_PUTENV
checking for flags to link embedded Perl... no
configure: error: could not determine flags for linking embedded Perl.
This probably means that ExtUtils::Embed or ExtUtils::MakeMaker is not
installed.

If I just include perl.h from a test file with gcc using the above flags it
fails to compile:
$ echo '#include <perl.h>' > test.c
$ gcc -DPERL_USE_SAFE_PUTENV -U__STRICT_ANSI__ -D_GNU_SOURCE -march=x86-64 -mtune=generic -O2 -pipe -fwrapv -fno-strict-aliasing -fstack-protector-strong test.c -c -I /c/dev/msys64/usr/lib/perl5/core_perl/CORE
In file included from test.c:1:
C:/dev/msys64/usr/lib/perl5/core_perl/CORE/perl.h:1003:13: fatal error: sys/wait.h: No such file or directory
 1003 | #   include <sys/wait.h>

and ldopts bleats

$ perl -MExtUtils::Embed -e ldopts
Warning (mostly harmless): No library found for -lpthread
Warning (mostly harmless): No library found for -ldl
   -Wl,--enable-auto-import -Wl,--export-all-symbols -Wl,--enable-auto-image-base -fstack-protector-strong  -L/usr/lib/perl5/core_perl/CORE -lperl -lcrypt

Greetings,

Andres Freund

---

From: John Naylor <[email-masked]>
Date: Tue, 12 Oct 2021 15:55:22 -0400

On Tue, Oct 12, 2021 at 4:37 AM Andres Freund <[email-masked]> wrote:

[Meson prototype]

The build code looks pretty approachable for someone with no prior
exposure, and feels pretty nice when running it (I couldn't get a build
working but I'll leave that aside for now).

> gcc-on-hpux support, a chunk more to add support for the proprietary

That would also have to work for all the dependencies, which were displayed
to me as:

ninja, gdbm, ca-certificates, [email-masked], readline, sqlite and [email-masked]

Also, could utility makefile targets be made to work? I'm thinking in
particular of update-unicode and reformat-dat-files, for example.

--
John Naylor
EDB: http://www.enterprisedb.com

---

From: Andrew Dunstan <[email-masked]>
Date: Tue, 12 Oct 2021 16:02:14 -0400

> If I just include perl.h from a test file with gcc using the above flags it

You need to build against a native perl, like Strawberry or ActiveState.
(I have had mixed success with Strawberry) You do that by putting a path
to it at the start of the PATH. The wrinkle in this is that you need
prove to point to one that understands virtual paths. So you do
something like this:

PATH="/c/perl/bin:$PATH" PROVE=/bin/core_perl/prove configure ...

cheers

andrew

---

From: Andres Freund <[email-masked]>
Date: Tue, 12 Oct 2021 13:42:56 -0700

Hi,

On 2021-10-12 16:02:14 -0400, Andrew Dunstan wrote:
> (I have had mixed success with Strawberry)

Do you understand why that is needed?

> You do that by putting a path to it at the start of the PATH. The wrinkle in
> this is that you need prove to point to one that understands virtual
> paths. So you do something like this:
> 
> 

Oh my.

I'll try that later... I wonder if we could make this easier from our side?
This is a lot of magic to know.

Greetings,

Andres Freund

---

From: Andres Freund <[email-masked]>
Date: Tue, 12 Oct 2021 13:59:45 -0700

Hi,

On 2021-10-12 15:55:22 -0400, John Naylor wrote:
> exposure, and feels pretty nice when running it

That's part of what attracted me...

> (I couldn't get a build working but I'll leave that aside for now).

If you want to do that separately, I'll try to fix it.

> ninja, gdbm, ca-certificates, [email-masked], readline, sqlite and [email-masked]

meson does depend on ninja (to execute the build) and of course python. But
the rest should be optional dependencies. ninja builds without any
dependencies as long as you don't change its parser sources. python builds on
aix, hpux etc.

Not sure what way gdbm [email-masked] and sqlite are pulled in? I assume readline
is for python...

> particular of update-unicode and reformat-dat-files, for example.

Yes, that shouldn't be a problem. You can run arbitrary code in targets
(there's plenty need for that already in what I have so far).

Greetings,

Andres Freund

---

From: Josef Šimánek <[email-masked]>
Date: Wed, 13 Oct 2021 01:19:27 +0200

> What OS / distribution / version is this?

Fedora 34 (64 bit)

> with a version of gssapi that we're not compatible with.

Yes, I can.

> You should be able to get past this by disabling gss using meson configure

I tried to clean and start from scratch, but I'm getting different
error probably related to wrongly configured JIT (LLVM wasn't found
during meson setup). I'll debug on my side to provide more info.

Whole build error could be found at https://pastebin.com/hCFqcPvZ.
Setup log could be found at https://pastebin.com/wjbE1w56.

---

From: Andres Freund <[email-masked]>
Date: Tue, 12 Oct 2021 16:54:35 -0700

Hi,

> during meson setup). I'll debug on my side to provide more info.

../src/backend/jit/jit.c:91:73: error: ‘DLSUFFIX’ undeclared (first use in this function)
   91 |         snprintf(path, MAXPGPATH, "%s/%s%s", pkglib_path, jit_provider, DLSUFFIX);
      |                                                                         ^~~~~~~~

This *very* likely is related to building in a source tree that also contains
a "non-meson" build "in place". The problem is that the meson build picks up
the pg_config.h generated by ./configure in the "normal" build, rather than
the one meson generated itself.

You'd need to execute make distclean or such, or use a separate git checkout.

I forgot about this issue because I only ever build postgres from outside the
source-tree (by invoking configure from a separate directory), so there's
never build products in it. I think at least I need to make the build emit a
warning / error if there's a pg_config.h in the source tree...

This is the part of the jit code that's built regardless of llvm availability
- you'd get the same error in a few other places unrelated to jit.

Greetings,

Andres Freund

---

From: Andres Freund <[email-masked]>
Date: Tue, 12 Oct 2021 18:03:04 -0700

Hi,

> This is a lot of magic to know.

I managed to get this working. At first it failed because I don't have
pexports - it's not available inside msys as far as I could tell. And seems to
be unmaintained. But replacing pexports with gendef fixed that.

There's this comment in src/pl/plperl/GNUmakefile

# Perl on win32 ships with import libraries only for Microsoft Visual C++,
# which are not compatible with mingw gcc. Therefore we need to build a
# new import library to link with.

but I seem to be able to link fine without going through that song-and-dance?

Greetings,

Andres Freund

---

From: Daniel Gustafsson <[email-masked]>
Date: Wed, 13 Oct 2021 13:54:10 +0200

> On 12 Oct 2021, at 21:01, Andres Freund <[email-masked]> wrote:

> output quality suffering too much.

I added a --tap option for TAP output to pg_regress together with Jinbao Chen
for giggles and killing some time a while back.  It's not entirely done and
sort of PoC, but most of it works.  Might not be of interest here, but in case
it is I've refreshed it slightly and rebased it.  There might be better ways to
do it, but the aim was to make the diff against the guts of pg_regress small
and instead extract output functions for the different formats.

It omits the test timings, but that could be added either as a diagnostic line
following each status or as a YAML block in TAP 13 (the attached is standard
TAP, not version 13 but the change would be trivial).

If it's helpful and there's any interest for this I'm happy to finish it up now.

One thing that came out of this, is that we don't really handle the ignored
tests in the way the code thinks it does for normal output, the attached treats
ignored tests as SKIP tests.

--
Daniel Gustafsson		https://vmware.com/

---

From: Andrew Dunstan <[email-masked]>
Date: Wed, 13 Oct 2021 08:55:38 -0400

> but I seem to be able to link fine without going through that song-and-dance?

It looks like you're not building a native postgres, but rather one
targeted at msys. To build one that's native (i.e. runs without any
presence of msys) you need to do these things before building:

    MSYSTEM=MINGW64
    MSYSTEM_CHOST=x86_64-w64-mingw32
    PATH="/mingw64/bin:$PATH"

pexports will be in the resulting path, and the build will use the
native compiler.

You can use fairywren's config as a guide.

cheers

andrew

---

From: John Naylor <[email-masked]>
Date: Wed, 13 Oct 2021 11:51:03 -0400

> If you want to do that separately, I'll try to fix it.

Okay, I pulled the latest commits and tried again:

[51/950] Compiling C object
src/interfaces/libpq/libpq.5.dylib.p/fe-connect.c.o
FAILED: src/interfaces/libpq/libpq.5.dylib.p/fe-connect.c.o
ccache cc -Isrc/interfaces/libpq/libpq.5.dylib.p -Isrc/interfaces/libpq
-I../src/interfaces/libpq -Isrc/port -I../src/port -Isrc/include
-I../src/include
-I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/LDAP.framework/Headers
-I/usr/local/opt/readline/include -I/usr/local/opt/gettext/include
-I/usr/local/opt/zlib/include -I/usr/local/opt/openssl/include
-fcolor-diagnostics -Wall -Winvalid-pch -Wextra -O0 -g -isysroot
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -fno-strict-aliasing
-fwrapv -Wmissing-prototypes -Wpointer-arith -Werror=vla -Wendif-labels
-Wmissing-format-attribute -Wformat-security -Wdeclaration-after-statement
-Wno-unused-command-line-argument -Wno-missing-field-initializers
-Wno-sign-compare -Wno-unused-parameter -msse4.2
-F/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/LDAP.framework
-DFRONTEND -MD -MQ src/interfaces/libpq/libpq.5.dylib.p/fe-connect.c.o -MF
src/interfaces/libpq/libpq.5.dylib.p/fe-connect.c.o.d -o
src/interfaces/libpq/libpq.5.dylib.p/fe-connect.c.o -c
../src/interfaces/libpq/fe-connect.c
In file included from ../src/interfaces/libpq/fe-connect.c:72:
In file included from
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/LDAP.framework/Headers/ldap.h:1:

[the last line is repeated a bunch of times, then...]

/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/LDAP.framework/Headers/ldap.h:1:10:
error: #include nested too deeply
#include <ldap.h>
         ^

Then the expected "undeclared identifier" errors that would arise from a
missing header. I tried compiling --with-ldap with the Make build, and only
got warnings about deprecated declarations -- that build completed.

I tried disabling ldap with the Meson build but I'll spare the details of
what went wrong there in case I did something wrong, so we can take things
one step at a time.

> Not sure what way gdbm [email-masked] and sqlite are pulled in? I assume
readline

Hmm, weird.

---

From: Andres Freund <[email-masked]>
Date: Wed, 13 Oct 2021 09:37:21 -0700

Hi,

> [the last line is repeated a bunch of times, then...]

Oh. I actually saw that on CI at some point... That one is definitely
odd. Currently CI for OSX builds like

    - brew install make coreutils ccache icu4c lz4 tcl-tk openldap
    - brew install meson ninja [email-masked]
..
        PKG_CONFIG_PATH="/usr/local/opt/openssl/lib/pkgconfig:$PKG_CONFIG_PATH"
        PKG_CONFIG_PATH="/usr/local/opt/icu4c/lib/pkgconfig:$PKG_CONFIG_PATH"
        PKG_CONFIG_PATH="/usr/local/opt/openldap/lib/pkgconfig:$PKG_CONFIG_PATH"

        export PKG_CONFIG_PATH

        meson setup --buildtype debug -Dcassert=true -Dssl=openssl build

but I set that up knowing little about macos.

For the autoconf build CI currently does something similar via
        LIBS="/usr/local/lib:$LIBS"
        INCLUDES="/usr/local/include:$INCLUDES"
...
        LIBS="/usr/local/opt/openldap/lib:$LIBS"
        INCLUDES="/usr/local/opt/openldap/include:$INCLUDES"
        ...
          --with-includes="$INCLUDES" \
          --with-libs="$LIBS" \

are you doing something like that? Or does it work for you without? I vaguely
recall hitting a similar problem as you report when not passing
/usr/local/... to configure.

> i tried disabling ldap with the meson build but i'll spare the details of
> what went wrong there in case i did something wrong, so we can take things
> one step at a time.

you can change it for an existing builddir with
meson configure -dldap=disabled or when setting up a new builddir by passing
-dldap=disabled at that time.

> > meson does depend on ninja (to execute the build) and of course python.
> but
> > not sure what way gdbm [email-masked] and sqlite are pulled in? i assume
> readline
> > is for python...
> 
> Hmm, weird.

They're homebrew python deps: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/[email-masked]#L28
which are optional things enabled explicitly:
https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/[email-masked]#L123

Greetings,

Andres Freund

---

From: John Naylor <[email-masked]>
Date: Wed, 13 Oct 2021 13:19:36 -0400

> /usr/local/... to configure.

I didn't do anything like that for the autoconf build. I have in the past
done things retail, like

--with-icu ICU_CFLAGS='-I/usr/local/opt/icu4c/include/'
ICU_LIBS='-L/usr/local/opt/icu4c/lib/ -licui18n -licuuc -licudata'

> -dldap=disabled at that time.

Somehow our emails got lower-cased down here, but I tried it with capital D:

meson configure -Dldap=disabled

inside the build dir and got this:

../meson.build:278:2: ERROR: Tried to assign the invalid value "None" of
type NoneType to variable.

Line 278 is

  ldap_r = ldap = dependency('', required : false)

---

From: Andres Freund <[email-masked]>
Date: Wed, 13 Oct 2021 10:26:54 -0700

Hi,

>     PATH="/mingw64/bin:$PATH"

I had a config equivalent to this (slight difference in PATH, but the same gcc
being picked), and I just verified that it still works if I set up PATH like
that.  I get a working plperl out of it. Without msys on PATH or such.

where perl526.dll
C:\perl\strawberry-5.26.3.1-64bit\perl\bin\perl526.dll

dumpbin /imports 'C:/Users/anfreund/src/pg-meson/build-mingw/tmp_install/lib/plperl.dll'|grep dll

Dump of file C:\Users\anfreund\src\pg-meson\build-mingw\tmp_install\lib\plperl.dll
    KERNEL32.dll
    msvcrt.dll
    perl526.dll

dumpbin /imports .\build-mingw\tmp_install\bin\postgres.exe|grep dll
    ADVAPI32.dll
    KERNEL32.dll
    msvcrt.dll
    Secur32.dll
    WLDAP32.dll
    WS2_32.dll

do $$elog(NOTICE, "blob");$$ language plperl;
NOTICE:  blob
DO

To me this looks like it's a plperl built without the import file recreation,
without being msys dependent?

> pexports will be in the resulting path, and the build will use the

I don't see pexports anywhere in the msys installation. I can see it available
on sourceforge, and I see a few others asking where to get it from in the
context of msys, and being pointed to manually downloading it.

Seems like we should consider using gendef instead of pexports, given it's
available in msys?

$ pacman -Fy
$ pacman -F gendef.exe
...
mingw64/mingw-w64-x86_64-tools-git 9.0.0.6316.acdc7adc9-1 (mingw-w64-x86_64-toolchain) [installed]
    mingw64/bin/gendef.exe
..
$ pacman -F pexports.exe
$ pacman -Fx pexports
<bunch of packages containing smtpexports.h>

Greetings,

Andres Freund

---

From: Andres Freund <[email-masked]>
Date: Wed, 13 Oct 2021 10:42:46 -0700

Hi,

> I didn't do anything like that for the autoconf build. I have in the past

I'll try to see how this works / what causes the breakage.

> Somehow our emails got lower-cased down here, but I tried it with capital D:

:)

>   ldap_r = ldap = dependency('', required : false)

Oops, I broke that when trying to clean things up. I guess I write too much C
;). It needs to be two lines.

I pushed the fix for that.

Greetings,

Andres Freund

---

From: John Naylor <[email-masked]>
Date: Wed, 13 Oct 2021 14:40:19 -0400

> I pushed the fix for that.

Ok great, it builds now! :-) Now something's off with dynamic loading.
There are libraries in ./tmp_install/usr/local/lib/ but apparently initdb
doesn't know to look for them there:

$ cat /Users/john/pgdev/meson/build/testrun/main/pg_regress/log/initdb.log
dyld: Library not loaded: /usr/local/lib/libpq.5.dylib
  Referenced from:
/Users/john/pgdev/meson/build/tmp_install/usr/local/bin/initdb
  Reason: image not found

---

From: Andrew Dunstan <[email-masked]>
Date: Wed, 13 Oct 2021 16:06:32 -0400

> context of msys, and being pointed to manually downloading it.

Weird. fairywren has it, which means that it must have been removed from
the packages at some stage, fairly recently as fairywren isn't that old.
I just confirmed the absence on a 100% fresh install.

It is in Strawberry's c/bin directory.

> Seems like we should consider using gendef instead of pexports, given it's

Yeah. It's missing on my ancient msys animal (frogmouth), but it doesn't
build --with-perl.

jacana seems to have it.

If you prep a patch I'll test it.

cheers

andrew

---

From: Andres Freund <[email-masked]>
Date: Wed, 13 Oct 2021 14:46:31 -0700

Hi,

> If you prep a patch I'll test it.

Well, right now I'm wondering if the better fix is to just remove the whole
win32 block. I don't know how far back, but afaict it's not needed. Seems to
have been needed for narwhal at some point, according to 02b61dd08f99. But
narwhal is long dead.

Greetings,

Andres Freund

---

From: Josef Šimánek <[email-masked]>
Date: Wed, 13 Oct 2021 23:58:12 +0200

>    91 |         snprintf(path, MAXPGPATH, "%s/%s%s", pkglib_path, jit_provider, DLSUFFIX);

Hello, thanks for the hint. I can finally build using meson and run
regress tests.

The only problem I do have currently is auto-detection of perl. I'm
getting error related to missing "Opcode.pm". PERL is autodetected and
enabled (https://pastebin.com/xfRRrDcU).

I do get the same error when I enforce perl for current master build
(./configure --with-perl). Using ./configure with perl autodetection
skips plperl extension on my system.

Disabling perl manually for meson build (meson setup build
--reconfigure --buildtype debug -Dperl=disabled) works for me.

---

From: Andrew Dunstan <[email-masked]>
Date: Wed, 13 Oct 2021 19:11:10 -0400

> have been needed for narwhal at some point, according to 02b61dd08f99. But
Ok, I'll test it out.

cheers

andrew

---

From: Thomas Munro <[email-masked]>
Date: Thu, 14 Oct 2021 17:27:08 +1300

On Thu, Oct 14, 2021 at 4:51 AM John Naylor
<[email-masked]> wrote:
> /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/LDAP.framework/Headers/ldap.h:1:10: error: #include nested too deeply
> #include <ldap.h>
>          ^

I vaguely recall that PostgreSQL should build OK against Apple's copy
of OpenLDAP.  That recursive include loop is coming from a "framework"
header that contains just a couple of lines like #include <ldap.h> to
try to include the real header, which should also be in the include
path, somewhere like
/Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/ldap.h.
I think we'd need to figure out where that
-I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/LDAP.framework/Headers
directive is coming from and get rid of it, so we can include the real
header directly.

---

From: Dagfinn Ilmari Mannsåker <[email-masked]>
Date: Thu, 14 Oct 2021 14:14:39 +0100

> enabled (https://pastebin.com/xfRRrDcU).

Your Perl (not PERL) installation seems to be incomplete. Opcode.pm is a
core module, and should be in /usr/lib64/perl5, judging by the paths in
the error message.

Which OS is this?  Some Linux distributions have separate packages for
the interpreter itself and the included modules, and the packages can be
named confusingly.  E.g. on older Redhat/Fedora versions you have to
install the 'perl-core' package to get all the modules, 'perl' is just
the interpreter and the bare minimum set of strictily necessary modules.

They've fixed this in recent versions (Fedora 34 and Redhat 8, IIRC), so
that 'perl' gives you the hole bundle, and 'perl-interpeter' is the
minimal one.

- ilmari

---

From: Josef Šimánek <[email-masked]>
Date: Thu, 14 Oct 2021 15:19:18 +0200

> that 'perl' gives you the hole bundle, and 'perl-interpeter' is the

I'm using Fedora 34 and I still see perl-Opcode.x86_64 as a separate
package. Anyway it behaves differently with autoconf tools and the
meson build system. Is perl disabled by default in the current build
system?

---

From: Alvaro Herrera <[email-masked]>
Date: Thu, 14 Oct 2021 10:29:42 -0300

> meson build system. Is perl disabled by default in the current build

Yes, you have to use --with-perl in order to get it.

-- 
Álvaro Herrera              Valdivia, Chile  —  https://www.EnterpriseDB.com/
"Puedes vivir sólo una vez, pero si lo haces bien, una vez es suficiente"

---

From: Dagfinn Ilmari Mannsåker <[email-masked]>
Date: Thu, 14 Oct 2021 14:32:49 +0100

Josef Šimánek <[email-masked]> writes:

> package.`

Yes, it's a separate package, but the 'perl' package depends on all the
core module packages, so installing that should fix things.  You appear
to only have 'perl-interpreter' installed.

> Anyway it behaves differently with autoconf tools and the meson build
> system. Is perl disabled by default in the current build system?

configure doesn't auto-detect any optional features, they have to be
explicitly enabled using --with-foo switches.

- ilmari

---

From: Andres Freund <[email-masked]>
Date: Thu, 14 Oct 2021 10:24:12 -0700

Hi,

> > meson build system. Is perl disabled by default in the current build

Hm, so it seems we should make the test separately verify that perl -M{Opcode,
ExtUtils::Embed, ExtUtils::ParseXS} doesn't fail, so that we can fail perl
detection with a useful message?

> Yes, you have to use --with-perl in order to get it.

With the meson prototype I set most optional features to "auto", except for
LLVM, as that increases compile times noticeably.

For configure we didn't/don't want to do much auto-detection, because that
makes life harder for distributors. But meson has one switch controlling all
features set to 'auto' and not explicitly enabled/disabled:
  --auto-features {enabled,disabled,auto}                               Override value of all 'auto' features (default: auto).
so the argument doesn't apply to the same degree there. We could default
auto-features to something else too.

There were two other reasons:

1) I got tired of needing to disable zlib, readline to be able to build on
   windows.
2) Exercising all the dependency detection / checking seems important at this
   stage

Greetings,

Andres Freund

---

From: Andres Freund <[email-masked]>
Date: Thu, 14 Oct 2021 10:26:38 -0700

Hi,

> Hello, thanks for the hint. I can finally build using meson and run

I yesterday pushed code that should detect this case (with an error). Should
now detect the situation both when you first run configure in tree, and then
meson, and the other way round (by the dirty hack of ./configure touch'ing
meson.build at the end for in-tree builds).

> --reconfigure --buildtype debug -Dperl=disabled) works for me.

Yay, thanks for testing!

Greetings,

Andres Freund

---

From: John Naylor <[email-masked]>
Date: Thu, 14 Oct 2021 15:14:16 -0400

>   Reason: image not found

After poking a bit more, this only happens when trying to run the tests. If
I specify a prefix, I can install, init, and start the server just fine, so
that much works.

---

From: Josef Šimánek <[email-masked]>
Date: Thu, 14 Oct 2021 21:29:38 +0200

> to only have 'perl-interpreter' installed.

You're right. Installing "perl" or "perl-Opcode" manually fixes this
problem. Currently I only have "perl-interpreter" installed.

---

From: Josef Šimánek <[email-masked]>
Date: Thu, 14 Oct 2021 21:36:52 +0200

> detection with a useful message?

I can confirm "perl -MOpcode" fails. ExtUtils::Embed and
ExtUtils::ParseXS are present. Looking at the local system history of
perl-interpreter package, it seems to be installed by default on
Fedora 34. Friendly error message would be welcomed.

> 2) Exercising all the dependency detection / checking seems important at this

Clear, thanks for the info.

---

From: Andres Freund <[email-masked]>
Date: Thu, 14 Oct 2021 13:34:11 -0700

Hi,

>I specify a prefix, I can install, init, and start the server just fine, so

Is this a Mac with SIP enabled? The Mac CI presumably has that disabled, which is why I didn't see this issue there. Probably need to implement whatever Tom figured out to do about that for the current way of running tests.

Andres
-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

---

From: John Naylor <[email-masked]>
Date: Thu, 14 Oct 2021 16:54:34 -0400

On Thu, Oct 14, 2021 at 4:34 PM Andres Freund <[email-masked]> wrote:

> Is this a Mac with SIP enabled? The Mac CI presumably has that disabled,
which is why I didn't see this issue there. Probably need to implement
whatever Tom figured out to do about that for the current way of running
tests.

System Information says it's disabled. Running "csrutil status" complains
of an unsupported configuration, which doesn't sound good, so I should
probably go fix that independent of anything else. :-/

---

From: John Naylor <[email-masked]>
Date: Thu, 14 Oct 2021 17:16:15 -0400

I wrote:

> > Is this a Mac with SIP enabled? The Mac CI presumably has that
disabled, which is why I didn't see this issue there. Probably need to
implement whatever Tom figured out to do about that for the current way of
running tests.

Looking online, I wonder if the "unsupported" message might be overly
cautious. In any case, I do remember turning something off to allow a
debugger to run. Here are all the settings, in case it matters:

Apple Internal: disabled
Kext Signing: enabled
Filesystem Protections: enabled
Debugging Restrictions: disabled
DTrace Restrictions: enabled
NVRAM Protections: enabled
BaseSystem Verification: enabled

---

From: Sergey Shinderuk <[email-masked]>
Date: Fri, 15 Oct 2021 00:41:20 +0300

Hi,

On 14.10.2021 23:54, John Naylor wrote:
> On Thu, Oct 14, 2021 at 4:34 PM Andres Freund <[email-masked] 
> <mailto:[email-masked]>> wrote:
> 
> implement whatever Tom figured out to do about that for the current way 
> of running tests.
> 
> System Information says it's disabled. Running "csrutil status" 
> complains of an unsupported configuration, which doesn't sound good, so 
> I should probably go fix that independent of anything else. :-/

Maybe you could check that DYLD_LIBRARY_PATH is working for you?

% DYLD_FALLBACK_LIBRARY_PATH= 
DYLD_LIBRARY_PATH=./tmp_install/usr/local/lib 
./tmp_install/usr/local/bin/psql --version
psql (PostgreSQL) 15devel

Without DYLD_LIBRARY_PATH I get the error, as expected:

% DYLD_FALLBACK_LIBRARY_PATH= ./tmp_install/usr/local/bin/psql --version
dyld: Library not loaded: /usr/local/lib/libpq.5.dylib
   Referenced from: 
/Users/shinderuk/src/postgres-meson/build/./tmp_install/usr/local/bin/psql
   Reason: image not found

I add "DYLD_FALLBACK_LIBRARY_PATH=" because otherwise dyld falls back to 
/usr/lib/libpq.5.dylib provided by Apple (I am testing on Catalina).

% DYLD_PRINT_LIBRARIES=1 ./tmp_install/usr/local/bin/psql --version 2>&1 
| grep libpq
dyld: loaded: <4EDF735E-2104-32AD-BE7B-B400ABFCF57C> /usr/lib/libpq.5.dylib

Regards,

-- 
Sergey Shinderuk		https://postgrespro.com/

---

From: Tom Lane <[email-masked]>
Date: Thu, 14 Oct 2021 17:48:23 -0400

> BaseSystem Verification: enabled

I remember having seen that report too, after some previous software
upgrade that had started from a "SIP disabled" status.  I'm mostly
guessing here, but my guess is that

(a) csrutil only considers the all-enabled and all-disabled states
of these individual flags to be "supported" cases.

(b) some one or more of these flags came along in a macOS update,
and if you did the update starting from a "disabled" state, you
nonetheless ended up with the new flags enabled, leading to the
mixed state that csrutil complains about.

I've lost count of the number of times I've seen macOS updates
be sloppy about preserving non-default settings, so I don't find
theory (b) to be even slightly surprising.

Whether the mixed state is actually problematic in any way,
I dunno.  I don't recall having had any problems before noticing
that that was what I had.

			regards, tom lane

---

From: Tom Lane <[email-masked]>
Date: Thu, 14 Oct 2021 18:00:49 -0400

> Is this a Mac with SIP enabled? The Mac CI presumably has that disabled, which is why I didn't see this issue there. Probably need to implement whatever Tom figured out to do about that for the cur…

AFAIR the only cases we've made work are

(1) disable SIP

(2) avoid the need for (1) by always doing "make install" before
"make check".

Peter E. did some hacking towards another solution awhile ago,
but IIRC it involved changing the built binaries, and I think
we concluded that the benefits didn't justify that.

			regards, tom lane

---

From: Tom Lane <[email-masked]>
Date: Thu, 14 Oct 2021 18:08:58 -0400

> detection with a useful message?

Our existing policy is that we should check this at configure time,
not later.  Since plperl won't work at all without Opcode, it seems
appropriate to add a check there if you say --with-perl.  I wasn't
aware that Red Hat had unbundled that from the minimal perl
installation :-(.

OTOH, if they've not unbundled ExtUtils::Embed or ExtUtils::ParseXS,
I doubt it's worth the configure cycles to check for those separately.

			regards, tom lane

---

From: Andrew Dunstan <[email-masked]>
Date: Thu, 14 Oct 2021 18:19:45 -0400

>> have been needed for narwhal at some point, according to 02b61dd08f99. But

confirmed that jacana doesn't need this code to build or test plperl
(all I did was change the test from win32 to win32x). There would still
be work to do to fix the contrib bool_plperl, jsonb_plperl and
hstore_plperl modules.

cheers

andrew

---

From: Thomas Munro <[email-masked]>
Date: Fri, 15 Oct 2021 11:23:00 +1300

> we concluded that the benefits didn't justify that.

Yeah, by now there are lots of useful blogs from various projects
figuring out that you can use the install_name_tool to adjust the
paths it uses to be absolute or relative to certain magic words, like
@executable_path/../lib/blah.dylib, which is tempting, but...
realistically, for serious hacking on a Mac, SIP is so annoying that
it isn't the only reason you'll want to turn it off: it stops
dtrace/dtruss/... from working, and somehow prevents debuggers from
working when you've ssh'd in from a remote machine with a proper
keyboard, and probably more things that I'm forgetting.

I wish I could find the Xnu source that shows exactly how and when the
environment is suppressed in this way to understand better, but it
doesn't jump out of Apple's github; maybe it's hiding in closed source
machinery...

---

From: Tom Lane <[email-masked]>
Date: Thu, 14 Oct 2021 18:40:06 -0400

> doesn't jump out of Apple's github; maybe it's hiding in closed source

I recall that we figured out awhile ago that the environment gets trimmed
when make (or whatever) executes some command via the shell; seemingly,
Apple has decided that /bin/sh is a security-critical program that mustn't
be run with a non-default DYLD_LIBRARY_PATH.  Dunno if that helps you
find where the damage is done exactly.

(The silliness of this policy, when you pair it with the fact that they
don't reset PATH at the same time, seems blindingly obvious to me.  But
apparently not to Apple.)

			regards, tom lane

---

From: Andres Freund <[email-masked]>
Date: Thu, 14 Oct 2021 15:55:09 -0700

Hi,

> (2) avoid the need for (1) by always doing "make install" before

Ah, I thought it was more than that. In that case, John, does meson's test
succeed after you did the "proper" install? Assuming it's in a path that's
allowed to provide shared libraries?

Greetings,

Andres Freund

---

From: Tom Lane <[email-masked]>
Date: Thu, 14 Oct 2021 19:04:27 -0400

> find where the damage is done exactly.

BTW, here's the evidence for this theory:

[tgl@pro ~]$ cat checkenv.c
#include <stdio.h>
#include <stdlib.h>

int
main(int argc, char **argv)
{
  char *pth = getenv("DYLD_LIBRARY_PATH");

  if (pth)
    printf("DYLD_LIBRARY_PATH = %s\n", pth);
  else
    printf("DYLD_LIBRARY_PATH is unset\n");

  return 0;
}
[tgl@pro ~]$ gcc checkenv.c
[tgl@pro ~]$ ./a.out
DYLD_LIBRARY_PATH is unset
[tgl@pro ~]$ export DYLD_LIBRARY_PATH=/Users/tgl/pginstall/lib
[tgl@pro ~]$ ./a.out
DYLD_LIBRARY_PATH = /Users/tgl/pginstall/lib
[tgl@pro ~]$ sh -c ./a.out
DYLD_LIBRARY_PATH is unset
[tgl@pro ~]$ ./a.out
DYLD_LIBRARY_PATH = /Users/tgl/pginstall/lib
[tgl@pro ~]$ bash -c ./a.out
DYLD_LIBRARY_PATH is unset

You have to check the environment using an "unprivileged" program.
If you try to examine the environment using, say, "env", you will get
very misleading results.  AFAICT, /usr/bin/env is *also* considered
security-critical, because I cannot get it to ever report that
DYLD_LIBRARY_PATH is set.

Hmm ... /usr/bin/perl seems to act the same way.  It can see
ENV{'PATH'} but not ENV{'DYLD_LIBRARY_PATH'}.

This may indicate that they've applied this policy on a blanket
basis to everything in /bin and /usr/bin (and other system
directories, maybe), rather than singling out the shell.

			regards, tom lane

---

From: Andres Freund <[email-masked]>
Date: Thu, 14 Oct 2021 16:15:01 -0700

Hi,

> keyboard, and probably more things that I'm forgetting.

Meson has support for using install_name_tool to remove "build time" rpaths
and set "install time" rpaths during the installation process - which uses
install_name_tool on mac.

If, and perhaps that's too big an if, relative rpaths actually work despite
SIP, it might be worth setting a relative install_rpath, because afaict that
should then work both for a "real" installation and our temporary test one.

If absolute rpaths are required, it'd make the process a bit more expensive,
because we'd probably need to change a configure time option during the temporary
install. No actual rebuilds would be required, but still.

Greetings,

Andres Freund

---

From: Tom Lane <[email-masked]>
Date: Thu, 14 Oct 2021 19:23:58 -0400

---

From: John Naylor <[email-masked]>
Date: Thu, 14 Oct 2021 19:27:17 -0400

> allowed to provide shared libraries?

Oh, it can act like installcheck? [checks] Yep, "meson test" ran fine (*).
It still ran the temp install first, but in any case it worked. The full
"configure step"  was

meson setup build --buildtype debug -Dldap=disabled -Dcassert=true
-Dprefix=$(pwd)/inst

* (all passed but skipped subscription/t/012_collation.pl)

---

From: Andres Freund <[email-masked]>
Date: Thu, 14 Oct 2021 16:38:02 -0700

Hi,

On 2021-10-14 18:08:58 -0400, Tom Lane wrote:
> not later.

Yea, I was thinking of configure (and meson's equivalent) as well.

> Since plperl won't work at all without Opcode, it seems

On debian the perl binary, with a sparse set of modules is in
perl-base. ExtUtils::Embed and ExtUtils::ParseXS are in
perl-modules-x.yy. Whereas Opcode is in libperlx.yy. But libperlx.yy depends
on perl-modules-x.yy so I guess an Opcode.pm check would suffice.

Seems we can just check all of them at once with with something like

perl -MOpcode -MExtUtils::Embed -MExtUtils::ParseXSNotAvailable -e ''
Can't locate ExtUtils/ParseXSNotAvailable.pm in @INC (you may need to install the ExtUtils::ParseXS3 module) (@INC contains: /home/andres/bin/perl5/lib/perl5/x86_64-linux-gnu-thread-multi /home/andres/bin/perl5/lib/perl5 /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.32.1 /usr/local/share/perl/5.32.1 /usr/lib/x86_64-linux-gnu/perl5/5.32 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl-base /usr/lib/x86_64-linux-gnu/perl/5.32 /usr/share/perl/5.32 /usr/local/lib/site_perl).

Greetings,

Andres Freund

---

From: Tom Lane <[email-masked]>
Date: Thu, 14 Oct 2021 19:51:03 -0400

> Yea, I was thinking of configure (and meson's equivalent) as well.

Ah, sorry, I misunderstood what you meant by "test".

			regards, tom lane

---

From: Andres Freund <[email-masked]>
Date: Thu, 14 Oct 2021 17:02:05 -0700

Hi,

On 2021-10-14 19:27:17 -0400, John Naylor wrote:
> It still ran the temp install first, but in any case it worked.

As far as I understand Tom, our normal make check only works on OSX if
previously you ran make install. Which will have installed libpq into the
"proper" install location. Because all our binaries will, by default, have an
rpath to the library directory embedded, that then allows binaries in the
temporary install to work. But using the wrong libpq - which most of the time
turns out to be harmless, because libpq doesn't change that rapidly.

> * (all passed but skipped subscription/t/012_collation.pl)

That test requires ICU, so that's fine. I guess we could prevent the test from
being executed in the first place, but I don't think we've done that for cases
where it's one specific test in a t/ directory, where others in the same
directory do not have such dependencies.

Greetings,

Andres Freund

---

From: Thomas Munro <[email-masked]>
Date: Fri, 15 Oct 2021 13:36:28 +1300

> directories, maybe), rather than singling out the shell.

Looks like it.  If I've found the right code here, it looks like where
any common-or-garden Unix runtime linker would ignore LD_LIBRARY_PATH
for a setuid binary, they've trained theirs to whack DYLD_*, and also
for code-signed and __RESTRICT-marked executables.

https://github.com/opensource-apple/dyld/blob/master/src/dyld.cpp#L1681

I suppose you could point SHELL at an unsigned copy of sh (codesign
--remove-signature, or something from brew/ports/x) so that GNU make
should respect, but I don't know how many other exec("/bin/sh") calls
might be hiding around the place (I guess perl calls system()?) and
might require some kind of LD_PRELOAD hackery... not much fun.

---

From: Tom Lane <[email-masked]>
Date: Thu, 14 Oct 2021 22:46:07 -0400

> https://github.com/opensource-apple/dyld/blob/master/src/dyld.cpp#L1681

Ugh.  That explains it, all right.

> might require some kind of LD_PRELOAD hackery... not much fun.

Yeah.  I thought about invoking everything via a small wrapper
that restores the correct setting of DYLD_LIBRARY_PATH.  We could
perhaps make that work for the invocations of test postmasters
and psqls from "make" and TAP scripts, but hacking up our code's
sundry uses of system(3) like that seems like it'd be very messy,
if feasible at all.

BTW, the POSIX spec explicitly discourages letting SHELL affect the
behavior of system(3), so I bet that wouldn't help.

			regards, tom lane

---

From: Andres Freund <[email-masked]>
Date: Thu, 14 Oct 2021 20:20:25 -0700

Hi,

> sundry uses of system(3) like that seems like it'd be very messy,

It does sound like using relative rpaths might be the thing we want - and like
they've been available since 10.5 or something.

Is there a reason we're using absolute rpaths on a bunch of platforms, rather
than relative ones, which'd allow relocation?

Greetings,

Andres Freund

---

From: Andres Freund <[email-masked]>
Date: Fri, 15 Oct 2021 11:50:30 -0700

Hi,

On 2021-10-14 19:23:58 -0400, Tom Lane wrote:
> From what we know so far, it seems like SIP wouldn't interfere with
> that (if it works at all).  I think what SIP desires to prevent is
> messing with a program's execution by setting DYLD_LIBRARY_PATH.
> As long as the program executable itself is saying where to find
> the library, I don't see why they should interfere with that.

Well, there's *some* danger with relative rpaths, because they might
accidentally be pointing somewhere non-existing and user-creatable. Not a huge
risk, but as you say:

> (Again, it seems blindingly stupid to forbid this while not blocking
> PATH or any of the other environment variables that have always affected
> execution.  But what do I know.)

these aren't necessarily carefuly weighed considerations :/

But it seems to work well from what I gather.

> It'd also put the kibosh on relocatable install trees, though I dunno how
> much people really care about that.

We currently use absolute rpaths, or something equivalent.

The reason that running tests on macos works is that we set the "install_name"
of shared libraries to the intended installed location, using an absolute
path:
    LINK.shared		= $(COMPILER) -dynamiclib -install_name '$(libdir)/lib$(NAME).$(SO_MAJOR_VERSION)$(DLSUFFIX)' $(version_link) $(exported_symbols_list) -multiply_defined suppress
which on macos means that all libraries linking to that dylib reference it
under that absolute path.

On most other platforms we set an absolute rpath to the installation
directory, which has an equivalent effect:
rpathdir = $(libdir)

It seems to work quite well to change our own references to libpq in binaries
/ shared libs to be relative, but to leave the install_name of the libraries
intact. In combination with adding an rpath of @loader_path/../lib/ to
binaries and @loader_path/ to shlibs, the install will re relocatable.

It doesn't work as well to actually have a non-absolute install_name for
libraries (e.g. @rpath/libpq.dylib), because then external code linking to
libpq needs to add an rpath to the installation to make it work.

The advantage of this approach over Peter's is that it's not temp-install
specific - due to the relative paths, it makes installations relocatable
without relying [DY]LD_LIBRARY_PATH.

On other unixoid systems this whole mess is simpler, because we can just add
$ORIGIN to shared libraries and $ORIGIN/../lib/ to binaries. We don't need to
leave some absolute path in the libraries themself intact.

Greetings,

Andres Freund

---

From: Andres Freund <[email-masked]>
Date: Fri, 15 Oct 2021 15:36:16 -0700

Hi,

> leave some absolute path in the libraries themself intact.

I implemented this for the meson build, and it seems to work nicely. The macos
part was harder than I hoped due to the install_name stuff, which meson
doesn't solve.

https://github.com/anarazel/postgres/commit/a35379c28989469cc4b701a8d7a22422e6302e09

After that the build directory is relocatale.

I don't immediately see a way to do this reasonably for the autoconf
build. We'd need a list of our own shared libraries from somewhere, and then
replace the references after building the objects?

Greetings,

Andres Freund

---

From: Andres Freund <[email-masked]>
Date: Fri, 15 Oct 2021 15:47:05 -0700

Hi,

> After that the build directory is relocatale.

Well, now that I think about it, it's still only relocatable in the sense that
postgres itself will continue to work. Outside code linking to e.g. libpq will
get the wrong path after relocation the source tree, due to the absolute
install_name.

But that doesn't seem solvable, unless we make the installed install_name to
be '@rpath/libpq...dylib' and require code linking to libpq to pass
-Wl,-rpath,/path/to/libpq when linking to libpq.

Greetings,

Andres Freund

---

From: Andres Freund <[email-masked]>
Date: Tue, 19 Oct 2021 11:34:19 -0700

Hi Tom,

> gcc-on-hpux support, a chunk more to add support for the proprietary

Tom, wrt HPUX on pa-risc, what are your thoughts there? IIRC we gave up
supporting HP's compiler on pa-risc a while ago.

As I said it'd probably not be too hard to add meson support for hpux on hppa,
it's probably just a few branches. But that'd require access somewhere. The
gcc compile farm does not have a hppa member anymore...

I did notice that gcc will declare hppa-hpux obsolete in gcc 12 and will
remove at some point:
"The hppa[12]*-*-hpux10* and hppa[12]*-*-hpux11* configurations targeting 32-bit PA-RISC with HP-UX have been obsoleted and will be removed in a future release."
https://gcc.gnu.org/gcc-12/changes.html

Greetings,

Andres Freund

---

From: Tom Lane <[email-masked]>
Date: Tue, 19 Oct 2021 15:22:15 -0400

> supporting HP's compiler on pa-risc a while ago.

Right.  I am still testing with gcc on HP-PA.  I'd kind of like to
keep it running just as an edge case for our spinlock support, but
I'm not sure that I want to do any huge amount of work on meson
to keep that going.

I do have a functioning OpenBSD installation on that machine, so
one alternative if the porting costs look too high is to replace
gaur with an OpenBSD animal.  However, last I checked, OpenBSD
was about half the speed of HPUX on that hardware, so I'm not
real eager to go that way.  gaur's already about the slowest
animal in the farm :-(

> gcc compile farm does not have a hppa member anymore...

If you've got an idea where to look, I could add that to my
to-do queue.

In any case, I don't think we need to consider HPUX as a blocker
for the meson approach.  The value-add from keeping gaur going
probably isn't terribly much.  I'm more concerned about the
effort involved in getting meson going on some other old animals,
such as prairiedog.

			regards, tom lane

---

From: John Naylor <[email-masked]>
Date: Tue, 19 Oct 2021 17:57:31 -0400

Hi,

I know this is still in the evaluation stage, but I did notice some
discrepencies in the Flex flags. With the attached patch, the read-only
data segment seems to match up pretty well now.

---

From: Andres Freund <[email-masked]>
Date: Tue, 19 Oct 2021 17:31:22 -0700

Hi,

> data segment seems to match up pretty well now.

Good catch. I think I just copied them around...

I wish we had a bit more consistency in the flags, so we could centralize
them. Seems there's no reason to not use -p -p and -b everywhere?

I also need to make meson use our flex wrapper for the relevant versions... I
can see the warning that'd be fixed by it on macos CI. Will do that and push
it out to my github repo together with your changes.

Thanks!

Andres

---

From: Andres Freund <[email-masked]>
Date: Tue, 19 Oct 2021 18:08:59 -0700

Hi,

> I'm not sure that I want to do any huge amount of work on meson

Makes sense.  While that does test an odd special case for our spinlock
implementation, it's also the only supported platform with that edge case, and
it seems extremely unlikely that there ever will be a new platform with such
odd/limited atomic operations.

> real eager to go that way.  gaur's already about the slowest

Yea, that doesn't sound enticing. Seems like we either should keep it running
on hp-ux or just drop parisc support?

> If you've got an idea where to look, I could add that to my to-do queue.

It might even just work. Looks like meson does have pa-risc detection. While
it doesn't have any specifically for hpux, it just falls back to python's
sys.platform in that case. python3 -c 'import sys;print(sys.platform)'

meson generates output for ninja to execute (basically a faster make that's
partially faster by being much less flexible. Intended to be output by more
user-friendly buildsystems ). Ninja can be built by a minimal python script,
or with cmake. The former doesn't seem to have hpux support, the latter does I
think.
https://github.com/ninja-build/ninja

So it could be interesting to see if ninja builds.

I've not taught the PG meson the necessary stuff for a 32 bit build. So
there's no point is trying whether meson works that much. I'll try to do that,
and let you know.

> I'm more concerned about the effort involved in getting meson going on some
> other old animals, such as prairiedog.

Yea, that's an *old* OS version. One version too old to have support for
@rpath, added in 10.5 :(. Is there a reason to run 10.4 specifically?
According to wikipedia 10.5 is the last version to support ppc.

Looks like python still supports building back to 10.4.

Greetings,

Andres Freund

---

From: Tom Lane <[email-masked]>
Date: Tue, 19 Oct 2021 21:10:02 -0400

> them. Seems there's no reason to not use -p -p and -b everywhere?

I don't think we care enough about performance of most of the scanners
to make them all backup-free, so -1 to that idea.

We could possibly replace the command line switches with %option
entries in the files themselves.  But I think the reason we haven't
done so for -b is that the Makefile still needs to know about it
so as to know what to do with the lex.backup output file.

			regards, tom lane

---

From: Tom Lane <[email-masked]>
Date: Tue, 19 Oct 2021 21:26:53 -0400

> According to wikipedia 10.5 is the last version to support ppc.

My notes say

    Currently running OSX 10.4.11 (last release of Tiger); although 10.5 Leopard
    supports PPCs, it refuses to install if CPU speed < 867MHz, well beyond the
    Cube's ability.  Wikipedia does suggest it's possible to run Leopard, but...
    https://en.wikipedia.org/wiki/Mac_OS_X_Leopard#Usage_on_unsupported_hardware

I'm not sure that I have install media for 10.5 anymore, either --- ISTR
some machine's CD drive failing and not letting me get the CD back out.
If I did have it, I don't think there'd be a way to update past 10.5.0
(surely Apple no longer has those updaters on-line?), so on the whole
I think that path is a nonstarter.

I do have 10.5 running on an old G4 PowerMac, but that machine is (a)
noisy (b) power-hungry and (c) getting flaky, so I'm uneager to spin up
a buildfarm animal on it.

As with the HPPA, a potential compromise is to spin up some newer
BSD-ish system on it.  I agree that OSX 10.4 is uninteresting as a
software platform, but I'd like to keep 32-bit PPC represented in
the farm.

			regards, tom lane

---

From: Andres Freund <[email-masked]>
Date: Tue, 19 Oct 2021 18:35:05 -0700

Hi,

> it out to my github repo together with your changes.

That turned out to be more work than I anticipated, so I pushed your changes
out separately.

There's this bit in plflex.pl that talks about adjusting yywrap() for msvc. I
didn't implement that and didn't see any compilation problems. Looks like that
originally hails from 2011, in 08a0c2dabc3b9d59d72d7a79ed867b8e37d275a7

Hm. Seems not worth carrying forward unless it actually causes trouble?

Greetings,

Andres Freund

---

From: Andres Freund <[email-masked]>
Date: Tue, 19 Oct 2021 18:49:43 -0700

Hi,

> I think that path is a nonstarter.

That does indeed sound like a nonstarter.

> a buildfarm animal on it.

Understandable.

> software platform, but I'd like to keep 32-bit PPC represented in

I assume the reason 32-bit PPC is interesting is that it's commonly run big
endian?

I wonder when it'll be faster to run 32bit ppc via qemu than natively :)

Greetings,

Andres Freund

---

From: Tom Lane <[email-masked]>
Date: Tue, 19 Oct 2021 22:04:51 -0400

> I assume the reason 32-bit PPC is interesting is that it's commonly run big

Aside from bit width and endianness, I believe it's a somewhat smaller
instruction set than the newer CPUs.

> I wonder when it'll be faster to run 32bit ppc via qemu than natively :)

I think qemu would have a ways to go for that.  More to the point,
I've found that its emulation is not as precise as one might wish...

			regards, tom lane

---

From: Andres Freund <[email-masked]>
Date: Tue, 19 Oct 2021 19:41:56 -0700

Hi,

> I wonder when it'll be faster to run 32bit ppc via qemu than natively :)

Freebsd didn't seem to want to boot, but surprisingly a debian buster image
started at least the installer without problems... Will probably take a while
to see if it actually works.

I assume to make it acceptable from a build-speed perspective one would have
to use distcc with the compiler running outside.

Greetings,

Andres Freund

---

From: Andres Freund <[email-masked]>
Date: Wed, 20 Oct 2021 09:01:56 -0700

Hi,

> to see if it actually works.

The build was quite slow (cold ccache cache, only 1 cpu):
real	106m33.418s
user	86m36.363s
sys	17m33.830s

But the actual test time wasn't *too* bad, compared to the 32bit ppc animals

real	12m14.944s
user	0m51.622s
sys	0m44.743s

Greetings,

Andres Freund

---

From: Andres Freund <[email-masked]>
Date: Thu, 21 Oct 2021 14:48:02 -0700

Hi,

> particular of update-unicode and reformat-dat-files, for example.

Implementing reformat-dat-files was trivial:
https://github.com/anarazel/postgres/commit/29c1ce1ad4731290714978da5ce81e99ef051bec

However, update-unicode is a bit harder.  Partially not directly because of
meson, but because update-unicode as-is afaict doesn't support VPATH builds,
and meson enforces those.

make update-unicode
...
make -C src/common/unicode update-unicode
'/usr/bin/perl' generate-unicode_norm_table.pl
Can't open perl script "generate-unicode_norm_table.pl": No such file or directory

It's not too hard to fix. See attached for the minimal stuff that I
immediately found to be needed. There's likely more,
e.g. src/backend/utils/mb/Unicode - but I didn't immediately see where that's
invoked from.

The slightly bigger issue making update-unicode work with meson is that meson
doesn't provide support for invoking build targets in specific directories
(because it doesn't map nicely to e.g. msbuild). But scripts like
src/common/unicode/generate-unicode_norm_table.pl rely on CWD. It's not hard
to work around that, but IMO it's better for such scripts to not rely on CWD.

Greetings,

Andres Freund
diff --git i/src/common/unicode/Makefile w/src/common/unicode/Makefile
index a3683dd86b9..e69054d4671 100644
--- i/src/common/unicode/Makefile
+++ w/src/common/unicode/Makefile
@@ -12,14 +12,14 @@ subdir = src/common/unicode
 top_builddir = ../../..
 include $(top_builddir)/src/Makefile.global
 
-override CPPFLAGS := -DFRONTEND $(CPPFLAGS)
+override CPPFLAGS := -DFRONTEND -I$(abs_top_builddir)/src/common/unicode $(CPPFLAGS)
 LIBS += $(PTHREAD_LIBS)
 
 # By default, do nothing.
 all:
 
 update-unicode: unicode_norm_table.h unicode_combining_table.h unicode_east_asian_fw_table.h unicode_normprops_table.h unicode_norm_hashfunc.h
-	mv $^ ../../../src/include/common/
+	mv $^ $(top_srcdir)/src/include/common/
 	$(MAKE) normalization-check
 
 # These files are part of the Unicode Character Database. Download
@@ -33,7 +33,7 @@ UnicodeData.txt EastAsianWidth.txt DerivedNormalizationProps.txt CompositionExcl
 unicode_norm_hashfunc.h: unicode_norm_table.h
 
 unicode_norm_table.h: generate-unicode_norm_table.pl UnicodeData.txt CompositionExclusions.txt
-	$(PERL) generate-unicode_norm_table.pl
+	$(PERL) $^
 
 unicode_combining_table.h: generate-unicode_combining_table.pl UnicodeData.txt
 	$(PERL) $^ >$@
@@ -58,7 +58,7 @@ submake-common:
 	$(MAKE) -C .. all
 
 norm_test_table.h: generate-norm_test_table.pl NormalizationTest.txt
-	perl generate-norm_test_table.pl NormalizationTest.txt $@
+	perl $^ $@
 
 .PHONY: normalization-check
 
diff --git i/contrib/unaccent/Makefile w/contrib/unaccent/Makefile
index b8307d1601e..d6c466e07ad 100644
--- i/contrib/unaccent/Makefile
+++ w/contrib/unaccent/Makefile
@@ -27,12 +27,12 @@ include $(top_builddir)/src/Makefile.global
 include $(top_srcdir)/contrib/contrib-global.mk
 endif
 
-update-unicode: unaccent.rules
+update-unicode: $(srcdir)/unaccent.rules
 
 # Allow running this even without --with-python
 PYTHON ?= python
 
-unaccent.rules: generate_unaccent_rules.py ../../src/common/unicode/UnicodeData.txt Latin-ASCII.xml
+$(srcdir)/unaccent.rules: generate_unaccent_rules.py ../../src/common/unicode/UnicodeData.txt Latin-ASCII.xml
 	$(PYTHON) $< --unicode-data-file $(word 2,$^) --latin-ascii-file $(word 3,$^) >$@
 
 # Only download it once; dependencies must match src/common/unicode/

---

From: John Naylor <[email-masked]>
Date: Fri, 22 Oct 2021 11:55:05 -0400

> immediately found to be needed.

Thanks for doing that, it works well enough for demonstration. With your
patch, and using an autoconf VPATH build, the unicode tables work fine, but
it complains of a permission error in generate_unaccent_rules.py. That
seems to be because the script is invoked directly rather than as an
argument to the python interpreter.

> to work around that, but IMO it's better for such scripts to not rely on
CWD.

Yeah. I encountered a further issue: With autoconf on HEAD, with a source
tree build executed in contrib/unaccent:

$ touch generate_unaccent_rules.py
$ make update-unicode
generate_unaccent_rules.py --unicode-data-file
../../src/common/unicode/UnicodeData.txt --latin-ascii-file Latin-ASCII.xml
>unaccent.rules
/bin/sh: generate_unaccent_rules.py: command not found
make: *** [unaccent.rules] Error 127
make: *** Deleting file `unaccent.rules'

...so in this case it seems not to know to use CWD here.

Anyway, this can be put off until the very end, since it's not run often.
You've demonstrated how these targets would work, and that's good enough
for now.

---

From: Andres Freund <[email-masked]>
Date: Mon, 15 Nov 2021 10:34:15 -0800

Hi,

FWIW, I tried building postgres on a few other operating systems using
meson, after I got access to the gcc compile farm. Here's the results:

- openbsd: Compiled fine. Hit one issue running tests:

  openbsd has *completely* broken $ORIGIN support. It uses CWD as $ORIGIN
  rpaths, which obviously breaks for binaries invoked via PATH. So there
  goes the idea to only use $ORIGIN to run tests. Still seems worth to use
  on other platforms, particularly because it works with SIP on macos

  I understand not supporting $ORIGIN at all. But implementing it this way
  seems insane.

  I also ran into some problems with the semaphore limits. I had to switch to
  USE_NAMED_POSIX_SEMAPHORES to make the tests pass at all.

- netbsd: Compiled fine after some minor fix. There's a bit more to fix around
  many libraries not being in the normal library directory, but in
  /usr/pkg/lib, which is not in the library search path (i.e. we need to add
  an rpath for that in a few more places).

- AIX: Compiled and basic postgres runs fine after a few fixes (big endian
  test, converting exports.txt into the right format). Doesn't yet
  successfully run more than trivial tests, because I didn't implement the
  necessary generation of import files for postgres, but that's just a bit of
  work.

  This is hampered by the fact that the vanilla postgres crashes for me. I
  haven't quite figured out what's the problem. Might be a system issue -
  lots of other tools, e.g. perl, segfault frequently.

  One important thing to call out: Meson has support for the AIX linker, but
  *not* the xlc compiler. I.e. one has to use gcc (or clang, but I didn't
  try). I don't know if we'd require adding support for xlc to meson - xlc is
  pretty buggy and it doesn't seem particularly crucial to support such an old
  crufty compiler on a platform that's not used to a significant degree?

Greetings,

Andres

---

From: Tom Lane <[email-masked]>
Date: Mon, 15 Nov 2021 14:11:25 -0500

>   crufty compiler on a platform that's not used to a significant degree?

While I have no particular interest in AIX or xlc specifically, I do
worry about us becoming a builds-on-gcc-or-workalikes-only project.
I suppose MSVC provides a little bit of a cross-check, but I don't
really like giving up on other compilers.  Discounting gcc+clang+MSVC
leaves just a few buildfarm animals, and the xlc ones are a significant
part of that population.  (In fact, unless somebody renews fossa/husky's
icc license, the three xlc animals will be an outright majority of
them, because wrasse and anole are the only other active animals with
non-mainstream compilers.)

Having said that, I don't plan to be the one trying to get meson
to add xlc support ...

			regards, tom lane

---

From: Andres Freund <[email-masked]>
Date: Mon, 15 Nov 2021 11:23:40 -0800

Hi,

On 2021-11-15 14:11:25 -0500, Tom Lane wrote:
> part of that population.

Yea, that's a reasonable concern. I wonder if there's some non-mainstream
compiler that actually works on, um, more easily available platforms that we
could utilize.

> (In fact, unless somebody renews fossa/husky's

It should probably be doable to get somebody to run another icc animal. Icc is
supported by meson, fwiw.

> Having said that, I don't plan to be the one trying to get meson

It'd probably not be too hard. But given that it's quite hard to get access to
AIX + xlc, I'm not sure it's something I want to propose. There's no resources
to run halfway regular tests on that I found...

It's good to make sure we're not growing too reliant on some compiler(s), but
imo only really makes sense if the alternative compilers are meaningfully
available and maintained.

Greetings,

Andres Freund

---

From: Robert Haas <[email-masked]>
Date: Mon, 15 Nov 2021 14:48:48 -0500

> available and maintained.

That's a sensible position. I do worry that with this proposed move
we're going to be giving up some of the flexibility that we have right
now. I'm not sure exactly what that means in practice. But make is
just a way of running shell commands, and so you can run any shell
commands you want. The concept of some compiler not being supported
isn't really a thing that even makes sense in a world that is powered
by make. With a big enough hammer you can run any commands you like,
including any compilation commands you like. The whole thing is likely
to be a bit crufty which is a downside, and you might spend more time
fiddling with it than you really want. But nothing is really ever
blocked.

-- 
Robert Haas

---

From: Thomas Munro <[email-masked]>
Date: Tue, 16 Nov 2021 11:08:15 +1300

> to run halfway regular tests on that I found...

FWIW there's a free-as-in-beer edition of xlc for Linux (various
distros, POWER only) so you could use qemu, though of course there
will be differences WRT AIX especially around linking, and I suppose a
big part of that work would be precisely understanding stuff like
linker details.

It looks like we have two xlc 12.1 compilers in the farm, but those
compilers are EOL'd[1].  The current release is 16.1, and we have one
of those.  The interesting thing about 16.1 is that you can invoke it
as xlclang to get the new clang frontend and, I think, possibly use
more clang/gcc-ish compiler switches[2].

[1] https://www.ibm.com/support/pages/lifecycle/search?q=xl%20c%2Fc%2B%2B
[2] https://www.ibm.com/docs/en/xl-c-and-cpp-aix/16.1?topic=new-clang-based-front-end

---

From: Tom Lane <[email-masked]>
Date: Mon, 15 Nov 2021 17:34:33 -0500

Thomas Munro <[email-masked]> writes:
> ... The interesting thing about 16.1 is that you can invoke it

Ho, that's an earful ;-).  Though I wonder whether that frontend
hides the AIX-specific linking issues you mentioned.  (Also, although
I see /opt/IBM/xlc/16.1.0/ on gcc119, there's no xlclang there.
So whether we have useful access to it right now is unclear.)

This plays into something that was nagging at me while I wrote my
upthread screed about not giving up on non-gcc/clang compilers:
are those compilers outcompeting all the proprietary ones, to the
extent that the latter will be dead soon anyway?  I think Microsoft
is rich enough and stubborn enough to keep on developing MSVC no
matter what, but other compiler vendors may see the handwriting
on the wall.  Writing C compilers can't be a growth industry these
days.

			regards, tom lane

---

From: Andres Freund <[email-masked]>
Date: Mon, 15 Nov 2021 14:46:49 -0800

Hi,

> So whether we have useful access to it right now is unclear.)

It's actually available there, but in /opt/IBM/xlC/16.1.0/bin/xlclang++ (note
the upper case C).

It doesn't really hide the linking issues afaict. I think they're basically an
ABI rather than a linker invocation issue. It's not that hard to address them
though, it's basically making mkldexport.sh a tiny bit more general and
integrating it into src/backend/postgres' build.

We don't have to generate export files for shared libraries anymore though,
afaict, because there's 'expall', which suffices for our purposes. dlopen()
doesn't require an import file.

> extent that the latter will be dead soon anyway?

I think that's a pretty clear trend. The ones that aren't dying seem to be
incrementally onto more and more rebasing onto llvm tooling.

It doesn't help that most of those compilers are primarily for OSs that, uh,
aren't exactly growing. Which limits their potential usability significantly.

Greetings,

Andres Freund

---

From: Thomas Munro <[email-masked]>
Date: Tue, 16 Nov 2021 11:52:33 +1300

On Tue, Nov 16, 2021 at 11:08 AM Thomas Munro <[email-masked]> wrote:
> distros, POWER only) so you could use qemu,

(It's also known to be possible to run AIX 7.2 on qemu, but the
install media is not made available to developers for testing/CI
without a hardware serial number.  Boo.)

---

From: Thomas Munro <[email-masked]>
Date: Fri, 19 Nov 2021 09:44:37 +1300

> supported by meson, fwiw.

FWIW, in case someone is interested in bringing ICC back to the farm,
some light googling tells me that newer editions of "classic" ICC (as
opposed to "data parallel" ICC, parts of some kind of rebrand) no
longer require regular licence bureaucracy, and can be installed in
modern easier to maintain ways.  For example, I see that some people
add Intel's APT repository and apt-get install the compiler inside CI
jobs, on Ubuntu.

---

From: Andres Freund <[email-masked]>
Date: Wed, 25 May 2022 17:47:31 -0700

Hi Tom,

in the meson unconference session you'd spotted flex flags for psqlscanslash.l
(I think) being "hardcoded". As far as I can tell that's largely just copied
from the Makefile):

src/backend/parser/Makefile:scan.c: FLEXFLAGS = -CF -p -p
src/backend/utils/adt/Makefile:jsonpath_scan.c: FLEXFLAGS = -CF -p -p
src/bin/psql/Makefile:psqlscanslash.c: FLEXFLAGS = -Cfe -p -p
src/fe_utils/Makefile:psqlscan.c: FLEXFLAGS = -Cfe -p -p

note that it's not even FLEXFLAGS += or such.

I honestly don't know enough about the various flex flags to judge what a
better approach would be? Looks like these flags are case specific? Perhaps we
could group them, i.e. have centrally defined "do compress" "don't compress"
flex flags?

Greetings,

Andres Freund

---

From: Tom Lane <[email-masked]>
Date: Wed, 25 May 2022 21:38:33 -0400

> src/fe_utils/Makefile:psqlscan.c: FLEXFLAGS = -Cfe -p -p

Hmm, OK.  There *is* a FLEXFLAGS definition supplied by configure, and
I believe many of our scanners do use it, but evidently we're just
overriding it for the ones where we really care about using specific
flags.  It also looks like the configure-supplied version is usually
empty, so the fact that this variable exists may be mostly a holdover
from Autoconf practice rather than something we ever cared about.

I think the main thing I didn't like about the way you have it in the
meson file is the loss of greppability.  I could investigate this
question in a few seconds just now, but if we drop the use of
FLEXFLAGS as a macro it'll become much harder to figure out which
places use what.

			regards, tom lane

---

From: Andres Freund <[email-masked]>
Date: Wed, 25 May 2022 19:41:59 -0700

Hi,

> from Autoconf practice rather than something we ever cared about.

Yea, it looks like that.

ISTM that it'd still be good to have something like FLEXFLAGS. But it doesn't
look great, nor really intentional, that FLEXFLAGS is overwritten rather than
appended?

> FLEXFLAGS as a macro it'll become much harder to figure out which

I disliked a bunch of repetitiveness as I had it, so I'm polishing that part
just now.

What would you want to grep for? Places that specify additional flags? Or just
places using flex?

Greetings,

Andres Freund

---

From: Tom Lane <[email-masked]>
Date: Wed, 25 May 2022 22:58:39 -0400

Andres Freund <[email-masked]> writes:
>> meson file is the loss of greppability.

> What would you want to grep for? Places that specify additional flags? Or just

Well, the consistency of having a single name for "flags given to
flex" seems to me to be worth something.

			regards, tom lane

---

From: Aleksander Alekseev <[email-masked]>
Date: Thu, 26 May 2022 11:47:13 +0300

Hi Andres,

Thanks for working on this! I'm very enthusiastic about this effort and I was
glad to see on PGCon Unconference that the majority of the community seems
to be as well.

> The halfway decent list includes, I think:
> 1) cmake
> 2) bazel
> 3) meson

Was SCons considered as an option? It is widely adopted and written in Python
as well. Personally, I like the fact that with SCons you write config files
in pure Python, not some dialect you have to learn additionally. There is
a free e-book available [1].

What pros and cons do you see that make Meson a better choice?

[1]: https://scons.org/doc/production/PDF/scons-user.pdf

-- 
Best regards,
Aleksander Alekseev

---

From: Andres Freund <[email-masked]>
Date: Sat, 28 May 2022 12:14:42 -0700

Hi,

On 2022-05-26 11:47:13 +0300, Aleksander Alekseev wrote:
> Was SCons considered as an option?

I looked at it and quickly discarded it. From what I could see there's not
been meaningful moves to it in the last couple years, if anything adoption has
been dropping. And I don't think we want to end up relying on yet another half
maintained tool.

Not having a ninja backend etc didn't strike me as great either - the builds
with scons I've done weren't fast at all.

Greetings,

Andres Freund

---

From: Aleksander Alekseev <[email-masked]>
Date: Tue, 31 May 2022 16:49:17 +0300

Hi Andres,

> with scons I've done weren't fast at all.

I must admit, personally I never used Scons, I just know that it was considered
(an / the only?) alternative to CMake for many years. The Scons 4.3.0 release
notes say that Ninja is supported [1], but according to the user guide [2]
Ninja support is considered experimental.

Don't get me wrong, I don't insist on using Scons. I was just curious if it was
considered. Actually, a friend of mine pointed out that the fact that Scons
build files are literally a Python code could be a disadvantage. There is less
control of this code, basically it can do anything. It could complicate the
diagnosis of certain issues, etc.

Since you invested so much effort into Meson already let's just focus on it.

I tried the branch on GitHub on MacOS Monterey 12.3.1 and Ubuntu 20.04 LTS.
I was going to test it against several third party extensions, but it looks like
it is a bit early for this. On Ubuntu I got the following error:

```
../src/include/parser/kwlist.h:332:25: error: ‘PARAMETER’ undeclared here (not
in a function)
332 | PG_KEYWORD("parameter", PARAMETER, UNRESERVED_KEYWORD, BARE_LABEL)

../src/interfaces/ecpg/preproc/keywords.c:32:55: note: in definition of macro
‘PG_KEYWORD’
32 | #define PG_KEYWORD(kwname, value, category, collabel) value,
```

On MacOS I got multiple errors regarding LDAP:

```
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/
LDAP.framework/Headers/ldap.h:1:10: error: #include nested too deeply
#include <ldap.h>

../src/interfaces/libpq/fe-connect.c:4816:2: error: use of undeclared
identifier 'LDAP'
    LDAP      *ld = NULL;
    ^

../src/interfaces/libpq/fe-connect.c:4817:2: error: use of undeclared
identifier 'LDAPMessage'
    LDAPMessage *res,
    ^
... etc...
```

I didn't invest much time into investigating these issues. For now I just
wanted to report them. Please let me know if you need any help with these
and/or additional information.

[1]: https://scons.org/scons-430-is-available.html
[2]: https://scons.org/doc/production/PDF/scons-user.pdf

-- 
Best regards,
Aleksander Alekseev

---

From: Andres Freund <[email-masked]>
Date: Tue, 31 May 2022 12:25:40 -0700

Hi,

> it is a bit early for this. On Ubuntu I got the following error:

What do those extensions use to build? Since the unconference I added some
rudimentary PGXS compatibility, but it's definitely not complete yet.

> 32 | #define PG_KEYWORD(kwname, value, category, collabel) value,

Huh. I've not seen this before - could you provide a bit more detail about
what you did? CI isn't testing ubuntu, but it is testing Debian, so I'd expect
this to work.

> On MacOS I got multiple errors regarding LDAP:

Ah, yes. Sorry, that's an open issue that I need to fix. -Dldap=disabled for
the rescue.  There's some crazy ordering dependency in macos framework
headers. The ldap framework contains an "ldap.h" header that includes
"ldap.h". So if you end up with the framework on the include path, you get
endless recursion.

Greetings,

Andres Freund

---

From: Aleksander Alekseev <[email-masked]>
Date: Wed, 1 Jun 2022 12:39:50 +0300

Hi Andres,

> rudimentary PGXS compatibility, but it's definitely not complete yet.

We mostly use CMake and Cargo, the Rust package manager. So I don't
anticipate many problems here, just want to make sure it's going to
work as expected.

> what you did? CI isn't testing ubuntu, but it is testing Debian, so I'd expect

I used PIP to install Meson, since the default APT package is too old, v0.53:

$ pip3 install --user meson
$ meson --version
0.62.1
$ ninja --version
1.10.0

The branch was checked out as it was described in the first email.
Then to reproduce the issue:

$ git status
On branch meson
Your branch is up to date with 'andres/meson'.
$ git fetch andres
$ git rebase -i andres/meson
$ meson setup build --buildtype debug
$ cd build
$ ninja

This is pretty much the default Ubuntu 20.04.4 LTS system with all the
recent updates installed, so it shouldn't be a problem to reproduce
the issue with a VM.

> "ldap.h". So if you end up with the framework on the include path, you get

Thanks, this helped. I did the following:

$ meson configure -Dldap=disabled
$ meson configure -Dssl=openssl
$ meson configure -Dprefix=/Users/eax/pginstall
$ ninja
$ meson test
$ meson install

... and it terminated successfully. I was also able to configure and
run Postgres instance using my regular scripts, with some
modifications [1]

Then I decided to compile TimescaleDB against the newly installed
Postgres. Turns out there is a slight problem.

The extension uses CMake and also requires PostgreSQL to be compiled
with OpenSSL support. CMakeLists.txt looks for a
"--with-(ssl=)?openssl" regular expression in the "pg_config
--configure" output. The output is empty although Postgres was
compiled with OpenSSL support. The full output of pg_config looks like
this:

```
CONFIGURE =
CC = not recorded
CPPFLAGS = not recorded
CFLAGS = not recorded
CFLAGS_SL = not recorded
... etc ...
```

I get a bunch of errors from the compiler if I remove this particular
check from CMakeLists, but I have to investigate these a bit more
since the branch is based on PG15 and we don't officially support PG15
yet. It worked last time we checked a month or so ago, but the
situation may have changed.

[1]: https://github.com/afiskon/pgscripts/blob/master/single-install.sh

-- 
Best regards,
Aleksander Alekseev

---

From: Andres Freund <[email-masked]>
Date: Wed, 1 Jun 2022 14:05:28 -0700

Hi,

> recent updates installed, so it shouldn't be a problem to reproduce

Will test.

> Thanks, this helped.

Cool. I think I pushed a fix/workaround for the issue now. Still can't decide
whether it's apple's or meson's fault.

> I did the following:

FYI, you can set multiple options in one go ;)

Cool.

> compiled with OpenSSL support.

Makes sense. Currently we don't fill the --configure thing, because there
configure wasn't used. We could try to generate something compatible from
meson options, but I'm not sure that's a good plan.

> The full output of pg_config looks like
> this:
> 
> ```
> CONFIGURE =
> CC = not recorded
> CPPFLAGS = not recorded
> CFLAGS = not recorded
> CFLAGS_SL = not recorded
> ... etc ...
> ```
> 

I suspect the errors might be due to CC/CPPFLAGS/... not being defined. I can
try to make it output something roughly compatible with the old output, for
most of those I already started to compute values for the PGXS compat stuff I
was hacking on recently.

Greetings,

Andres Freund

---

From: Aleksander Alekseev <[email-masked]>
Date: Thu, 2 Jun 2022 15:34:23 +0300

Hi Andres,

> whether it's apple's or meson's fault.

Many thanks! The fix solved the problem, I can compile with -Dldap=enabled now.
The code passes the tests too.

> FYI, you can set multiple options in one go ;)

Thanks! ;)

> meson options, but I'm not sure that's a good plan.

If pg_config output was 100% backward compatible with Autotools one, that would
simplify the lives of the extension developers for sure. However, considering
that at PGCon we agreed that both Autotools and Meson will be maintained for
several releases, personally I wouldn't say that this compatibility is
necessary, nor is it realistically deliverable. Nevertheless, IMO there should
be a stable and documented way to determine the PostgreSQL version (this can be
done with `pg_config --version` for both Autotools and Meson), the build tool
used (no way to determine) and the build options (no way to determine
for Meson).

> most of those I already started to compute values for the PGXS compat stuff I

Yes, that could explain the problem. Just for the record, I get several errors
regarding src/export.h in TimescaleDB code [1]:

```
/Users/eax/projects/c/timescaledb/src/export.h:26:5: error: pasting formed
  ')87628', an invalid preprocessing token [clang-diagnostic-error]
#if TS_EMPTY(PGDLLEXPORT)
             ^
/Users/eax/projects/c/timescaledb/src/export.h:17:22: note: expanded from
  macro 'TS_EMPTY'
#define TS_EMPTY(x) (TS_CAT(x, 87628) == 87628)
                     ^
/Users/eax/projects/c/timescaledb/src/export.h:15:23: note: expanded from
  macro 'TS_CAT'
#define TS_CAT(x, y) x##y
                      ^
/Users/eax/projects/c/timescaledb/src/export.h:26:14: error: function-like
  macro '__attribute__' is not defined [clang-diagnostic-error]
#if TS_EMPTY(PGDLLEXPORT)
             ^
/Users/eax/pginstall/include/postgresql/server/c.h:1339:21: note: expanded
  from macro 'PGDLLEXPORT'
#define PGDLLEXPORT __attribute__((visibility("default")))
                    ^
/Users/eax/projects/c/timescaledb/src/export.h:30:2: error: "PGDLLEXPORT is
  already defined" [clang-diagnostic-error]
#error "PGDLLEXPORT is already defined"
 ^
1 warning and 3 errors generated.
Error while processing /Users/eax/projects/c/timescaledb/src/extension.c
```

[1]: https://github.com/timescale/timescaledb/blob/main/src/export.h

-- 
Best regards,
Aleksander Alekseev

---

From: Andres Freund <[email-masked]>
Date: Thu, 2 Jun 2022 08:22:57 -0700

Hi,

Cool.

> regarding src/export.h in TimescaleDB code [1]:

I think this is timescale's issue. Why are you defining / undefining
PGDLLEXPORT?

Part of the patch series is to use visibility attributes, and your #if
TS_EMPTY(PGDLLEXPORT) thing can't handle that.

Greetings,

Andres Freund

---

From: Andres Freund <[email-masked]>
Date: Thu, 2 Jun 2022 09:39:30 -0700

Hi,

> recent updates installed, so it shouldn't be a problem to reproduce

Chatting with a colleague (who unbeknownst to me hit something similar in the
past) I think we figured it out. It's not due to Ubuntu 20.04 or such. It's
likely due to previously having an in-tree build with autoconf, doing make
clean, doing a git pull, then building with meson. The meson build doesn't yet
handle pre-existing flex / bison output.

I had tried to defend against conflicts with in-tree builds by detecting an
in-tree pg_config.h, but that doesn't help with files that aren't removed by
make clean. Like bison / flex output.

And I didn't notice this problem because it doesn't cause visible issues until
the lexer / grammar changes...

I'm not quite sure what the proper behaviour is when doing an out-of-tree
build with meson (all builds are out-of-tree), with a pre-existing flex /
bison output in the source tree that is out of date.

Greetings,

Andres Freund

---

From: Tom Lane <[email-masked]>
Date: Thu, 02 Jun 2022 13:08:49 -0400

> bison output in the source tree that is out of date.

Definitely sounds like a gotcha.

On the one hand, there's been some discussion already of removing all
derived files from tarballs and just insisting that users provide all
needed tools when building from source.  If we did that, it could be
sufficient for the meson build to check that no such files are present
in the source tree.  (Checking a couple of them would be enough, likely.)

On the other hand, I'm not sure that I want such a change to be forced
by a toolchain change.  It definitely seems a bit contrary to the plan
we'd formed of allowing meson and make-based builds to coexist for
a few years, because we'd be breaking at least some make-based build
processes.

Could we have the meson build check that, say, if gram.c exists it
is newer than gram.y?  Or get it to ignore an in-tree gram.c?

			regards, tom lane

---

From: Andres Freund <[email-masked]>
Date: Thu, 2 Jun 2022 10:26:09 -0700

Hi,

> in the source tree.  (Checking a couple of them would be enough, likely.)

There already is a check for pg_config.h, so the most obvious source of this
is addressed. Just didn't think about the files that make clean doesn't remove
:/.

> a few years, because we'd be breaking at least some make-based build

Agreed. I think it'd be pretty reasonable to not include flex / bison
output. They're not hard to acquire. The docs are perhaps another story.

I think it might be fine to say that make reallyclean (*) is required if
there's some conflicting in-source tree file?

> is newer than gram.y?  Or get it to ignore an in-tree gram.c?

I suspect the problem with ignoring is gram.h, that's probably a bit harder to
ignore. Right now I'm leaning towards either always erroring out if there's
bison/flex output in the source tree (with a hint towards make
reallyclean(*)), or erroring out if they're out of date (again with a hint
towards reallyclean)?

Alternatively we could just remove the generated .c/h files from the source
dir, as a part of regenerating them in the build dir? But I like the idea of
the source dir being readonly outside of explicit targets modifying sources
(e.g. update-unicode or such).

Greetings,

Andres Freund

(*) do we really not have a target that removes bison / flex output?

---

From: Tom Lane <[email-masked]>
Date: Thu, 02 Jun 2022 13:33:51 -0400

> (*) do we really not have a target that removes bison / flex output?

maintainer-clean

			regards, tom lane

---

From: Andres Freund <[email-masked]>
Date: Thu, 2 Jun 2022 10:48:45 -0700

Hi,

> > (*) do we really not have a target that removes bison / flex output?

Don't think so:

# gram.c, gram.h, and scan.c are in the distribution tarball, so they
# are not cleaned here.
clean distclean maintainer-clean:
	rm -f lex.backup

Greetings,

Andres Freund

---

From: Tom Lane <[email-masked]>
Date: Thu, 02 Jun 2022 15:05:10 -0400

> Don't think so:

See about line 300 in src/backend/Makefile.  In any case, it's
easy to show by experiment that it does.

$ make maintainer-clean
...
$ git status --ignored
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean

			regards, tom lane

---

From: Andres Freund <[email-masked]>
Date: Thu, 2 Jun 2022 12:17:55 -0700

Hi,

> nothing to commit, working tree clean

Oh. I executed maintainer-clean inside src/backend/parser/, and thus didn't
see it getting cleaned up.

It seems pretty darn grotty that src/backend/parser/Makefile explicitly states
that gram.c ... aren't cleaned "here", but then src/backend/Makefile does
clean them up.

Greetings,

Andres Freund

---

From: Tom Lane <[email-masked]>
Date: Thu, 02 Jun 2022 15:53:50 -0400

> that gram.c ... aren't cleaned "here", but then src/backend/Makefile does

I agree the factorization of this ain't great.  I'd think about improving
it, were it not that we're trying to get rid of it.

(But with meson, the whole idea of building or cleaning just part of the
tree is out the window anyway, no?)

			regards, tom lane

---

From: Andres Freund <[email-masked]>
Date: Thu, 2 Jun 2022 14:13:33 -0700

Hi,

> it, were it not that we're trying to get rid of it.

+1. I think I just wanted to excuse my confusion...

> tree is out the window anyway, no?)

Cleaning parts of the tree isn't supported as far as I know (not that I've
needed it). You can build parts of the tree by specifying the target
(e.g. ninja src/backend/postgres) or by specifying meta-targets (e.g. ninja
contrib backend). I've thought about contributing a patch to meson to
automatically generate targets for each directory that has sub-targets - it's
just a few lines.

Greetings,

Andres Freund

---

From: Aleksander Alekseev <[email-masked]>
Date: Fri, 3 Jun 2022 12:35:45 +0300

Hi hackers,

> easy to show by experiment that it does.

`make maintainer-clean` did the trick, thanks. I suggest modifying meson.build
accordingly:

-run make distclean in the source tree.
+run `make maintainer-clean` in the source tree.

> I think this is timescale's issue. Why are you defining / undefining

That's a great question.

As I understand some time ago the developers had a problem with a collision of
exported symbols on *nix platforms [1] and chose to solve it by re-defining
PGDLLEXPORT to __attribute__((visibility ("default"))) for GCC and CLang.
I agree that this is a questionable approach. Redefining a macro provided
by Postgres doesn't strike me as a good idea. I tried to remove this
re-definition, but it didn't go well [2]. So apparently it should be addressed
somehow differently.

> TS_EMPTY(PGDLLEXPORT) thing can't handle that.

Out of curiosity, how come a patchset that adds an alternative build system
changes the visibility attributes? I would guess they should be the same
for both Autotools and Meson. Is it necessary in order to make Meson work?
If not, maybe it should be a separate patch.

[1]: https://github.com/timescale/timescaledb/commit/027b7b29420a742d7615c70d9f19b2b99c488c2c
[2]: https://github.com/timescale/timescaledb/pull/4413

-- 
Best regards,
Aleksander Alekseev

---

From: Andres Freund <[email-masked]>
Date: Fri, 3 Jun 2022 09:23:21 -0700

Hi,

On 2022-06-03 12:35:45 +0300, Aleksander Alekseev wrote:
> changes the visibility attributes?

It was the simplest path - on windows (and AIx) extension symbols need to be
explicitly exported. We did that by building the objects constituting
extension libraries, collecting the symbols, generating an export file with
all the symbols, which then is passed to the linker.  It was a lot less work
to just add the necessary PGDLLEXPORT annotations than make that export file
generation work for extensions.

> I would guess they should be the same for both Autotools and Meson.

It is, the patch adds it to both.

> Is it necessary in order to make Meson work?

Yes, or at least the simplest path.

> If not, maybe it should be a separate patch.

It is.

https://commitfest.postgresql.org/38/3396/

Greetings,

Andres Freund

---

From: Andres Freund <[email-masked]>
Date: Tue, 9 Aug 2022 00:10:55 -0700

Hi,

I was looking at re-unifying gendef2.pl that the meson patchset had introduced
for temporary ease during hacking with gendef.pl. Testing that I noticed that
either I and my machine is very confused, or gendef.pl's check whether it can
skip work is bogus.

I noticed that, despite having code to avoid rerunning when the input files
are older than the .def file, it always runs.

# if the def file exists and is newer than all input object files, skip
# its creation
if (-f $deffile
    && (-M $deffile > max(map { -M } <$ARGV[0]/*.obj>)))
{
    print "Not re-generating $defname.DEF, file already exists.\n";
    exit(0);
}

My understanding of -M is that it returns the time delta between the file
modification and the start of the script. Which makes the use of max() bogus,
since it'll return the oldest time any input has been modified, not the
newest. And the condition needs to be inverted, because we want to skip the
work if $deffile is *newer*, right?

Am I missing something here?

I'm tempted to just remove the not-regenerating logic - gendef.pl shouldn't
run if there's nothing to do, and it'll e.g. not notice if there's an
additional input that wasn't there during the last invocation of gendef.pl.

Greetings,

Andres Freund

---

From: Andrew Dunstan <[email-masked]>
Date: Tue, 9 Aug 2022 08:37:16 -0400

> Am I missing something here?

No, you're right, this is bogus. Reversing the test and using min
instead of max is the obvious fix.

> additional input that wasn't there during the last invocation of gendef.pl.

Maybe, need to think about that more.

cheers

andrew

---

From: Andres Freund <[email-masked]>
Date: Wed, 10 Aug 2022 10:19:35 -0700

Hi,

On 2022-06-02 10:26:09 -0700, Andres Freund wrote:
> ignore.

I tried to ignore various generated files in the source tree, but I don't
think it's doable for all of them. Consider
e.g. src/backend/utils/misc/guc-file.c which is gets built via #include
"guc-file.c" from gram.c

Because it's a "" include, the search path starts in the current directory and
only then -I is searched. To my knowledge there's no way of changing
that. Quoting the gcc manpage:

       -I dir
       -iquote dir
       -isystem dir
       -idirafter dir
           Add the directory dir to the list of directories to be searched for header files during preprocessing.  If dir begins with = or $SYSROOT, then
           the = or $SYSROOT is replaced by the sysroot prefix; see --sysroot and -isysroot.

           Directories specified with -iquote apply only to the quote form of the directive, "#include "file"".  Directories specified with -I, -isystem, or
           -idirafter apply to lookup for both the "#include "file"" and "#include <file>" directives.

           You can specify any number or combination of these options on the command line to search for header files in several directories.  The lookup
           order is as follows:

           1.  For the quote form of the include directive, the directory of the current file is searched first.

           2.  For the quote form of the include directive, the directories specified by -iquote options are searched in left-to-right order, as they appear
               on the command line.

           3.  Directories specified with -I options are scanned in left-to-right order.
           [...]

Except for copying guc.c from source to build tree before building, I don't
see a way of ignoring the in-build-tree guc-file.c.

Not sure what a good way of dealing with this is. For now I'll make it just
error out if there's any known such file in the source tree, but that's not a
good solution forever.  If it were just "normal" build leftovers I'd propose
to (optionally) just remove them, but that's not good for tarballs.

Greetings,

Andres Freund

---

From: John Naylor <[email-masked]>
Date: Thu, 11 Aug 2022 10:33:39 +0700

> e.g. src/backend/utils/misc/guc-file.c which is gets built via #include

With a bit of work, we could probably get rid of those includes. See
27199058d98ef7f for one example.

---

From: Tom Lane <[email-masked]>
Date: Wed, 10 Aug 2022 23:37:53 -0400

> 27199058d98ef7f for one example.

Yeah --- it would mean creating gram.h files for all the bison grammars
not just a few of them, but it's certainly do-able if there's motivation
to make the changes.  Most of the files that are done that way date
from before we knew about flex's %top.

			regards, tom lane

---

From: Tom Lane <[email-masked]>
Date: Wed, 10 Aug 2022 23:45:26 -0400

> from before we knew about flex's %top.

BTW, 72b1e3a21 is another useful precedent in this area.

			regards, tom lane

---

From: John Naylor <[email-masked]>
Date: Thu, 11 Aug 2022 10:57:33 +0700

> from before we knew about flex's %top.

I'll volunteer to work on this unless an easier solution happens to
come along in the next couple days. (aside: guc-file.l doesn't have a
grammar, so not yet sure if that makes the issue easier or harder...)

---

From: Andres Freund <[email-masked]>
Date: Wed, 10 Aug 2022 21:07:48 -0700

Hi,

On 2022-08-11 10:57:33 +0700, John Naylor wrote:
> come along in the next couple days.

Cool!

> (aside: guc-file.l doesn't have a grammar, so not yet sure if that makes the
> issue easier or harder...)

I think we should consider compiling it separately from guc.c. guc.c already
compiles quite slowly (iirc beat only by ecpg and main grammar), and it's a
relatively commonly changed source file.

It might even be a good idea to split guc.c so it only contains the settings
arrays + direct dependencies...

Greetings,

Andres Freund

---

From: Tom Lane <[email-masked]>
Date: Thu, 11 Aug 2022 00:32:36 -0400

> grammar, so not yet sure if that makes the issue easier or harder...)

That one's probably mostly about the issue mentioned in the other
commit you identified.  Without %top, it's impossible to make a
standalone flex module honor the rule about thou-shalt-have-no-
other-includes-before-postgres.h.  So embedding it in some other
file was originally a necessity for that.  Now that we know how
to fix that, it's just a matter of making sure that any other stuff
the scanner needs is available from a .h file.

			regards, tom lane

---

From: Andres Freund <[email-masked]>
Date: Sat, 20 Aug 2022 17:42:34 -0700

Hi,

> Maybe, need to think about that more.

Any thoughts?

I'd like to commit 0003 in
https://postgr.es/m/20220811002012.ju3rrz47i2e5tdha%40awork3.anarazel.de
fairly soon.

I did fix the bogus "die" message I added during some debugging since posting
that...

Greetings,

Andres Freund