Subject: meson oddities

From: Andrew Dunstan <[email-masked]>
Date: Mon, 14 Nov 2022 17:41:54 -0500

Here's a couple of things I've noticed.

andrew@ub22:HEAD $ inst.meson/bin/pg_config --libdir --ldflags
/home/andrew/pgl/pg_head/root/HEAD/inst.meson/lib/x86_64-linux-gnu
-fuse-ld=lld -DCOPY_PARSE_PLAN_TREES -DRAW_EXPRESSION_COVERAGE_TEST
-DWRITE_READ_PARSE_PLAN_TREES

Are we really intending to add a new subdirectory to the default layout?
Why is that x84_64-linux-gnu there?

Also, why have the CPPFLAGS made their way into the LDFLAGS? That seems
wrong.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

---

From: Michael Paquier <[email-masked]>
Date: Tue, 15 Nov 2022 08:22:59 +0900

> Also, why have the CPPFLAGS made their way into the LDFLAGS? That seems

Not only CPPFLAGS.  I pass down some custom CFLAGS to the meson
command as well, and these find their way to LDFLAGS on top of
CFLAGS for the user-defined entries.  I would not have expected that,
either.
--
Michael

---

From: Andres Freund <[email-masked]>
Date: Mon, 14 Nov 2022 15:24:07 -0800

Hi,

> Why is that x84_64-linux-gnu there?

It's the platform default on, at least, debian derived distros - that's how
you can install 32bit/64bit libraries and libraries with different ABIs
(e.g. linking against glibc vs linking with musl) in parallel.

We could override meson inferring that from the system if we want to, but it
doesn't seem like a good idea?

> Also, why have the CPPFLAGS made their way into the LDFLAGS? That seems

Because these days meson treats CPPFLAGS as part of CFLAGS as it apparently
repeatedly confused build system writers and users when e.g. header-presence
checks would only use CPPFLAGS. Some compiler options aren't entirely clearly
delineated, consider e.g. -isystem (influencing warning behaviour as well as
preprocessor paths).  Not sure if that's the best choice, but it's imo
defensible.

Greetings,

Andres Freund

---

From: Andres Freund <[email-masked]>
Date: Mon, 14 Nov 2022 15:48:12 -0800

Hi,

On 2022-11-15 08:22:59 +0900, Michael Paquier wrote:
> I pass down some custom CFLAGS to the meson command as well, and these find
> their way to LDFLAGS on top of CFLAGS for the user-defined entries.  I would
> not have expected that, either.

We effectively do that with autoconf as well, except that we don't mention
that in pg_config --ldflags. Our linking rules include CFLAGS, see e.g.:

%: %.o
	$(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)

postgres: $(OBJS)
	$(CC) $(CFLAGS) $(call expand_subsys,$^) $(LDFLAGS) $(LDFLAGS_EX) $(export_dynamic) $(LIBS) -o $@

ifdef PROGRAM
$(PROGRAM): $(OBJS)
	$(CC) $(CFLAGS) $(OBJS) $(PG_LIBS_INTERNAL) $(LDFLAGS) $(LDFLAGS_EX) $(PG_LIBS) $(LIBS) -o $@$(X)
endif

# Rule for building a shared library from a single .o file
%.so: %.o
	$(CC) $(CFLAGS) $< $(LDFLAGS) $(LDFLAGS_SL) -shared -o $@

Should we try that fact in pg_configin the meson build as well?

Meson automatically includes compiler flags during linking because a)
apparently many dependencies (.pc files etc) specify linker flags in CFLAGS b)
at least some kinds of LTO requires compiler flags being present during
"linking".

Greetings,

Andres Freund

---

From: Andrew Dunstan <[email-masked]>
Date: Tue, 15 Nov 2022 08:04:29 -0500

> doesn't seem like a good idea?

That's a decision that packagers make. e.g. on my Ubuntu system
configure has been run with:

--libdir=${prefix}/lib/x86_64-linux-gnu

Incidentally, Redhat flavored systems don't use this layout. they have
/lib and /lib64, so it's far from universal.

But ISTM we shouldn't be presuming what packagers will do, and that
there is some virtue in having a default layout under ${prefix} that is
consistent across platforms, as is now the case with autoconf/configure.

> preprocessor paths).  Not sure if that's the best choice, but it's imo

Yes, I get that there is confusion around CPPFLAGS. One of my otherwise
extremely knowledgeable colleagues told me a year or two back that he
had thought the CPP in CPPFLAGS referred to C++ rather that C
preprocessor. And the authors of meson seem to have labored under a
similar misapprehension, so they use 'cpp' instead of 'cxx' like just
about everyone else.

But it's less clear to me that a bunch of defines belong in LDFLAGS.
Shouldn't that be only things that ld itself will recognize?

cheers

andrew

---

From: Peter Eisentraut <[email-masked]>
Date: Tue, 15 Nov 2022 17:07:05 +0100

> Should we try that fact in pg_configin the meson build as well?

It's up to the consumer of pg_config to apply CFLAGS and LDFLAGS as they 
need.  But pg_config and pkg-config etc. should report them separately.

---

From: Andres Freund <[email-masked]>
Date: Tue, 15 Nov 2022 11:04:25 -0800

Hi,

> --libdir=${prefix}/lib/x86_64-linux-gnu

Sure - but that doesn't mean that it's a good idea to break the distribution's
layout when you install from source.

> /lib and /lib64, so it's far from universal.

Meson infers that and uses lib64 as the default libdir.

> consistent across platforms, as is now the case with autoconf/configure.

I don't think it's a virtue to break the layout of the platform by
e.g. installing 64bit libs into the directory containing 32bit libs.

> And the authors of meson seem to have labored under a similar
> misapprehension, so they use 'cpp' instead of 'cxx' like just about everyone
> else.

Yea, not a fan of that either. I don't think it was a misapprehension, but a
decision I disagree with...

> Shouldn't that be only things that ld itself will recognize?

I don't think there's a clear cut line what is for ld and what
isn't. Including stuff that influences both preprocessor and
linker. -ffreestanding will e.g. change preprocessor, compiler (I think), and
linker behaviour.

Greetings,

Andres Freund

---

From: Andrew Dunstan <[email-masked]>
Date: Tue, 15 Nov 2022 15:47:39 -0500

> e.g. installing 64bit libs into the directory containing 32bit libs.

You might end up surprising people who have installed from source for
years and will have the layout suddenly changed, especially on RedHat
flavored systems.

I can work around it in the buildfarm, which does make some assumptions
about the layout (e.g. in the cross version pg_upgrade stuff), by
explicitly using --libdir.

> linker. -ffreestanding will e.g. change preprocessor, compiler (I think), and

Well it sure looks odd.

cheers

andrew

---

From: Tom Lane <[email-masked]>
Date: Tue, 15 Nov 2022 16:08:35 -0500

> years and will have the layout suddenly changed, especially on RedHat

Yeah, I'm not too pleased with this idea either.  The people who want
to install according to some platform-specific plan have already figured
out how to do that.  People who are accustomed to the way PG has done
it in the past are not likely to think this is an improvement.	 

Also, unless you intend to drop the special cases involving whether
the install path string contains "postgres" or "pgsql", it's already
not platform-standard.

			regards, tom lane

---

From: Andres Freund <[email-masked]>
Date: Tue, 15 Nov 2022 15:40:42 -0800

Hi,

> > years and will have the layout suddenly changed, especially on RedHat

Just to make sure that's clear: meson defaults to lib/ or lib64/ (depending on
bitness obviously) on RedHat systems, not lib/i386-linux-gnu/ or
lib/x86_64-linux-gnu.

> it in the past are not likely to think this is an improvement.

I think that's a good argument to not change the default for configure, but
imo not a good argument for forcing 'lib' rather than the appropriate platform
default in the meson build, given that that already requires changing existing
recipes.

Small note: I didn't intentionally make that change during the meson porting
work, it's just meson's default.

I can live with forcing lib/, but I don't think it's the better solution long
term. And this seems like the best point for switching we're going to get.

We'd just have to add 'libdir=lib' to the default_options array in the
toplevel meson.build.

> the install path string contains "postgres" or "pgsql", it's already

For me that's the best argument for forcing 'lib'. Still not quite enough to
swing me around, because it's imo a pretty reasonable thing to want to install
a 32bit and 64bit libpq, and I don't think we should make that harder.

Somewhat relatedly, I wonder if we should have a better way to enable/disable
the 'pgsql' path logic. It's pretty annoying that prefix basically doesn't
work if it doesn't contain 'pgsql' or 'postgres'.

Greetings,

Andres Freund

---

From: Peter Eisentraut <[email-masked]>
Date: Wed, 16 Nov 2022 10:53:59 +0100

> work if it doesn't contain 'pgsql' or 'postgres'.

Could you explain this in more detail?

---

From: Andres Freund <[email-masked]>
Date: Wed, 16 Nov 2022 08:40:10 -0800

Hi,

> Could you explain this in more detail?

If I just want to install postgres into a prefix without 'postgresql' added in
a bunch of directories, e.g. because I already have pg-$version to be in the
prefix, there's really no good way to do so - you can't even specify
--sysconfdir or such, because we just override that path.

And because many of our binaries are major version specific you pretty much
need to include the major version in the prefix, making the 'postgresql' we
add redundant.

I think the easiest way today is to use a temporary prefix and then just
rename the installation path. But that obviously doesn't deal well with
rpaths, at least as long as we don't use relative rpaths.

Greetings,

Andres Freund

---

From: Tom Lane <[email-masked]>
Date: Wed, 16 Nov 2022 11:54:10 -0500

> --sysconfdir or such, because we just override that path.

At least for the libraries, the point of the 'postgresql' subdir IMO
is to keep backend-loadable extensions separate from random libraries.
It's not great that we may fail to do that depending on what the
initial part of the library path is.

I could get behind allowing the user to specify that path explicitly
and then not modifying it; but when we're left to our own devices
I think we should preserve that separation.

			regards, tom lane

---

From: Andres Freund <[email-masked]>
Date: Wed, 16 Nov 2022 09:07:47 -0800

Hi,

> initial part of the library path is.

Agreed, extensions really should never be in a path searched by the dynamic
linker, even if the prefix contains 'postgres'.

To me that's a separate thing from adding postgresql to datadir, sysconfdir,
includedir, docdir... On a green field I'd say the 'extension library'
directory should just always be extensions/ or such.

Greetings,

Andres Freund