Subject: When to drop src/tools/msvc support

From: Andres Freund <[email-masked]>
Date: Sat, 8 Apr 2023 12:10:07 -0700

Hi,

I'd planned to write this soon anyway, but it was just brought up in [1].

Originally we had planned to drop src/tools/msvc support shortly after meson
went in. Unfortunately, it took a bit longer than originally hoped for to
merge meson support and then longer than hoped to add buildfarm support. I
don't think there's been any buildfarm client release with meson support yet -
but we do have windows buildfarm animals using it.

Do we want to drop src/tools/msvc support in 16 (i.e. now), or do it early in
17?

I do have a set of patches removing src/tools/msvc. There are a few loose ends
that I know of (my eyes glaze over every time I try to reconcile the
src/tools/perl.m4 comments with the referenced comments in Mkvcbuild.pm), and
probably more small references that grep terms didn't find.

Greetings,

Andres Freund

[1] https://postgr.es/m/3598664.1680976414%40sss.pgh.pa.us

---

From: "Jonathan S. Katz" <[email-masked]>
Date: Sat, 8 Apr 2023 15:19:20 -0400

> probably more small references that grep terms didn't find.

(reads [1])

Can we treat this as an "open item" for completing the transition to 
meson for builds as part of v16?

With my personal hat on, it seems silly to wait until v17 to do this, 
and I don't see why we would want to wait. If there's limited risk in 
doing this and it'll make our builds both more stable + faster, it seems 
like we should just do it.

Thanks,

Jonathan

---

From: Tom Lane <[email-masked]>
Date: Sat, 08 Apr 2023 15:30:20 -0400

> Do we want to drop src/tools/msvc support in 16 (i.e. now), or do it early in

On the one hand, it feels like something we shouldn't do after
feature freeze.  On the other hand, continuing to maintain three
build systems is a real drag (although you could argue that there
shouldn't be much churn there until the tree opens for 17).

We clearly can't consider it in any case until the buildfarm
is prepared, with all the Windows animals updated to a compatible
client script.  I don't know what timeline Andrew has in mind
for that.

I guess I'd vote for pulling the trigger in v16 if we can get that
done by the end of April.  Once we're close to beta I think it
must wait for v17 to open.

			regards, tom lane

---

From: Robert Haas <[email-masked]>
Date: Mon, 10 Apr 2023 12:34:47 -0400

> must wait for v17 to open.

I think that sounds reasonable. It would be to the project's advantage
not to have to maintain three build systems for an extra year, but we
can't still be whacking things around right up until the moment we
expect to ship a beta.

However, if this is the direction we're going, we probably need to
give pgsql-packagers a heads up ASAP, because anybody who is still
relying on the MSVC system to build Windows binaries is presumably
going to need some time to adjust. If we rip out the build system
somebody is using a couple of weeks before beta, that might make it
difficult for that person to get the beta out promptly. And I think
there's probably more than just EDB who would be in that situation.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

---

From: Tom Lane <[email-masked]>
Date: Mon, 10 Apr 2023 12:56:07 -0400

> there's probably more than just EDB who would be in that situation.

Oh ... that's a good point.  Is there anyone besides EDB shipping
MSVC-built executables?  Would it even be practical to switch to
meson with a month-or-so notice?  Seems kind of tight, and it's
not like the packagers volunteered to make this switch.

Maybe we have to bite the bullet and keep MSVC for v16.
If we drop it as soon as v17 opens, there's probably not that
much incremental work involved compared to dropping for v16.

			regards, tom lane

---

From: Robert Haas <[email-masked]>
Date: Mon, 10 Apr 2023 13:34:35 -0400

> not like the packagers volunteered to make this switch.

I can't really speak to those questions with confidence.

Perhaps instead of telling pgsql-packagers what we're doing, we could
instead ask them if it would work for them if we did XYZ. Then we
could use that information to inform our decision-making.

---

From: Dave Page <[email-masked]>
Date: Mon, 10 Apr 2023 19:55:35 +0100

> could use that information to inform our decision-making.

Projects other than the EDB installers use the MSVC build system - e.g.
pgAdmin uses it’s own builds of libpq and other tools (psql, pg_dump etc)
that are pretty heavily baked into a fully automated build system (even the
build servers and all their requirements are baked into Ansible).

Changing that lot would be non-trivial, though certainly possible, and I
suspect we’re not the only ones doing that sort of thing.

-- 
-- 
Dave Page
https://pgsnake.blogspot.com

EDB Postgres
https://www.enterprisedb.com

---

From: Magnus Hagander <[email-masked]>
Date: Mon, 10 Apr 2023 22:39:28 +0200

> much incremental work involved compared to dropping for v16.

Not involved with any such build tasks anymore, but I think we can
definitely assume there are others than EDB who do that. It's also
used by people who build add-on modules to be loaded in the
EDB-installer-installed systems, I'm sure.

It seems a bit aggressive to those to drop the entire build system out
just before beta.

Thus, +1 on actually keeping it up and dropping it immediately as v17
opens, giving them a year of advantage. And probably updating the docs
(if anybody were to read them.. but at least then we tried) stating
that it's deprecated and will be removed in v17.

-- 
 Magnus Hagander
 Me: https://www.hagander.net/
 Work: https://www.redpill-linpro.com/

---

From: Tom Lane <[email-masked]>
Date: Mon, 10 Apr 2023 16:50:20 -0400

> that it's deprecated and will be removed in v17.

Yeah, I think that's the only feasible answer at this point.
Maybe a month or two back we could have done differently,
but there's not a lot of runway now.

Once we do drop src/tools/msvc from HEAD, we should make a point
of reminding -packagers about it, in hopes that they'll work on
the transition sooner than next May.

			regards, tom lane

---

From: Andres Freund <[email-masked]>
Date: Mon, 10 Apr 2023 15:27:38 -0700

Hi,

> suspect we’re not the only ones doing that sort of thing.

Do you have a link to the code for that, if it's open? Just to get an
impression for how hard it'd be to switch over?

Greetings,

Andres Freund

---

From: Andres Freund <[email-masked]>
Date: Mon, 10 Apr 2023 15:32:19 -0700

Hi,

> the transition sooner than next May.

So the plan is:

- add note to docs in HEAD that the src/tools/msvc style of build is
  deprecated
- give -packagers a HEADS up, once the deprecation notice has been added to
  the docs
- have a patch ready to drop src/tools/msvc from HEAD once 16 has branched off
  (i.e. a polished version of what I posted upthread)

On IM Thomas made some point about CI - I wonder if we should add building 16
with src/tools/msvc as an optional CI task? We can't enable it by default
(yet), because we'd not have enough resources to also run that for cfbot. Once
16 forked, we then could set to run automatically in the 16 branch, as cfbot
won't run those.

Greetings,

Andres Freund

---

From: Michael Paquier <[email-masked]>
Date: Tue, 11 Apr 2023 08:52:57 +0900

> 16 forked, we then could set to run automatically in the 16 branch, as cfbot

Getting a CI job able to do some validation for MSVC would be indeed
nice.  What's the plan in the buildfarm with this coverage?  Would all
the animals switch to meson (Chocolatey + StrawberryPerl, I assume)
for the job or will there still be some coverage with MSVC for v16
there?
--
Michael

---

From: "Jonathan S. Katz" <[email-masked]>
Date: Mon, 10 Apr 2023 19:53:15 -0400

> the transition sooner than next May.

[personal opinion, not RMT]

The last point would be my reasoning for "why not now" given deadlines 
are a pretty good motivator to get things done.

That said, if the plan is to do this "shortly thereafter" for v17 and it 
makes the transition easier, I'm all for that.

Jonathan

---

From: Magnus Hagander <[email-masked]>
Date: Tue, 11 Apr 2023 09:09:41 +0200

> impression for how hard it'd be to switch over?

The pgadmin docs/readme refers to
https://github.com/pgadmin-org/pgadmin4/tree/master/pkg/win32

It clearly doesn't have the full automation stuff, but appears to have
the parts about building the postgres dependency.

---

From: Dave Page <[email-masked]>
Date: Tue, 11 Apr 2023 09:05:31 +0100

> the parts about building the postgres dependency.

Yeah, that's essentially the manual process, though I haven't tested it in
a while. The Ansible stuff is not currently public. I suspect (or rather,
hope) that we can pull in all the additional packages required using
Chocolatey which shouldn't be too onerous.

Probably my main concern is that the Meson build can use the same version
of the VC++ compiler that we use (v14), which is carefully matched for
compatibility with all the various components, just in case anything passes
CRT pointers around. Python is the one thing we don't build ourselves on
Windows and the process will build modules like gssapi and psycopg (which
links with libpq of course), so we're basically following what they use.

-- 
Dave Page
Blog: https://pgsnake.blogspot.com
Twitter: @pgsnake

EDB: https://www.enterprisedb.com

---

From: Andrew Dunstan <[email-masked]>
Date: Tue, 11 Apr 2023 06:58:43 -0400

On 2023-04-11 Tu 04:05, Dave Page wrote:
> Yeah, that's essentially the manual process, though I haven't tested 
> it in a while. The Ansible stuff is not currently public. I suspect 
> (or rather, hope) that we can pull in all the additional packages 
> required using Chocolatey which shouldn't be too onerous.
>
> Probably my main concern is that the Meson build can use the same 
> version of the VC++ compiler that we use (v14), which is carefully 
> matched for compatibility with all the various components, just in 
> case anything passes CRT pointers around. Python is the one thing we 
> don't build ourselves on Windows and the process will build modules 
> like gssapi and psycopg (which links with libpq of course), so we're 
> basically following what they use.
>
>

For meson you just need to to "pip install meson ninja" in your python 
distro and you should be good to go (they will be installed in python's 
Scripts directory). Don't use chocolatey to install meson/ninja - I ran 
into issues doing that.

AFAICT meson will use whatever version of VC you have installed, 
although I have only been testing with VC2019.

cheers

andrew

--
Andrew Dunstan
EDB:https://www.enterprisedb.com

---

From: Dave Page <[email-masked]>
Date: Tue, 11 Apr 2023 12:54:42 +0100

On Tue, 11 Apr 2023 at 11:58, Andrew Dunstan <[email-masked]> wrote:

> AFAICT meson will use whatever version of VC you have installed, although
> I have only been testing with VC2019.
>
OK, that sounds easy enough then (famous last words!)

Thanks!

---

From: Andres Freund <[email-masked]>
Date: Tue, 11 Apr 2023 09:23:05 -0700

Hi,

On 2023-04-11 09:05:31 +0100, Dave Page wrote:
> CRT pointers around.

FWIW, Independent of meson, I don't think support for VC 2015 in postgres is
long for the world. Later versions of msvc have increased the C standard
compliance a fair bit... It's also a few years out of normal support.

I've not tested building with 2015, but I don't know of anything that should
prevent building with meson with it.  I am fairly sure that it can't build
tab-complete.c, but you're presumably not building with tab completion support
anyway?

Greetings,

Andres Freund

---

From: Alvaro Herrera <[email-masked]>
Date: Tue, 11 Apr 2023 19:44:20 +0200

> for the job or will there still be some coverage with MSVC for v16

If we keep MSVC support in 16, then I agree we should have a CI task for
it -- and IMO we should make ti automatically triggers whenever any
Makefile or meson.build is modified.  Hopefully we won't touch them
much now that the branch is feature-frozen, but it could still happen.

Do we have code for it already, even if incomplete?

-- 
Álvaro Herrera         PostgreSQL Developer  —  https://www.EnterpriseDB.com/

---

From: Peter Eisentraut <[email-masked]>
Date: Wed, 12 Apr 2023 10:54:54 +0200

> Do we want to drop src/tools/msvc support in 16 (i.e. now), or do it early in

Can you build using meson from a distribution tarball on Windows?