Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mXrFJ-0007sw-OO for psycopg@arkaria.postgresql.org; Tue, 05 Oct 2021 20:41:05 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1mXrFI-0001Gx-K7 for psycopg@arkaria.postgresql.org; Tue, 05 Oct 2021 20:41:04 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mXrFI-0001Gk-An for psycopg@lists.postgresql.org; Tue, 05 Oct 2021 20:41:04 +0000 Received: from mail-ua1-x934.google.com ([2607:f8b0:4864:20::934]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1mXrFF-0006VG-H0 for psycopg@lists.postgresql.org; Tue, 05 Oct 2021 20:41:03 +0000 Received: by mail-ua1-x934.google.com with SMTP id u11so213727uaw.3 for ; Tue, 05 Oct 2021 13:41:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=IghUGxL1IVMgoyCru3THarMeoMnvoCs0hVnC1ONr8Dw=; b=T3o1QWAsxho0rPFRj2ZK5645iWt6c5AM+QFGCwu7BweNIyuOrRfXWvcDruLsBxYt+4 5ZG5eN+k+KaEkyiV5rcHUyMiCLJfFJGvo5VAw5AGxF+wDCh2NmYsn4IwI351xl5IOpvj xESyhz6E0zxbmbu/CMK51loNteLizfNzzbM+hY13I3pwAdoGlhY9u4II/tHIISv3Q2GU NobHxkecJGjmzW0w5yZbzEspyIFpjlAu92LV76W+81ZlIJfv9K9/weZYT7UpCNGnGMqB 9U3UVtjjdMmu65E7x+fy/O3iOTFqXC9Fkc5LwPgHn7wrIH3n/Aa1OrSBWphO7U8q/QmA 4BqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=IghUGxL1IVMgoyCru3THarMeoMnvoCs0hVnC1ONr8Dw=; b=pQrmzAXomFYVdm0djGrtL03o4m5MdVF1kRzqnHZo4l0fqYRrJU+L7s7zB9b+/mCrKW ej6NaTFEtcpn0rORNpBfbYtH96bbUBqZeApS8DR6FUCmsKlplWrq5IoApO+ymB0ICtBg OOj1pMR0EEfqZpXSABblwT0wpLL8OvJhpMEducF3vz04TDg3u8ulHkRljAnR8/DsWU/e RZBXs1xF77LZdG0ZPCLy40h2H+u0SRZQK0Hib2snhItRSljChM3P1ON6qgiq1ERJVdLk 6bNkXhXOjB5lFHt1IP65zaw2puw3XqxqwoBJPiXLI3HOYUTbV6sm0BEelUQmB5zaV+G8 epig== X-Gm-Message-State: AOAM532PE7XVaAKXb3xEFw+eUQ7b8EMcIipTShur019uHwJerH998Nvc yncKZMeNTsJP7efNXCR2+/vgAw7/zNuoktbbjpc= X-Google-Smtp-Source: ABdhPJyo8pPTb6PihEvF+8CqWDkt/I1x0GW8qq4ZFp3YC32VChp9VYv/wHO/9CVj9iBImV5ZjbNsUMh/kTl0ZWpBPWI= X-Received: by 2002:ab0:7587:: with SMTP id q7mr15125222uap.92.1633466459632; Tue, 05 Oct 2021 13:40:59 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Dan Davis Date: Tue, 5 Oct 2021 16:40:48 -0400 Message-ID: Subject: Re: How to build statically on Windows To: Jason Erickson Cc: Daniele Varrazzo , psycopg@lists.postgresql.org Content-Type: multipart/alternative; boundary="0000000000001437ca05cda10cc5" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --0000000000001437ca05cda10cc5 Content-Type: text/plain; charset="UTF-8" Jason, I tried to call the appveyor script, but it was looking for a number of Environment variables I did not have. I got through PY_VER=39 and then thought better of it. Do you think it would be easier to do it this way: * fork psycopg2 * make my own appveyor account that builds the versions I need... Simulate the appveyor environment on my own workstation Wow, what a lot I've forgotten since before manylinux1 - I used to do this for lxml, PyCrypto, and a lot of other packages :). Still, they were all more or less one-offs - I never got as automated as modern DevOps allows these days. On Tue, Oct 5, 2021 at 2:11 PM Jason Erickson wrote: > Hi Dan, > > Yeah, unfortunately --static-libpq doesn't do what it should anymore. We > should remove that option. > > When building pyscopg2 from source, the binary Windows Postgres packages > does not include the statically linked library (In fairness, I haven't > checked the latest packages, but that is how it has been in the past). The > library included with the binary packages was the DLL import library and it > was named libpq.lib. This is an issue, because originally when you built > from source on windows, the file libpq.lib was the static link library, > whereas the DLL import library was named libpqdll.lib. So we have a name > inconsistency and no way to link statically with the Postgres binary > distribution. > > This got even more convoluted a few major versions back with the source of > Postgres, as the Windows perl build scripts quit creating the build file > for the static link library, only building the DLL import library AND > naming it libpq.lib. With our Appveyor build script, I cheated and > modified the perl script to build the library instead of the DLL at this > line: > file_replace('Mkvcbuild.pm', "'libpq', 'dll'", "'libpq', 'lib'") > > With that said, I am not happy with that solution and always intended to > revisit the setup script portion for windows, but always had more questions > than answers, some of them: > * If static libraries are not part of the Postgres binary distribution (or > even the build from source option by default), do we concern ourselves with > them? Personally I prefer static libraries because I think it has less > support issues, but??? > * If people are building psycopg from source, what libraries do we assume > they have installed? pg_config probably would not be working, so > include/library paths would have to be passed. What about other > dependencies (ie openssl, still use the has_ssl flag?)? There are slightly > different link libraries between some Postgres, so versions matter, too. > * Do we try to differentiate between the DLL import library and the static > library since the name can be the same between them (size?)? > * Or do we say heck with it and just link against a file named libpq.lib, > letting the builder point to the libpq they want to use? > > Maybe an option is to have setup.py on windows call the appveyor script > (with some modifications) and build everything from scratch? It takes > about 40 minutes to build everything, tho, and does require some storage > space for the source and build files, so not the best solution. > > -jason > > > On Mon, Oct 4, 2021 at 5:13 PM Daniele Varrazzo < > daniele.varrazzo@gmail.com> wrote: > >> Hi Dan, >> >> On Tue, 5 Oct 2021 at 01:01, Dan Davis wrote: >> > >> > Can anyone give me a solution to build psycopg2 statically on Windows? >> > >> > I have succeeded in building it, but when I run dumpbin /dependents on >> the generated file (the PYD file), it still depends on libpq.dll even when >> I pass --static-libpq. >> >> I haven't personally used --static-libpq in a long time, and neither >> have I used windows for a while. As far as I know that part of the set >> up might have bitrotten. >> >> If anyone can help Dan it would be appreciated. >> >> Dan, there is a ticket/MR in the tracker of which I've never been able >> to make completely sense: https://github.com/psycopg/psycopg2/pull/758 >> Would you like to check if that's the issue that doesn't allow >> building the lib statically and if so can you propose a MR or just >> acknowledge that the proposed one works as expected? >> >> Thank you everyone >> >> -- Daniele >> >> >> --0000000000001437ca05cda10cc5 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Jason,

I tried to call the appveyor scr= ipt, but it was looking for a number of Environment variables I did not hav= e.=C2=A0 I got through PY_VER=3D39 and then thought better of it.=C2=A0

Do you think it would be easier to do it this way:
* fork psycopg2
* make my own appveyor account that build= s the=C2=A0versions I need...

Simulate the appveyo= r environment on my own workstation

Wow, what a lo= t I've forgotten since before manylinux1 - I used to do this for lxml, = PyCrypto, and a lot of other packages :).=C2=A0 =C2=A0Still, they were all = more or less one-offs - I never got as automated as modern DevOps allows th= ese days.



On Tue, Oct 5, 2021 at 2:11 PM= Jason Erickson <jerickso@st= ickpeople.com> wrote:
Hi Dan,

Yeah, u= nfortunately --static-libpq doesn't do what it should anymore.=C2=A0 We= should remove that option.

When building pyscopg2= from source, the binary Windows Postgres packages does not include the sta= tically linked library (In fairness, I haven't checked the latest=C2=A0= packages, but that is how it has been in the past).=C2=A0 The library inclu= ded with the binary packages was the DLL import library and it was named li= bpq.lib.=C2=A0 This is an issue, because originally when you built from sou= rce on windows, the file libpq.lib was the static link library, whereas the= DLL import library was named libpqdll.lib.=C2=A0 So we have a name inconsi= stency and=C2=A0no way to link statically with the Postgres binary distribu= tion.

This got even more convoluted = a few major versions back with the source of Postgres, as the Windows perl = build scripts quit creating the build file for the static link library, onl= y building the DLL import library AND naming it libpq.lib.=C2=A0 With our A= ppveyor build script, I cheated and modified the perl script to build the l= ibrary instead of the DLL at this line:
=C2=A0 =C2=A0 =C2=A0 file_replace('Mkvcbuild.pm', "'= ;libpq', 'dll'", "'libpq', '= lib'")

With that said, I am not ha= ppy with that solution and always intended to revisit the setup script port= ion for windows, but always had more questions than answers, some of them:<= /div>
* If static libraries are not part of the Postgres=C2= =A0binary distribution (or even the build from source option by default), d= o we concern ourselves with them? Personally I prefer static libraries beca= use I think it has less support issues, but???
* If people ar= e building psycopg from source, what libraries do we assume they have insta= lled?=C2=A0 pg_config probably would not be working, so include/library pat= hs would have to be passed. What about other dependencies (ie openssl, stil= l use the has_ssl flag?)?=C2=A0 There are slightly different link libraries= between some Postgres, so versions matter, too.
* Do we try = to differentiate between the DLL import library and the static library sinc= e the name can be the same between them (size?)?
* Or do we say h= eck with it and just link against a file named libpq.lib, letting the build= er point to the libpq they want to use?

Maybe an o= ption is to have setup.py on windows call the appveyor script (with some mo= difications) and build everything from scratch?=C2=A0 It takes about 40 min= utes to build everything, tho, and does require some storage space for the = source and build files, so not the best solution.

= -jason


On Mon, Oct 4, 2021 at 5:13 PM Daniele Varrazzo = <daniele= .varrazzo@gmail.com> wrote:
Hi Dan,

On Tue, 5 Oct 2021 at 01:01, Dan Davis <dansmood@gmail.com> wrote:
>
> Can anyone give me a solution to build psycopg2 statically on Windows?=
>
> I have succeeded in building it, but when I run dumpbin /dependents on= the generated file (the PYD file), it still depends on libpq.dll even when= I pass --static-libpq.

I haven't personally used --static-libpq in a long time, and neither have I used windows for a while. As far as I know that part of the set
up might have bitrotten.

If anyone can help Dan it would be appreciated.

Dan, there is a ticket/MR in the tracker of which I've never been able<= br> to make completely sense: https://github.com/psycopg/psy= copg2/pull/758
Would you like to check if that's the issue that doesn't allow
building the lib statically and if so can you propose a MR or just
acknowledge that the proposed one works as expected?

Thank you everyone

-- Daniele


--0000000000001437ca05cda10cc5--