pg.ddx.io  pgsql-bugs@postgresql.org mailing list archive  
help / color / mirror / Atom feed
BUG #15789: libpq compilation with OpenSSL 1.1.1b fails on Windows with Visual Studio 2017
18+ messages / 5 participants
[nested] [flat]

Commits from this thread:
  2019-06-26 d993e0fb82 0e08a3a1f5 a559805597 5329606693 d908a0087e 05b2758c39  Add support for OpenSSL 1.1.0 and newer versions in MSVC scripts
* BUG #15789: libpq compilation with OpenSSL 1.1.1b fails on Windows with Visual Studio 2017
@ 2019-05-02 13:57  PG Bug reporting form <noreply@postgresql.org>
  0 siblings, 1 reply; 18+ messages in thread

From: PG Bug reporting form @ 2019-05-02 13:57 UTC (permalink / raw)
  To: pgsql-bugs@lists.postgresql.org; +Cc: serpashk@gmail.com

The following bug has been logged on the website:

Bug reference:      15789
Logged by:          Sergey Pashkov
Email address:      serpashk@gmail.com
PostgreSQL version: 11.2
Operating system:   Windows 10
Description:        

libssh2 and libpq are used in the same product.
It is necessary to use OpenSSL 1.1.1b as it includes EdDSA support required
for libssh2.

Here are parts of our build scripts.

1. Building OpenSSL:
perl Configure VC-WIN64A no-shared no-asm enable-ssl3 enable-ssl3-method
nmake

2. Building libpq:
XCOPY /s /i "%OPENSSL_SRCS%" "%OPENSSL_PATH%"
MKDIR "%OPENSSL_PATH%\lib"
COPY "%THIRD_LIBS%\libssl_64.lib" "%OPENSSL_PATH%\lib\ssleay32.lib"
COPY "%THIRD_LIBS%\libcrypto_64.lib" "%OPENSSL_PATH%\lib\libeay32.lib"

perl mkvcbuild.pl
COPY config_default.pl config.pl
sed -i "s/openssl   => undef/openssl   => \"%OPENSSL_PATH_ESC%\"/g"
config.pl
perl mkvcbuild.pl
msbuild pgsql.sln /t:interfaces\libpq /p:Configuration="Release" /m

The following errors are encountered:

       "C:\Users\User\AppData\Local\Temp\postgres_64\libpq.vcxproj" (default
target) (6) ->
       (ClCompile target) ->
        
c:\users\user\appdata\local\temp\postgres_64\src\interfaces\libpq\fe-secure-openssl.c(1467):
error C2037: left
        of 'ptr' specifies undefined struct/union 'bio_st'
[C:\Users\User\AppData\Local\Temp\postgres_64\libpq.vcxproj]
        
c:\users\user\appdata\local\temp\postgres_64\src\interfaces\libpq\fe-secure-openssl.c(1467):
error C2198: 'pqs
       ecure_raw_read': too few arguments for call
[C:\Users\User\AppData\Local\Temp\postgres_64\libpq.vcxproj]
        
c:\users\user\appdata\local\temp\postgres_64\src\interfaces\libpq\fe-secure-openssl.c(1497):
error C2037: left
        of 'ptr' specifies undefined struct/union 'bio_st'
[C:\Users\User\AppData\Local\Temp\postgres_64\libpq.vcxproj]
        
c:\users\user\appdata\local\temp\postgres_64\src\interfaces\libpq\fe-secure-openssl.c(1497):
error C2198: 'pqs
       ecure_raw_write': too few arguments for call
[C:\Users\User\AppData\Local\Temp\postgres_64\libpq.vcxproj]
        
c:\users\user\appdata\local\temp\postgres_64\src\interfaces\libpq\fe-secure-openssl.c(1556):
error C2027: use
       of undefined type 'bio_method_st'
[C:\Users\User\AppData\Local\Temp\postgres_64\libpq.vcxproj]
        
c:\users\user\appdata\local\temp\postgres_64\src\interfaces\libpq\fe-secure-openssl.c(1559):
error C2027: use
       of undefined type 'bio_method_st'
[C:\Users\User\AppData\Local\Temp\postgres_64\libpq.vcxproj]
        
c:\users\user\appdata\local\temp\postgres_64\src\interfaces\libpq\fe-secure-openssl.c(1560):
error C2037: left
        of 'bread' specifies undefined struct/union 'bio_method_st'
[C:\Users\User\AppData\Local\Temp\postgres_64\libpq
       .vcxproj]
        
c:\users\user\appdata\local\temp\postgres_64\src\interfaces\libpq\fe-secure-openssl.c(1561):
error C2037: left
        of 'bwrite' specifies undefined struct/union 'bio_method_st'
[C:\Users\User\AppData\Local\Temp\postgres_64\libp
       q.vcxproj]
        
c:\users\user\appdata\local\temp\postgres_64\src\interfaces\libpq\fe-secure-openssl.c(1587):
error C2037: left
        of 'ptr' specifies undefined struct/union 'bio_st'
[C:\Users\User\AppData\Local\Temp\postgres_64\libpq.vcxproj]

Headers of the recent OpenSSL versions don't include bio_st definition, as
it was in 1.0.2.

If I manually define the following macros in pg_config.h:

HAVE_BIO_GET_DATA
HAVE_BIO_METH_NEW 

- compilation passes but SSL connection cannot be established anyway.

This question was raised in the following thread:
https://www.postgresql.org/message-id/flat/CAAw-Mseg9JYpp%3DA%3D51HR3rKiQtbvT0MWw%2BaYFwNeJEbdNr%3DC...

No solution was proposed.

Thank you!



^ permalink  raw  reply  [nested|flat] 18+ messages in thread

* Re: BUG #15789: libpq compilation with OpenSSL 1.1.1b fails on Windows with Visual Studio 2017
@ 2019-05-28 05:31  Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>
  parent: PG Bug reporting form <noreply@postgresql.org>
  0 siblings, 1 reply; 18+ messages in thread

From: Juan José Santamaría Flecha @ 2019-05-28 05:31 UTC (permalink / raw)
  To: serpashk@gmail.com; pgsql-bugs@lists.postgresql.org

On Thu, May 2, 2019 at 3:58 PM PG Bug reporting form <noreply@postgresql.org>
wrote:

>
> This question was raised in the following thread:
>
> https://www.postgresql.org/message-id/flat/CAAw-Mseg9JYpp%3DA%3D51HR3rKiQtbvT0MWw%2BaYFwNeJEbdNr%3DC...
>
> No solution was proposed.
>
>
Can you check if you get a sane compilation with the attached patch?

It is just a compilation issue, once the software is built with the 1.1.x
libraries you should not have any further problems to use SSL.

Regards,

Juan José Santamaría Flecha

Attachments:

0001_windows_openssl_1.1.0_build_v11.patch (application/x-patch, 3.7K) download diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm index 78db247..31a7a77 100644 --- a/src/tools/msvc/Solution.pm +++ b/src/tools/msvc/Solution.pm @@ -174,7 +174,17 @@ sub GenerateFiles if ($self->{options}->{asserts}); print $o "#define USE_LDAP 1\n" if ($self->{options}->{ldap}); print $o "#define HAVE_LIBZ 1\n" if ($self->{options}->{zlib}); - print $o "#define USE_OPENSSL 1\n" if ($self->{options}->{openssl}); + if ($self->{options}->{openssl}) + { + print $o "#define USE_OPENSSL 1\n"; + # We need to know if we are using version 1.1.0 or above + my ($major, $minor) = $self->GetOpenSSLVersion(); + if ($major == 1 && $minor == 1) + { + print $o "#define HAVE_BIO_GET_DATA 1\n"; + print $o "#define HAVE_BIO_METH_NEW 1\n"; + } + } print $o "#define ENABLE_NLS 1\n" if ($self->{options}->{nls}); print $o "#define BLCKSZ ", 1024 * $self->{options}->{blocksize}, @@ -579,21 +589,52 @@ sub AddProject if ($self->{options}->{openssl}) { $proj->AddIncludeDir($self->{options}->{openssl} . '\include'); - if (-e "$self->{options}->{openssl}/lib/VC/ssleay32MD.lib") + my ($major, $minor) = $self->GetOpenSSLVersion(); + if ($major == 1 && $minor == 1) { - $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\VC\ssleay32.lib', 1); - $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\VC\libeay32.lib', 1); + # Startint at version 1.1.0 OpenSSL have changed their library names from: + # libeay to libcrypto + # ssleay to libssl + if (-e "$self->{options}->{openssl}/lib/VC/libssl32MD.lib") + { + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\VC\libssl32.lib', 1); + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\VC\libcrypto32.lib', 1); + } + elsif (-e "$self->{options}->{openssl}/lib/VC/libssl64MD.lib") + { + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\VC\libssl64.lib', 1); + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\VC\libcrypto64.lib', 1); + } + else + { + # We don't expect the config-specific library to be here, + # so don't ask for it in last parameter + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libssl.lib', 0); + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libcrypto.lib', 0); + } } else { - # We don't expect the config-specific library to be here, - # so don't ask for it in last parameter - $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\ssleay32.lib', 0); - $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\libeay32.lib', 0); + if (-e "$self->{options}->{openssl}/lib/VC/ssleay32MD.lib") + { + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\VC\ssleay32.lib', 1); + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\VC\libeay32.lib', 1); + } + else + { + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\ssleay32.lib', 0); + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libeay32.lib', 0); + } } } if ($self->{options}->{nls}) @@ -955,4 +996,26 @@ MinimumVisualStudioVersion = $self->{MinimumVisualStudioVersion} |; } +sub GetOpenSSLVersion +{ + my $self = shift; + + # Attempt to get OpenSSL version and location. + # Assume openssl.exe in specified dir. + my $opensslprog = '\bin\openssl.exe version 2>&1'; + my $opensslcmd = + $self->{options}->{openssl} . $opensslprog; + my $sslout = `$opensslcmd`; + die "Unable to determine OpenSSL: The openssl.exe command was not found." if $?; + + if ($sslout =~ /(\d+)\.(\d+)\.(\d+)/m) + { + return ($1, $2) + } + else + { + die "Unable to determine OpenSSL version: The openssl.exe version could not be determined."; + } +} + 1;
^ permalink raw reply [nested|flat] 18+ messages in thread

* Re: BUG #15789: libpq compilation with OpenSSL 1.1.1b fails on Windows with Visual Studio 2017
@ 2019-05-28 10:58  Michael Paquier <michael@paquier.xyz>
  parent: Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>
  0 siblings, 1 reply; 18+ messages in thread

From: Michael Paquier @ 2019-05-28 10:58 UTC (permalink / raw)
  To: Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>; +Cc: serpashk@gmail.com; pgsql-bugs@lists.postgresql.org

On Tue, May 28, 2019 at 07:31:55AM +0200, Juan José Santamaría Flecha wrote:
> Can you check if you get a sane compilation with the attached patch?
> 
> It is just a compilation issue, once the software is built with the 1.1.x
> libraries you should not have any further problems to use SSL.

Could you add your patch to the upcoming commit fest please?  Here it
is:
https://commitfest.postgresql.org/23/

The scripts in src/tools/msvc/ make efforts for being able to compile
with OpenSSL 1.0.2 which is the latest LTS version of upstream, but we
lack facility to make them more dynamic depending on the version of
OpenSSL so as the compile flags of pg_config.h can be enforced
correctly.  So what you are doing in GetOpenSSLVersion() is something
that we are going to need badly, and OpenSSL has broken many
interfaces between 1.0.2 and 1.1.0.

+   # Startint at version 1.1.0 OpenSSL have changed their library names from:
+   #     libeay to libcrypto
+   #     ssleay to libssl

s/startint/starting/
Are these from the installers we recommend in the docs?  I mean these
ones:
https://slproweb.com/products/Win32OpenSSL.html

+   if (-e "$self->{options}->{openssl}/lib/VC/libssl32MD.lib")
Why not using a version-specific logic here?

+   my ($major, $minor) = $self->GetOpenSSLVersion();
+   if ($major == 1 && $minor == 1)
+   {
+       print $o "#define HAVE_BIO_GET_DATA 1\n";
+       print $o "#define HAVE_BIO_METH_NEW 1\n";
+   }
I think that you are missing HAVE_OPENSSL_INIT_SSL and
HAVE_ASN1_STRING_GET0_DATA here.  Please see commit message of
bde64eb.
--
Michael

Attachments:

  signature.asc (application/pgp-signature, 832B) download

^ permalink  raw  reply  [nested|flat] 18+ messages in thread

* Re: BUG #15789: libpq compilation with OpenSSL 1.1.1b fails on Windows with Visual Studio 2017
@ 2019-05-31 14:11  Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>
  parent: Michael Paquier <michael@paquier.xyz>
  0 siblings, 1 reply; 18+ messages in thread

From: Juan José Santamaría Flecha @ 2019-05-31 14:11 UTC (permalink / raw)
  To: Michael Paquier <michael@paquier.xyz>; +Cc: serpashk@gmail.com; pgsql-bugs@lists.postgresql.org

On Tue, May 28, 2019 at 12:58 PM Michael Paquier <michael@paquier.xyz>
wrote:

>
> Could you add your patch to the upcoming commit fest please?  Here it
> is:
> https://commitfest.postgresql.org/23/
>
>
Sure, done as  [1]. The attached patch is still for 11, will it be back
patched?


>
> +   # Startint at version 1.1.0 OpenSSL have changed their library names
> from:
> +   #     libeay to libcrypto
> +   #     ssleay to libssl
>
> s/startint/starting/
> Are these from the installers we recommend in the docs?  I mean these
> ones:
> https://slproweb.com/products/Win32OpenSSL.html
>
>
The change in the library names comes directly from OpenSSL [2].


> +   if (-e "$self->{options}->{openssl}/lib/VC/libssl32MD.lib")
> Why not using a version-specific logic here?
>
>
The version logic is just before that:

+ my ($major, $minor) = $self->GetOpenSSLVersion();
+ if ($major == 1 && $minor == 1)

I guess that what you mean is, why testing the 32/64 bits using the
libraries instead of using the 'platform'? I try to make it clearer in this
version.


> +   my ($major, $minor) = $self->GetOpenSSLVersion();
> +   if ($major == 1 && $minor == 1)
> +   {
> +       print $o "#define HAVE_BIO_GET_DATA 1\n";
> +       print $o "#define HAVE_BIO_METH_NEW 1\n";
> +   }
> I think that you are missing HAVE_OPENSSL_INIT_SSL and
> HAVE_ASN1_STRING_GET0_DATA here.  Please see commit message of
> bde64eb.
>
>
Yes, you are right. Since those do not break the compilation between 1.0.2
and 1.1.0 I did not notice them.

Regards,

Juan José Santamaría Flecha

[1] https://commitfest.postgresql.org/23/2134/
[2] https://mta.openssl.org/pipermail/openssl-dev/2016-August/008351.html

Attachments:

0001_windows_openssl_1.1.0_build_PG11_v1.patch (application/octet-stream, 4.2K) download diff --git a/src/tools/msvc/Project.pm b/src/tools/msvc/Project.pm index 0d35546..291ca86 100644 --- a/src/tools/msvc/Project.pm +++ b/src/tools/msvc/Project.pm @@ -130,17 +130,19 @@ sub AddReference sub AddLibrary { - my ($self, $lib, $dbgsuffix) = @_; + my ($self, $lib, $slib) = @_; + my $libcfg = "MD"; + my $xlib = defined $slib ? $slib : ''; - if ($lib =~ m/\s/) + $xlib =~ s/\.lib$/$libcfg.lib/; + if ($xlib eq '' || ! -e $xlib) { - $lib = '&quot;' . $lib . "&quot;"; + push @{ $self->{libraries} }, $lib; } - - push @{ $self->{libraries} }, $lib; - if ($dbgsuffix) + else { - push @{ $self->{suffixlib} }, $lib; + push @{ $self->{libraries} }, $slib; + push @{ $self->{suffixlib} }, $slib; } return; } diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm index 78db247..6560ba0 100644 --- a/src/tools/msvc/Solution.pm +++ b/src/tools/msvc/Solution.pm @@ -174,7 +174,19 @@ sub GenerateFiles if ($self->{options}->{asserts}); print $o "#define USE_LDAP 1\n" if ($self->{options}->{ldap}); print $o "#define HAVE_LIBZ 1\n" if ($self->{options}->{zlib}); - print $o "#define USE_OPENSSL 1\n" if ($self->{options}->{openssl}); + if ($self->{options}->{openssl}) + { + print $o "#define USE_OPENSSL 1\n"; + # We need to know if we are using version 1.1.0 or above + my ($major, $minor) = $self->GetOpenSSLVersion(); + if ($major == 1 && $minor == 1) + { + print $o "#define HAVE_BIO_GET_DATA 1\n"; + print $o "#define HAVE_BIO_METH_NEW 1\n"; + print $o "#define HAVE_OPENSSL_INIT_SSL 1\n"; + print $o "#define HAVE_ASN1_STRING_GET0_DATA 1\n"; + } + } print $o "#define ENABLE_NLS 1\n" if ($self->{options}->{nls}); print $o "#define BLCKSZ ", 1024 * $self->{options}->{blocksize}, @@ -579,21 +591,39 @@ sub AddProject if ($self->{options}->{openssl}) { $proj->AddIncludeDir($self->{options}->{openssl} . '\include'); - if (-e "$self->{options}->{openssl}/lib/VC/ssleay32MD.lib") + my ($major, $minor) = $self->GetOpenSSLVersion(); + if ($major == 1 && $minor == 1) { - $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\VC\ssleay32.lib', 1); - $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\VC\libeay32.lib', 1); + # Starting at version 1.1.0 OpenSSL have changed their library names from: + # libeay to libcrypto + # ssleay to libssl + if ($self->{platform} eq 'Win32') + { + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libssl.lib', + $self->{options}->{openssl} . '\lib\VC\libssl32.lib'); + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libcrypto.lib', + $self->{options}->{openssl} . '\lib\VC\libcrypto32.lib'); + } + else + { + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libssl.lib', + $self->{options}->{openssl} . '\lib\VC\libssl64.lib'); + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libcrypto.lib', + $self->{options}->{openssl} . '\lib\VC\libcrypto64.lib'); + } } else { - # We don't expect the config-specific library to be here, - # so don't ask for it in last parameter $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\ssleay32.lib', 0); + $self->{options}->{openssl} . '\lib\ssleay32.lib', + $self->{options}->{openssl} . '\lib\VC\ssleay32.lib'); $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\libeay32.lib', 0); + $self->{options}->{openssl} . '\lib\libeay32.lib', + $self->{options}->{openssl} . '\lib\VC\libeay32.lib'); } } if ($self->{options}->{nls}) @@ -955,4 +985,25 @@ MinimumVisualStudioVersion = $self->{MinimumVisualStudioVersion} |; } +sub GetOpenSSLVersion +{ + my $self = shift; + + # Attempt to get OpenSSL version and location. + # Assume openssl.exe in specified dir. + my $opensslprog = '\bin\openssl.exe version 2>&1'; + my $opensslcmd = '"' . $self->{options}->{openssl} . '"' . $opensslprog; + my $sslout = `$opensslcmd`; + die "Unable to determine OpenSSL: The openssl.exe command was not found." if $?; + + if ($sslout =~ /(\d+)\.(\d+)\.(\d+)/m) + { + return ($1, $2) + } + else + { + die "Unable to determine OpenSSL version: The openssl.exe version could not be determined."; + } +} + 1;
^ permalink raw reply [nested|flat] 18+ messages in thread

* Re: BUG #15789: libpq compilation with OpenSSL 1.1.1b fails on Windows with Visual Studio 2017
@ 2019-05-31 17:17  Michael Paquier <michael@paquier.xyz>
  parent: Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>
  0 siblings, 1 reply; 18+ messages in thread

From: Michael Paquier @ 2019-05-31 17:17 UTC (permalink / raw)
  To: Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>; +Cc: serpashk@gmail.com; pgsql-bugs@lists.postgresql.org

On Fri, May 31, 2019 at 04:11:37PM +0200, Juan José Santamaría Flecha wrote:
> Sure, done as  [1]. The attached patch is still for 11, will it be back
> patched?

I don't see a reason why we should not back-patch something like what
you propose where build with OpenSSL 1.1.0 is supported (aka down to
9.5) if that proves to be reliable enough.
--
Michael

Attachments:

  signature.asc (application/pgp-signature, 832B) download

^ permalink  raw  reply  [nested|flat] 18+ messages in thread

* Re: BUG #15789: libpq compilation with OpenSSL 1.1.1b fails on Windows with Visual Studio 2017
@ 2019-06-03 10:00  Sandeep Thakkar <sandeep.thakkar@enterprisedb.com>
  parent: Michael Paquier <michael@paquier.xyz>
  0 siblings, 1 reply; 18+ messages in thread

From: Sandeep Thakkar @ 2019-06-03 10:00 UTC (permalink / raw)
  To: Michael Paquier <michael@paquier.xyz>; +Cc: Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>; serpashk@gmail.com; pgsql-bugs@lists.postgresql.org

On Fri, May 31, 2019 at 10:47 PM Michael Paquier <michael@paquier.xyz>
wrote:

> On Fri, May 31, 2019 at 04:11:37PM +0200, Juan José Santamaría Flecha
> wrote:
> > Sure, done as  [1]. The attached patch is still for 11, will it be back
> > patched?
>
> I don't see a reason why we should not back-patch something like what
> you propose where build with OpenSSL 1.1.0 is supported (aka down to
> 9.5) if that proves to be reliable enough.
>

OpenSSL 1.1.0 and 1.0.2 both are going out of Support in 2019 and PG 9.4 is
supported till Feb2020.
Ideally, 1.1.1x support should be backpatched until 9.4



> --
> Michael
>


-- 
Sandeep Thakkar

^ permalink  raw  reply  [nested|flat] 18+ messages in thread

* Re: BUG #15789: libpq compilation with OpenSSL 1.1.1b fails on Windows with Visual Studio 2017
@ 2019-06-03 23:51  Michael Paquier <michael@paquier.xyz>
  parent: Sandeep Thakkar <sandeep.thakkar@enterprisedb.com>
  0 siblings, 1 reply; 18+ messages in thread

From: Michael Paquier @ 2019-06-03 23:51 UTC (permalink / raw)
  To: Sandeep Thakkar <sandeep.thakkar@enterprisedb.com>; +Cc: Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>; serpashk@gmail.com; pgsql-bugs@lists.postgresql.org

On Mon, Jun 03, 2019 at 03:30:49PM +0530, Sandeep Thakkar wrote:
> OpenSSL 1.1.0 and 1.0.2 both are going out of Support in 2019 and PG 9.4 is
> supported till Feb2020.
> Ideally, 1.1.1x support should be backpatched until 9.4

There is no point to patch a stable branch if it has no support for
the OpenSSL version we target.  Now, bb132cdd has added support for
OpenSSL 1.1.0 in 9.4, so there is actually no reason to not patch 9.4
as well and you are right.  Missed that in my initial lookup of the
commit logs as this was applied 7 months after the 9.5~ portions.
--
Michael

Attachments:

  signature.asc (application/pgp-signature, 832B) download

^ permalink  raw  reply  [nested|flat] 18+ messages in thread

* Re: BUG #15789: libpq compilation with OpenSSL 1.1.1b fails on Windows with Visual Studio 2017
@ 2019-06-05 07:29  Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>
  parent: Michael Paquier <michael@paquier.xyz>
  0 siblings, 1 reply; 18+ messages in thread

From: Juan José Santamaría Flecha @ 2019-06-05 07:29 UTC (permalink / raw)
  To: Michael Paquier <michael@paquier.xyz>; +Cc: Sandeep Thakkar <sandeep.thakkar@enterprisedb.com>; serpashk@gmail.com; pgsql-bugs@lists.postgresql.org

On Tue, Jun 4, 2019 at 1:51 AM Michael Paquier <michael@paquier.xyz> wrote:

> On Mon, Jun 03, 2019 at 03:30:49PM +0530, Sandeep Thakkar wrote:
> > OpenSSL 1.1.0 and 1.0.2 both are going out of Support in 2019 and PG 9.4
> is
> > supported till Feb2020.
> > Ideally, 1.1.1x support should be backpatched until 9.4
>
> There is no point to patch a stable branch if it has no support for
> the OpenSSL version we target.  Now, bb132cdd has added support for
> OpenSSL 1.1.0 in 9.4, so there is actually no reason to not patch 9.4
> as well and you are right.
>

The patches are intended to support OpenSSL 1.1.x, including 1.1.1x.

The attached patches are meant for all supported versions and HEAD.

Regards,

Juan José Santamaría Flecha

Attachments: download all (4 files, .tar.gz)

0001_windows_openssl_1.1.0_build_PG10_v1.patch (application/octet-stream, 4.2K) download diff --git a/src/tools/msvc/Project.pm b/src/tools/msvc/Project.pm index 9817b94..52b51de 100644 --- a/src/tools/msvc/Project.pm +++ b/src/tools/msvc/Project.pm @@ -124,18 +124,21 @@ sub AddReference sub AddLibrary { - my ($self, $lib, $dbgsuffix) = @_; + my ($self, $lib, $slib) = @_; + my $libcfg = "MD"; + my $xlib = defined $slib ? $slib : ''; - if ($lib =~ m/\s/) + $xlib =~ s/\.lib$/$libcfg.lib/; + if ($xlib eq '' || ! -e $xlib) { - $lib = '&quot;' . $lib . "&quot;"; + push @{ $self->{libraries} }, $lib; } - - push @{ $self->{libraries} }, $lib; - if ($dbgsuffix) + else { - push @{ $self->{suffixlib} }, $lib; + push @{ $self->{libraries} }, $slib; + push @{ $self->{suffixlib} }, $slib; } + return; } sub AddIncludeDir diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm index 17f86c5..0aaf835 100644 --- a/src/tools/msvc/Solution.pm +++ b/src/tools/msvc/Solution.pm @@ -169,7 +169,19 @@ s{PG_VERSION_STR "[^"]+"}{PG_VERSION_STR "PostgreSQL $self->{strver}$extraver, c if ($self->{options}->{asserts}); print $o "#define USE_LDAP 1\n" if ($self->{options}->{ldap}); print $o "#define HAVE_LIBZ 1\n" if ($self->{options}->{zlib}); - print $o "#define USE_OPENSSL 1\n" if ($self->{options}->{openssl}); + if ($self->{options}->{openssl}) + { + print $o "#define USE_OPENSSL 1\n"; + # We need to know if we are using version 1.1.0 or above + my ($major, $minor) = $self->GetOpenSSLVersion(); + if ($major == 1 && $minor == 1) + { + print $o "#define HAVE_BIO_GET_DATA 1\n"; + print $o "#define HAVE_BIO_METH_NEW 1\n"; + print $o "#define HAVE_OPENSSL_INIT_SSL 1\n"; + print $o "#define HAVE_ASN1_STRING_GET0_DATA 1\n"; + } + } print $o "#define ENABLE_NLS 1\n" if ($self->{options}->{nls}); print $o "#define BLCKSZ ", 1024 * $self->{options}->{blocksize}, @@ -529,21 +541,39 @@ sub AddProject if ($self->{options}->{openssl}) { $proj->AddIncludeDir($self->{options}->{openssl} . '\include'); - if (-e "$self->{options}->{openssl}/lib/VC/ssleay32MD.lib") + my ($major, $minor) = $self->GetOpenSSLVersion(); + if ($major == 1 && $minor == 1) { - $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\VC\ssleay32.lib', 1); - $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\VC\libeay32.lib', 1); + # Starting at version 1.1.0 OpenSSL have changed their library names from: + # libeay to libcrypto + # ssleay to libssl + if ($self->{platform} eq 'Win32') + { + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libssl.lib', + $self->{options}->{openssl} . '\lib\VC\libssl32.lib'); + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libcrypto.lib', + $self->{options}->{openssl} . '\lib\VC\libcrypto32.lib'); + } + else + { + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libssl.lib', + $self->{options}->{openssl} . '\lib\VC\libssl64.lib'); + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libcrypto.lib', + $self->{options}->{openssl} . '\lib\VC\libcrypto64.lib'); + } } else { - # We don't expect the config-specific library to be here, - # so don't ask for it in last parameter $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\ssleay32.lib', 0); + $self->{options}->{openssl} . '\lib\ssleay32.lib', + $self->{options}->{openssl} . '\lib\VC\ssleay32.lib'); $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\libeay32.lib', 0); + $self->{options}->{openssl} . '\lib\libeay32.lib', + $self->{options}->{openssl} . '\lib\VC\libeay32.lib'); } } if ($self->{options}->{nls}) @@ -890,4 +920,25 @@ MinimumVisualStudioVersion = $self->{MinimumVisualStudioVersion} |; } +sub GetOpenSSLVersion +{ + my $self = shift; + + # Attempt to get OpenSSL version and location. + # Assume openssl.exe in specified dir. + my $opensslprog = '\bin\openssl.exe version 2>&1'; + my $opensslcmd = '"' . $self->{options}->{openssl} . '"' . $opensslprog; + my $sslout = `$opensslcmd`; + die "Unable to determine OpenSSL: The openssl.exe command was not found." if $?; + + if ($sslout =~ /(\d+)\.(\d+)\.(\d+)/m) + { + return ($1, $2) + } + else + { + die "Unable to determine OpenSSL version: The openssl.exe version could not be determined."; + } +} + 1;
0001_windows_openssl_1.1.0_build_PG11_&_HEAD_v1.patch (application/octet-stream, 4.2K) download diff --git a/src/tools/msvc/Project.pm b/src/tools/msvc/Project.pm index 0d35546..291ca86 100644 --- a/src/tools/msvc/Project.pm +++ b/src/tools/msvc/Project.pm @@ -130,17 +130,19 @@ sub AddReference sub AddLibrary { - my ($self, $lib, $dbgsuffix) = @_; + my ($self, $lib, $slib) = @_; + my $libcfg = "MD"; + my $xlib = defined $slib ? $slib : ''; - if ($lib =~ m/\s/) + $xlib =~ s/\.lib$/$libcfg.lib/; + if ($xlib eq '' || ! -e $xlib) { - $lib = '&quot;' . $lib . "&quot;"; + push @{ $self->{libraries} }, $lib; } - - push @{ $self->{libraries} }, $lib; - if ($dbgsuffix) + else { - push @{ $self->{suffixlib} }, $lib; + push @{ $self->{libraries} }, $slib; + push @{ $self->{suffixlib} }, $slib; } return; } diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm index 78db247..6560ba0 100644 --- a/src/tools/msvc/Solution.pm +++ b/src/tools/msvc/Solution.pm @@ -174,7 +174,19 @@ sub GenerateFiles if ($self->{options}->{asserts}); print $o "#define USE_LDAP 1\n" if ($self->{options}->{ldap}); print $o "#define HAVE_LIBZ 1\n" if ($self->{options}->{zlib}); - print $o "#define USE_OPENSSL 1\n" if ($self->{options}->{openssl}); + if ($self->{options}->{openssl}) + { + print $o "#define USE_OPENSSL 1\n"; + # We need to know if we are using version 1.1.0 or above + my ($major, $minor) = $self->GetOpenSSLVersion(); + if ($major == 1 && $minor == 1) + { + print $o "#define HAVE_BIO_GET_DATA 1\n"; + print $o "#define HAVE_BIO_METH_NEW 1\n"; + print $o "#define HAVE_OPENSSL_INIT_SSL 1\n"; + print $o "#define HAVE_ASN1_STRING_GET0_DATA 1\n"; + } + } print $o "#define ENABLE_NLS 1\n" if ($self->{options}->{nls}); print $o "#define BLCKSZ ", 1024 * $self->{options}->{blocksize}, @@ -579,21 +591,39 @@ sub AddProject if ($self->{options}->{openssl}) { $proj->AddIncludeDir($self->{options}->{openssl} . '\include'); - if (-e "$self->{options}->{openssl}/lib/VC/ssleay32MD.lib") + my ($major, $minor) = $self->GetOpenSSLVersion(); + if ($major == 1 && $minor == 1) { - $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\VC\ssleay32.lib', 1); - $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\VC\libeay32.lib', 1); + # Starting at version 1.1.0 OpenSSL have changed their library names from: + # libeay to libcrypto + # ssleay to libssl + if ($self->{platform} eq 'Win32') + { + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libssl.lib', + $self->{options}->{openssl} . '\lib\VC\libssl32.lib'); + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libcrypto.lib', + $self->{options}->{openssl} . '\lib\VC\libcrypto32.lib'); + } + else + { + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libssl.lib', + $self->{options}->{openssl} . '\lib\VC\libssl64.lib'); + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libcrypto.lib', + $self->{options}->{openssl} . '\lib\VC\libcrypto64.lib'); + } } else { - # We don't expect the config-specific library to be here, - # so don't ask for it in last parameter $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\ssleay32.lib', 0); + $self->{options}->{openssl} . '\lib\ssleay32.lib', + $self->{options}->{openssl} . '\lib\VC\ssleay32.lib'); $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\libeay32.lib', 0); + $self->{options}->{openssl} . '\lib\libeay32.lib', + $self->{options}->{openssl} . '\lib\VC\libeay32.lib'); } } if ($self->{options}->{nls}) @@ -955,4 +985,25 @@ MinimumVisualStudioVersion = $self->{MinimumVisualStudioVersion} |; } +sub GetOpenSSLVersion +{ + my $self = shift; + + # Attempt to get OpenSSL version and location. + # Assume openssl.exe in specified dir. + my $opensslprog = '\bin\openssl.exe version 2>&1'; + my $opensslcmd = '"' . $self->{options}->{openssl} . '"' . $opensslprog; + my $sslout = `$opensslcmd`; + die "Unable to determine OpenSSL: The openssl.exe command was not found." if $?; + + if ($sslout =~ /(\d+)\.(\d+)\.(\d+)/m) + { + return ($1, $2) + } + else + { + die "Unable to determine OpenSSL version: The openssl.exe version could not be determined."; + } +} + 1;
0001_windows_openssl_1.1.0_build_PG94_v1.patch (application/octet-stream, 4.2K) download diff --git a/src/tools/msvc/Project.pm b/src/tools/msvc/Project.pm index 9ca5b1f..7360bcb 100644 --- a/src/tools/msvc/Project.pm +++ b/src/tools/msvc/Project.pm @@ -124,18 +124,21 @@ sub AddReference sub AddLibrary { - my ($self, $lib, $dbgsuffix) = @_; + my ($self, $lib, $slib) = @_; + my $libcfg = "MD"; + my $xlib = defined $slib ? $slib : ''; - if ($lib =~ m/\s/) + $xlib =~ s/\.lib$/$libcfg.lib/; + if ($xlib eq '' || ! -e $xlib) { - $lib = '&quot;' . $lib . "&quot;"; + push @{ $self->{libraries} }, $lib; } - - push @{ $self->{libraries} }, $lib; - if ($dbgsuffix) + else { - push @{ $self->{suffixlib} }, $lib; + push @{ $self->{libraries} }, $slib; + push @{ $self->{suffixlib} }, $slib; } + return; } sub AddIncludeDir diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm index d1a791c..cb92438 100644 --- a/src/tools/msvc/Solution.pm +++ b/src/tools/msvc/Solution.pm @@ -181,7 +181,18 @@ s{PG_VERSION_STR "[^"]+"}{__STRINGIFY(x) #x\n#define __STRINGIFY2(z) __STRINGIFY if ($self->{options}->{integer_datetimes}); print O "#define USE_LDAP 1\n" if ($self->{options}->{ldap}); print O "#define HAVE_LIBZ 1\n" if ($self->{options}->{zlib}); - print O "#define USE_SSL 1\n" if ($self->{options}->{openssl}); + if ($self->{options}->{openssl}) + { + print O "#define USE_SSL 1\n"; + # We need to know if we are using version 1.1.0 or above + my ($major, $minor) = $self->GetOpenSSLVersion(); + if ($major == 1 && $minor == 1) + { + print O "#define HAVE_BIO_METH_NEW 1\n"; + print O "#define HAVE_OPENSSL_INIT_SSL 1\n"; + print O "#define HAVE_RAND_OPENSSL 1\n"; + } + } print O "#define ENABLE_NLS 1\n" if ($self->{options}->{nls}); print O "#define BLCKSZ ", 1024 * $self->{options}->{blocksize}, "\n"; @@ -502,21 +513,39 @@ sub AddProject if ($self->{options}->{openssl}) { $proj->AddIncludeDir($self->{options}->{openssl} . '\include'); - if (-e "$self->{options}->{openssl}/lib/VC/ssleay32MD.lib") + my ($major, $minor) = $self->GetOpenSSLVersion(); + if ($major == 1 && $minor == 1) { - $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\VC\ssleay32.lib', 1); - $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\VC\libeay32.lib', 1); + # Starting at version 1.1.0 OpenSSL have changed their library names from: + # libeay to libcrypto + # ssleay to libssl + if ($self->{platform} eq 'Win32') + { + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libssl.lib', + $self->{options}->{openssl} . '\lib\VC\libssl32.lib'); + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libcrypto.lib', + $self->{options}->{openssl} . '\lib\VC\libcrypto32.lib'); + } + else + { + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libssl.lib', + $self->{options}->{openssl} . '\lib\VC\libssl64.lib'); + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libcrypto.lib', + $self->{options}->{openssl} . '\lib\VC\libcrypto64.lib'); + } } else { - # We don't expect the config-specific library to be here, - # so don't ask for it in last parameter $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\ssleay32.lib', 0); + $self->{options}->{openssl} . '\lib\ssleay32.lib', + $self->{options}->{openssl} . '\lib\VC\ssleay32.lib'); $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\libeay32.lib', 0); + $self->{options}->{openssl} . '\lib\libeay32.lib', + $self->{options}->{openssl} . '\lib\VC\libeay32.lib'); } } if ($self->{options}->{nls}) @@ -842,4 +871,25 @@ MinimumVisualStudioVersion = $self->{MinimumVisualStudioVersion} |; } +sub GetOpenSSLVersion +{ + my $self = shift; + + # Attempt to get OpenSSL version and location. + # Assume openssl.exe in specified dir. + my $opensslprog = '\bin\openssl.exe version 2>&1'; + my $opensslcmd = '"' . $self->{options}->{openssl} . '"' . $opensslprog; + my $sslout = `$opensslcmd`; + die "Unable to determine OpenSSL: The openssl.exe command was not found." if $?; + + if ($sslout =~ /(\d+)\.(\d+)\.(\d+)/m) + { + return ($1, $2) + } + else + { + die "Unable to determine OpenSSL version: The openssl.exe version could not be determined."; + } +} + 1;
0001_windows_openssl_1.1.0_build_PG95_&_PG96_v1.patch (application/octet-stream, 4.3K) download diff --git a/src/tools/msvc/Project.pm b/src/tools/msvc/Project.pm index 4ce0941..00e32eb 100644 --- a/src/tools/msvc/Project.pm +++ b/src/tools/msvc/Project.pm @@ -124,18 +124,21 @@ sub AddReference sub AddLibrary { - my ($self, $lib, $dbgsuffix) = @_; + my ($self, $lib, $slib) = @_; + my $libcfg = "MD"; + my $xlib = defined $slib ? $slib : ''; - if ($lib =~ m/\s/) + $xlib =~ s/\.lib$/$libcfg.lib/; + if ($xlib eq '' || ! -e $xlib) { - $lib = '&quot;' . $lib . "&quot;"; + push @{ $self->{libraries} }, $lib; } - - push @{ $self->{libraries} }, $lib; - if ($dbgsuffix) + else { - push @{ $self->{suffixlib} }, $lib; + push @{ $self->{libraries} }, $slib; + push @{ $self->{suffixlib} }, $slib; } + return; } sub AddIncludeDir diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm index 7e79920..a9ed1a0 100644 --- a/src/tools/msvc/Solution.pm +++ b/src/tools/msvc/Solution.pm @@ -177,7 +177,20 @@ s{PG_VERSION_STR "[^"]+"}{__STRINGIFY(x) #x\n#define __STRINGIFY2(z) __STRINGIFY if ($self->{options}->{integer_datetimes}); print O "#define USE_LDAP 1\n" if ($self->{options}->{ldap}); print O "#define HAVE_LIBZ 1\n" if ($self->{options}->{zlib}); - print O "#define USE_OPENSSL 1\n" if ($self->{options}->{openssl}); + if ($self->{options}->{openssl}) + { + print O "#define USE_OPENSSL 1\n"; + # We need to know if we are using version 1.1.0 or above + my ($major, $minor) = $self->GetOpenSSLVersion(); + if ($major == 1 && $minor == 1) + { + print O "#define HAVE_BIO_GET_DATA 1\n"; + print O "#define HAVE_BIO_METH_NEW 1\n"; + print O "#define HAVE_OPENSSL_INIT_SSL 1\n"; + print O "#define HAVE_ASN1_STRING_GET0_DATA 1\n"; + print O "#define HAVE_RAND_OPENSSL 1\n"; + } + } print O "#define ENABLE_NLS 1\n" if ($self->{options}->{nls}); print O "#define BLCKSZ ", 1024 * $self->{options}->{blocksize}, "\n"; @@ -498,21 +511,39 @@ sub AddProject if ($self->{options}->{openssl}) { $proj->AddIncludeDir($self->{options}->{openssl} . '\include'); - if (-e "$self->{options}->{openssl}/lib/VC/ssleay32MD.lib") + my ($major, $minor) = $self->GetOpenSSLVersion(); + if ($major == 1 && $minor == 1) { - $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\VC\ssleay32.lib', 1); - $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\VC\libeay32.lib', 1); + # Starting at version 1.1.0 OpenSSL have changed their library names from: + # libeay to libcrypto + # ssleay to libssl + if ($self->{platform} eq 'Win32') + { + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libssl.lib', + $self->{options}->{openssl} . '\lib\VC\libssl32.lib'); + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libcrypto.lib', + $self->{options}->{openssl} . '\lib\VC\libcrypto32.lib'); + } + else + { + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libssl.lib', + $self->{options}->{openssl} . '\lib\VC\libssl64.lib'); + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libcrypto.lib', + $self->{options}->{openssl} . '\lib\VC\libcrypto64.lib'); + } } else { - # We don't expect the config-specific library to be here, - # so don't ask for it in last parameter $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\ssleay32.lib', 0); + $self->{options}->{openssl} . '\lib\ssleay32.lib', + $self->{options}->{openssl} . '\lib\VC\ssleay32.lib'); $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\libeay32.lib', 0); + $self->{options}->{openssl} . '\lib\libeay32.lib', + $self->{options}->{openssl} . '\lib\VC\libeay32.lib'); } } if ($self->{options}->{nls}) @@ -839,4 +870,25 @@ MinimumVisualStudioVersion = $self->{MinimumVisualStudioVersion} |; } +sub GetOpenSSLVersion +{ + my $self = shift; + + # Attempt to get OpenSSL version and location. + # Assume openssl.exe in specified dir. + my $opensslprog = '\bin\openssl.exe version 2>&1'; + my $opensslcmd = '"' . $self->{options}->{openssl} . '"' . $opensslprog; + my $sslout = `$opensslcmd`; + die "Unable to determine OpenSSL: The openssl.exe command was not found." if $?; + + if ($sslout =~ /(\d+)\.(\d+)\.(\d+)/m) + { + return ($1, $2) + } + else + { + die "Unable to determine OpenSSL version: The openssl.exe version could not be determined."; + } +} + 1;
^ permalink raw reply [nested|flat] 18+ messages in thread

* Re: BUG #15789: libpq compilation with OpenSSL 1.1.1b fails on Windows with Visual Studio 2017
@ 2019-06-20 02:49  Michael Paquier <michael@paquier.xyz>
  parent: Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>
  0 siblings, 1 reply; 18+ messages in thread

From: Michael Paquier @ 2019-06-20 02:49 UTC (permalink / raw)
  To: Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>; +Cc: Sandeep Thakkar <sandeep.thakkar@enterprisedb.com>; serpashk@gmail.com; pgsql-bugs@lists.postgresql.org

On Wed, Jun 05, 2019 at 09:29:21AM +0200, Juan José Santamaría Flecha wrote:
> The patches are intended to support OpenSSL 1.1.x, including 1.1.1x.
> 
> The attached patches are meant for all supported versions and HEAD.

I have been looking at this patch, and here are some comments.  First,
I have looked at the differences in what gets installed between 1.0.2
and 1.1.0 and things are rather messy:
- On the Win64 and Win32 installers for 1.0.2, we have both
libssl32.lib and libeay.lib, with the same naming of libraries in
lib/VC/.  This keeps the Postgres scripts more simple, but it causes
conflicts with the installers when trying to manage both Win64 and
Win32, so I cannot blame them for the changes done.. 
- In 1.1.0~, the situation gets fancy:
-- For Win64 and Win32, we have both now libssl.lib and
libcrypto.lib, which is still consistent.
-- lib/VC/ has been heavily reworked so as we have now for example
libssl64MD.lib & co for Win64 and libssl32MD.lib & co for Win32.

Your patch catches the differences nicely.

sub AddLibrary
{
-       my ($self, $lib, $dbgsuffix) = @_;
+       my ($self, $lib, $slib) = @_;
The only reason why we have the debugging option in the original
interface is for OpenSSL, where Solution.pm checks for the presence of
lib/VC/ssleay32MD.lib before deciding if we should use the debug libs
or not.  I am confused by what you are doing though: what do $slib and
$xlib mean?  For simplicity and compatibility, I am not sure that we
should change the current interface, and just check for the debug libs
that we expect, as done previously.  If the new interface is more
advantageous, the patch needs an effort of documentation, but I would
keep that as a separate improvement.

+sub GetOpenSSLVersion
[...]
+       if ($sslout =~ /(\d+)\.(\d+)\.(\d+)/m)
+       {
+               return ($1, $2)
I really like this interface, but I think that we should return the
three digits instead to help with the decision making so as we don't
need to rework it later on.

+       my ($major, $minor) = $self->GetOpenSSLVersion();
+       if ($major == 1 && $minor == 1)
If one day OpenSSL bumps to 1.2.X, this code would fail. I think that
we should check that the major and minor digits are at least what we
expect them to be.  The same applies to the third digit.
--
Michael

Attachments:

  signature.asc (application/pgp-signature, 832B) download

^ permalink  raw  reply  [nested|flat] 18+ messages in thread

* Re: BUG #15789: libpq compilation with OpenSSL 1.1.1b fails on Windows with Visual Studio 2017
@ 2019-06-21 10:23  Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>
  parent: Michael Paquier <michael@paquier.xyz>
  0 siblings, 1 reply; 18+ messages in thread

From: Juan José Santamaría Flecha @ 2019-06-21 10:23 UTC (permalink / raw)
  To: Michael Paquier <michael@paquier.xyz>; +Cc: Sandeep Thakkar <sandeep.thakkar@enterprisedb.com>; serpashk@gmail.com; pgsql-bugs@lists.postgresql.org

On Thu, Jun 20, 2019 at 4:50 AM Michael Paquier <michael@paquier.xyz> wrote:
>
> - In 1.1.0~, the situation gets fancy:
> -- For Win64 and Win32, we have both now libssl.lib and
> libcrypto.lib, which is still consistent.
> -- lib/VC/ has been heavily reworked so as we have now for example
> libssl64MD.lib & co for Win64 and libssl32MD.lib & co for Win32.
>
> Your patch catches the differences nicely.
>

Great.

> sub AddLibrary
> {
> -       my ($self, $lib, $dbgsuffix) = @_;
> +       my ($self, $lib, $slib) = @_;
> The only reason why we have the debugging option in the original
> interface is for OpenSSL, where Solution.pm checks for the presence of
> lib/VC/ssleay32MD.lib before deciding if we should use the debug libs
> or not.  I am confused by what you are doing though: what do $slib and
> $xlib mean?  For simplicity and compatibility, I am not sure that we
> should change the current interface, and just check for the debug libs
> that we expect, as done previously.  If the new interface is more
> advantageous, the patch needs an effort of documentation, but I would
> keep that as a separate improvement.
>

Since the third parameter is only currently used with OpenSSL
libraries, I thought it would be ok to touch it in this patch. The
reasson behind doing so was to keep the logic for the new fancy
library naming tidy. Documenting better is not a thing I can argue
against, xlib stands for expanded library name as used in
GetAdditionalLinkerDependencies and maybe a comment like this can make
things clearer:

# The parameter $slib is for sufixed run-time library.
# When available it will be added, if not it silently defaults to $lib.
 sub AddLibrary

> +sub GetOpenSSLVersion
> [...]
> If one day OpenSSL bumps to 1.2.X, this code would fail. I think that
> we should check that the major and minor digits are at least what we
> expect them to be.  The same applies to the third digit.

No problem with that. Even more so, it can return 4 values: the 3
digits and the letter. For example, 1.1.1b would be (1, 1,  1, b).

Regards,

Juan José Santamaría Flecha





^ permalink  raw  reply  [nested|flat] 18+ messages in thread

* Re: BUG #15789: libpq compilation with OpenSSL 1.1.1b fails on Windows with Visual Studio 2017
@ 2019-06-22 02:44  Michael Paquier <michael@paquier.xyz>
  parent: Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>
  0 siblings, 1 reply; 18+ messages in thread

From: Michael Paquier @ 2019-06-22 02:44 UTC (permalink / raw)
  To: Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>; +Cc: Sandeep Thakkar <sandeep.thakkar@enterprisedb.com>; serpashk@gmail.com; pgsql-bugs@lists.postgresql.org

On Fri, Jun 21, 2019 at 12:23:21PM +0200, Juan José Santamaría Flecha wrote:
> Since the third parameter is only currently used with OpenSSL
> libraries, I thought it would be ok to touch it in this patch. The
> reasson behind doing so was to keep the logic for the new fancy
> library naming tidy. Documenting better is not a thing I can argue
> against, xlib stands for expanded library name as used in
> GetAdditionalLinkerDependencies and maybe a comment like this can make
> things clearer:
> 
> # The parameter $slib is for sufixed run-time library.
> # When available it will be added, if not it silently defaults to $lib.
>  sub AddLibrary

That's a sensible argument.  Perhaps your way of doing is much better
at the end.  I still found the patch hard to follow in the changes it
did though, so documenting things for the changes you are doing would
help anybody looking at this code in the future. 

> No problem with that. Even more so, it can return 4 values: the 3
> digits and the letter. For example, 1.1.1b would be (1, 1, 1, b).

There are two reasons why we need to know the version of OpenSSL
dynamically:
1) Enforce the correct compilation flags in pg_config.h
2) Select the correct library paths.
OpenSSL keeps API compatibility in their stable branches, so 1. would
not be an issue.  I suspect as well that the installers we use for
Postgres compilation are not going to change their layer in a minor
version update.  So having only 3 digits is enough.  2 likely not.  I
would just keep it at three for now.

Could you send an updated patch?  I would like to look at this stuff
rather sooner than later as OpenSSL 1.0.2 goes EOL in a couple of
months, so there is going to be demand for it.
--
Michael

Attachments:

  signature.asc (application/pgp-signature, 832B) download

^ permalink  raw  reply  [nested|flat] 18+ messages in thread

* Re: BUG #15789: libpq compilation with OpenSSL 1.1.1b fails on Windows with Visual Studio 2017
@ 2019-06-24 11:44  Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>
  parent: Michael Paquier <michael@paquier.xyz>
  0 siblings, 1 reply; 18+ messages in thread

From: Juan José Santamaría Flecha @ 2019-06-24 11:44 UTC (permalink / raw)
  To: Michael Paquier <michael@paquier.xyz>; +Cc: Sandeep Thakkar <sandeep.thakkar@enterprisedb.com>; serpashk@gmail.com; pgsql-bugs@lists.postgresql.org

On Sat, Jun 22, 2019 at 4:44 AM Michael Paquier <michael@paquier.xyz> wrote:
>
> Could you send an updated patch?  I would like to look at this stuff
> rather sooner than later as OpenSSL 1.0.2 goes EOL in a couple of
> months, so there is going to be demand for it.

The updated patches include:

1. Comments for AddLibrary.
2. The function GetOpenSSLVersion returns: major, minor and fix numbers.
3. The version check will default to 1.1.X behaviour for 1.Y.X
versions if Y > 0.

Regards,

Juan José Santamaría Flecha

Attachments: download all (4 files, .tar.gz)

0001_windows_openssl_1.1.0_build_PG11_&_HEAD_v2.patch (application/octet-stream, 4.3K) download diff --git a/src/tools/msvc/Project.pm b/src/tools/msvc/Project.pm index 0d35546..be002ed 100644 --- a/src/tools/msvc/Project.pm +++ b/src/tools/msvc/Project.pm @@ -128,19 +128,23 @@ sub AddReference return; } +# The parameter $slib is for sufixed run-time library. +# When available it will be added, if not it silently defaults to $lib. sub AddLibrary { - my ($self, $lib, $dbgsuffix) = @_; + my ($self, $lib, $slib) = @_; + my $libcfg = "MD"; + my $xlib = defined $slib ? $slib : ''; - if ($lib =~ m/\s/) + $xlib =~ s/\.lib$/$libcfg.lib/; + if ($xlib eq '' || ! -e $xlib) { - $lib = '&quot;' . $lib . "&quot;"; + push @{ $self->{libraries} }, $lib; } - - push @{ $self->{libraries} }, $lib; - if ($dbgsuffix) + else { - push @{ $self->{suffixlib} }, $lib; + push @{ $self->{libraries} }, $slib; + push @{ $self->{suffixlib} }, $slib; } return; } diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm index 78db247..d647429 100644 --- a/src/tools/msvc/Solution.pm +++ b/src/tools/msvc/Solution.pm @@ -174,7 +174,19 @@ sub GenerateFiles if ($self->{options}->{asserts}); print $o "#define USE_LDAP 1\n" if ($self->{options}->{ldap}); print $o "#define HAVE_LIBZ 1\n" if ($self->{options}->{zlib}); - print $o "#define USE_OPENSSL 1\n" if ($self->{options}->{openssl}); + if ($self->{options}->{openssl}) + { + print $o "#define USE_OPENSSL 1\n"; + # We need to know if we are using version 1.1.0 or above + my ($major, $minor, $fix) = $self->GetOpenSSLVersion(); + if ($major == 1 && $minor > 0) + { + print $o "#define HAVE_BIO_GET_DATA 1\n"; + print $o "#define HAVE_BIO_METH_NEW 1\n"; + print $o "#define HAVE_OPENSSL_INIT_SSL 1\n"; + print $o "#define HAVE_ASN1_STRING_GET0_DATA 1\n"; + } + } print $o "#define ENABLE_NLS 1\n" if ($self->{options}->{nls}); print $o "#define BLCKSZ ", 1024 * $self->{options}->{blocksize}, @@ -579,21 +591,39 @@ sub AddProject if ($self->{options}->{openssl}) { $proj->AddIncludeDir($self->{options}->{openssl} . '\include'); - if (-e "$self->{options}->{openssl}/lib/VC/ssleay32MD.lib") + my ($major, $minor, $fix) = $self->GetOpenSSLVersion(); + if ($major == 1 && $minor > 0) { - $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\VC\ssleay32.lib', 1); - $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\VC\libeay32.lib', 1); + # Starting at version 1.1.0 OpenSSL have changed their library names from: + # libeay to libcrypto + # ssleay to libssl + if ($self->{platform} eq 'Win32') + { + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libssl.lib', + $self->{options}->{openssl} . '\lib\VC\libssl32.lib'); + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libcrypto.lib', + $self->{options}->{openssl} . '\lib\VC\libcrypto32.lib'); + } + else + { + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libssl.lib', + $self->{options}->{openssl} . '\lib\VC\libssl64.lib'); + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libcrypto.lib', + $self->{options}->{openssl} . '\lib\VC\libcrypto64.lib'); + } } else { - # We don't expect the config-specific library to be here, - # so don't ask for it in last parameter $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\ssleay32.lib', 0); + $self->{options}->{openssl} . '\lib\ssleay32.lib', + $self->{options}->{openssl} . '\lib\VC\ssleay32.lib'); $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\libeay32.lib', 0); + $self->{options}->{openssl} . '\lib\libeay32.lib', + $self->{options}->{openssl} . '\lib\VC\libeay32.lib'); } } if ($self->{options}->{nls}) @@ -955,4 +985,25 @@ MinimumVisualStudioVersion = $self->{MinimumVisualStudioVersion} |; } +sub GetOpenSSLVersion +{ + my $self = shift; + + # Attempt to get OpenSSL version and location. + # Assume openssl.exe in specified dir. + my $opensslprog = '\bin\openssl.exe version 2>&1'; + my $opensslcmd = '"' . $self->{options}->{openssl} . '"' . $opensslprog; + my $sslout = `$opensslcmd`; + die "Unable to determine OpenSSL: The openssl.exe command was not found." if $?; + + if ($sslout =~ /(\d+)\.(\d+)\.(\d+)(\D)/m) + { + return ($1, $2, $3) + } + else + { + die "Unable to determine OpenSSL version: The openssl.exe version could not be determined."; + } +} + 1;
0001_windows_openssl_1.1.0_build_PG10_v2.patch (application/octet-stream, 4.4K) download diff --git a/src/tools/msvc/Project.pm b/src/tools/msvc/Project.pm index 9817b94..2794692 100644 --- a/src/tools/msvc/Project.pm +++ b/src/tools/msvc/Project.pm @@ -122,20 +122,25 @@ sub AddReference } } +# The parameter $slib is for sufixed run-time library. +# When available it will be added, if not it silently defaults to $lib. sub AddLibrary { - my ($self, $lib, $dbgsuffix) = @_; + my ($self, $lib, $slib) = @_; + my $libcfg = "MD"; + my $xlib = defined $slib ? $slib : ''; - if ($lib =~ m/\s/) + $xlib =~ s/\.lib$/$libcfg.lib/; + if ($xlib eq '' || ! -e $xlib) { - $lib = '&quot;' . $lib . "&quot;"; + push @{ $self->{libraries} }, $lib; } - - push @{ $self->{libraries} }, $lib; - if ($dbgsuffix) + else { - push @{ $self->{suffixlib} }, $lib; + push @{ $self->{libraries} }, $slib; + push @{ $self->{suffixlib} }, $slib; } + return; } sub AddIncludeDir diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm index 17f86c5..6db7a4f 100644 --- a/src/tools/msvc/Solution.pm +++ b/src/tools/msvc/Solution.pm @@ -169,7 +169,19 @@ s{PG_VERSION_STR "[^"]+"}{PG_VERSION_STR "PostgreSQL $self->{strver}$extraver, c if ($self->{options}->{asserts}); print $o "#define USE_LDAP 1\n" if ($self->{options}->{ldap}); print $o "#define HAVE_LIBZ 1\n" if ($self->{options}->{zlib}); - print $o "#define USE_OPENSSL 1\n" if ($self->{options}->{openssl}); + if ($self->{options}->{openssl}) + { + print $o "#define USE_OPENSSL 1\n"; + # We need to know if we are using version 1.1.0 or above + my ($major, $minor, $fix) = $self->GetOpenSSLVersion(); + if ($major == 1 && $minor > 0) + { + print $o "#define HAVE_BIO_GET_DATA 1\n"; + print $o "#define HAVE_BIO_METH_NEW 1\n"; + print $o "#define HAVE_OPENSSL_INIT_SSL 1\n"; + print $o "#define HAVE_ASN1_STRING_GET0_DATA 1\n"; + } + } print $o "#define ENABLE_NLS 1\n" if ($self->{options}->{nls}); print $o "#define BLCKSZ ", 1024 * $self->{options}->{blocksize}, @@ -529,21 +541,39 @@ sub AddProject if ($self->{options}->{openssl}) { $proj->AddIncludeDir($self->{options}->{openssl} . '\include'); - if (-e "$self->{options}->{openssl}/lib/VC/ssleay32MD.lib") + my ($major, $minor, $fix) = $self->GetOpenSSLVersion(); + if ($major == 1 && $minor > 0) { - $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\VC\ssleay32.lib', 1); - $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\VC\libeay32.lib', 1); + # Starting at version 1.1.0 OpenSSL have changed their library names from: + # libeay to libcrypto + # ssleay to libssl + if ($self->{platform} eq 'Win32') + { + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libssl.lib', + $self->{options}->{openssl} . '\lib\VC\libssl32.lib'); + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libcrypto.lib', + $self->{options}->{openssl} . '\lib\VC\libcrypto32.lib'); + } + else + { + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libssl.lib', + $self->{options}->{openssl} . '\lib\VC\libssl64.lib'); + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libcrypto.lib', + $self->{options}->{openssl} . '\lib\VC\libcrypto64.lib'); + } } else { - # We don't expect the config-specific library to be here, - # so don't ask for it in last parameter $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\ssleay32.lib', 0); + $self->{options}->{openssl} . '\lib\ssleay32.lib', + $self->{options}->{openssl} . '\lib\VC\ssleay32.lib'); $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\libeay32.lib', 0); + $self->{options}->{openssl} . '\lib\libeay32.lib', + $self->{options}->{openssl} . '\lib\VC\libeay32.lib'); } } if ($self->{options}->{nls}) @@ -890,4 +920,25 @@ MinimumVisualStudioVersion = $self->{MinimumVisualStudioVersion} |; } +sub GetOpenSSLVersion +{ + my $self = shift; + + # Attempt to get OpenSSL version and location. + # Assume openssl.exe in specified dir. + my $opensslprog = '\bin\openssl.exe version 2>&1'; + my $opensslcmd = '"' . $self->{options}->{openssl} . '"' . $opensslprog; + my $sslout = `$opensslcmd`; + die "Unable to determine OpenSSL: The openssl.exe command was not found." if $?; + + if ($sslout =~ /(\d+)\.(\d+)\.(\d+)(\D)/m) + { + return ($1, $2, $3) + } + else + { + die "Unable to determine OpenSSL version: The openssl.exe version could not be determined."; + } +} + 1;
0001_windows_openssl_1.1.0_build_PG94_v2.patch (application/octet-stream, 4.3K) download diff --git a/src/tools/msvc/Project.pm b/src/tools/msvc/Project.pm index 9ca5b1f..8b7759e 100644 --- a/src/tools/msvc/Project.pm +++ b/src/tools/msvc/Project.pm @@ -122,20 +122,25 @@ sub AddReference } } +# The parameter $slib is for sufixed run-time library. +# When available it will be added, if not it silently defaults to $lib. sub AddLibrary { - my ($self, $lib, $dbgsuffix) = @_; + my ($self, $lib, $slib) = @_; + my $libcfg = "MD"; + my $xlib = defined $slib ? $slib : ''; - if ($lib =~ m/\s/) + $xlib =~ s/\.lib$/$libcfg.lib/; + if ($xlib eq '' || ! -e $xlib) { - $lib = '&quot;' . $lib . "&quot;"; + push @{ $self->{libraries} }, $lib; } - - push @{ $self->{libraries} }, $lib; - if ($dbgsuffix) + else { - push @{ $self->{suffixlib} }, $lib; + push @{ $self->{libraries} }, $slib; + push @{ $self->{suffixlib} }, $slib; } + return; } sub AddIncludeDir diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm index d1a791c..b8ddc89 100644 --- a/src/tools/msvc/Solution.pm +++ b/src/tools/msvc/Solution.pm @@ -181,7 +181,18 @@ s{PG_VERSION_STR "[^"]+"}{__STRINGIFY(x) #x\n#define __STRINGIFY2(z) __STRINGIFY if ($self->{options}->{integer_datetimes}); print O "#define USE_LDAP 1\n" if ($self->{options}->{ldap}); print O "#define HAVE_LIBZ 1\n" if ($self->{options}->{zlib}); - print O "#define USE_SSL 1\n" if ($self->{options}->{openssl}); + if ($self->{options}->{openssl}) + { + print O "#define USE_SSL 1\n"; + # We need to know if we are using version 1.1.0 or above + my ($major, $minor, $fix) = $self->GetOpenSSLVersion(); + if ($major == 1 && $minor > 0) + { + print O "#define HAVE_BIO_METH_NEW 1\n"; + print O "#define HAVE_OPENSSL_INIT_SSL 1\n"; + print O "#define HAVE_RAND_OPENSSL 1\n"; + } + } print O "#define ENABLE_NLS 1\n" if ($self->{options}->{nls}); print O "#define BLCKSZ ", 1024 * $self->{options}->{blocksize}, "\n"; @@ -502,21 +513,39 @@ sub AddProject if ($self->{options}->{openssl}) { $proj->AddIncludeDir($self->{options}->{openssl} . '\include'); - if (-e "$self->{options}->{openssl}/lib/VC/ssleay32MD.lib") + my ($major, $minor, $fix) = $self->GetOpenSSLVersion(); + if ($major == 1 && $minor > 0) { - $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\VC\ssleay32.lib', 1); - $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\VC\libeay32.lib', 1); + # Starting at version 1.1.0 OpenSSL have changed their library names from: + # libeay to libcrypto + # ssleay to libssl + if ($self->{platform} eq 'Win32') + { + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libssl.lib', + $self->{options}->{openssl} . '\lib\VC\libssl32.lib'); + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libcrypto.lib', + $self->{options}->{openssl} . '\lib\VC\libcrypto32.lib'); + } + else + { + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libssl.lib', + $self->{options}->{openssl} . '\lib\VC\libssl64.lib'); + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libcrypto.lib', + $self->{options}->{openssl} . '\lib\VC\libcrypto64.lib'); + } } else { - # We don't expect the config-specific library to be here, - # so don't ask for it in last parameter $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\ssleay32.lib', 0); + $self->{options}->{openssl} . '\lib\ssleay32.lib', + $self->{options}->{openssl} . '\lib\VC\ssleay32.lib'); $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\libeay32.lib', 0); + $self->{options}->{openssl} . '\lib\libeay32.lib', + $self->{options}->{openssl} . '\lib\VC\libeay32.lib'); } } if ($self->{options}->{nls}) @@ -842,4 +871,25 @@ MinimumVisualStudioVersion = $self->{MinimumVisualStudioVersion} |; } +sub GetOpenSSLVersion +{ + my $self = shift; + + # Attempt to get OpenSSL version and location. + # Assume openssl.exe in specified dir. + my $opensslprog = '\bin\openssl.exe version 2>&1'; + my $opensslcmd = '"' . $self->{options}->{openssl} . '"' . $opensslprog; + my $sslout = `$opensslcmd`; + die "Unable to determine OpenSSL: The openssl.exe command was not found." if $?; + + if ($sslout =~ /(\d+)\.(\d+)\.(\d+)(\D)/m) + { + return ($1, $2, $3) + } + else + { + die "Unable to determine OpenSSL version: The openssl.exe version could not be determined."; + } +} + 1;
0001_windows_openssl_1.1.0_build_PG95_&_PG96_v2.patch (application/octet-stream, 4.4K) download diff --git a/src/tools/msvc/Project.pm b/src/tools/msvc/Project.pm index 4ce0941..c152fd3 100644 --- a/src/tools/msvc/Project.pm +++ b/src/tools/msvc/Project.pm @@ -122,20 +122,25 @@ sub AddReference } } +# The parameter $slib is for sufixed run-time library. +# When available it will be added, if not it silently defaults to $lib. sub AddLibrary { - my ($self, $lib, $dbgsuffix) = @_; + my ($self, $lib, $slib) = @_; + my $libcfg = "MD"; + my $xlib = defined $slib ? $slib : ''; - if ($lib =~ m/\s/) + $xlib =~ s/\.lib$/$libcfg.lib/; + if ($xlib eq '' || ! -e $xlib) { - $lib = '&quot;' . $lib . "&quot;"; + push @{ $self->{libraries} }, $lib; } - - push @{ $self->{libraries} }, $lib; - if ($dbgsuffix) + else { - push @{ $self->{suffixlib} }, $lib; + push @{ $self->{libraries} }, $slib; + push @{ $self->{suffixlib} }, $slib; } + return; } sub AddIncludeDir diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm index 7e79920..fb7f3f5 100644 --- a/src/tools/msvc/Solution.pm +++ b/src/tools/msvc/Solution.pm @@ -177,7 +177,20 @@ s{PG_VERSION_STR "[^"]+"}{__STRINGIFY(x) #x\n#define __STRINGIFY2(z) __STRINGIFY if ($self->{options}->{integer_datetimes}); print O "#define USE_LDAP 1\n" if ($self->{options}->{ldap}); print O "#define HAVE_LIBZ 1\n" if ($self->{options}->{zlib}); - print O "#define USE_OPENSSL 1\n" if ($self->{options}->{openssl}); + if ($self->{options}->{openssl}) + { + print O "#define USE_OPENSSL 1\n"; + # We need to know if we are using version 1.1.0 or above + my ($major, $minor, $fix) = $self->GetOpenSSLVersion(); + if ($major == 1 && $minor > 0) + { + print O "#define HAVE_BIO_GET_DATA 1\n"; + print O "#define HAVE_BIO_METH_NEW 1\n"; + print O "#define HAVE_OPENSSL_INIT_SSL 1\n"; + print O "#define HAVE_ASN1_STRING_GET0_DATA 1\n"; + print O "#define HAVE_RAND_OPENSSL 1\n"; + } + } print O "#define ENABLE_NLS 1\n" if ($self->{options}->{nls}); print O "#define BLCKSZ ", 1024 * $self->{options}->{blocksize}, "\n"; @@ -498,21 +511,39 @@ sub AddProject if ($self->{options}->{openssl}) { $proj->AddIncludeDir($self->{options}->{openssl} . '\include'); - if (-e "$self->{options}->{openssl}/lib/VC/ssleay32MD.lib") + my ($major, $minor, $fix) = $self->GetOpenSSLVersion(); + if ($major == 1 && $minor > 0) { - $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\VC\ssleay32.lib', 1); - $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\VC\libeay32.lib', 1); + # Starting at version 1.1.0 OpenSSL have changed their library names from: + # libeay to libcrypto + # ssleay to libssl + if ($self->{platform} eq 'Win32') + { + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libssl.lib', + $self->{options}->{openssl} . '\lib\VC\libssl32.lib'); + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libcrypto.lib', + $self->{options}->{openssl} . '\lib\VC\libcrypto32.lib'); + } + else + { + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libssl.lib', + $self->{options}->{openssl} . '\lib\VC\libssl64.lib'); + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libcrypto.lib', + $self->{options}->{openssl} . '\lib\VC\libcrypto64.lib'); + } } else { - # We don't expect the config-specific library to be here, - # so don't ask for it in last parameter $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\ssleay32.lib', 0); + $self->{options}->{openssl} . '\lib\ssleay32.lib', + $self->{options}->{openssl} . '\lib\VC\ssleay32.lib'); $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\libeay32.lib', 0); + $self->{options}->{openssl} . '\lib\libeay32.lib', + $self->{options}->{openssl} . '\lib\VC\libeay32.lib'); } } if ($self->{options}->{nls}) @@ -839,4 +870,25 @@ MinimumVisualStudioVersion = $self->{MinimumVisualStudioVersion} |; } +sub GetOpenSSLVersion +{ + my $self = shift; + + # Attempt to get OpenSSL version and location. + # Assume openssl.exe in specified dir. + my $opensslprog = '\bin\openssl.exe version 2>&1'; + my $opensslcmd = '"' . $self->{options}->{openssl} . '"' . $opensslprog; + my $sslout = `$opensslcmd`; + die "Unable to determine OpenSSL: The openssl.exe command was not found." if $?; + + if ($sslout =~ /(\d+)\.(\d+)\.(\d+)(\D)/m) + { + return ($1, $2, $3) + } + else + { + die "Unable to determine OpenSSL version: The openssl.exe version could not be determined."; + } +} + 1;
^ permalink raw reply [nested|flat] 18+ messages in thread

* Re: BUG #15789: libpq compilation with OpenSSL 1.1.1b fails on Windows with Visual Studio 2017
@ 2019-06-25 08:08  Michael Paquier <michael@paquier.xyz>
  parent: Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>
  0 siblings, 2 replies; 18+ messages in thread

From: Michael Paquier @ 2019-06-25 08:08 UTC (permalink / raw)
  To: Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>; +Cc: Sandeep Thakkar <sandeep.thakkar@enterprisedb.com>; serpashk@gmail.com; pgsql-bugs@lists.postgresql.org

On Mon, Jun 24, 2019 at 01:44:18PM +0200, Juan José Santamaría Flecha wrote:
> The updated patches include:
> 
> 1. Comments for AddLibrary.
> 2. The function GetOpenSSLVersion returns: major, minor and fix numbers.
> 3. The version check will default to 1.1.X behaviour for 1.Y.X
> versions if Y > 0.

Thanks for the new patch set!  I have been looking at that in depths
and I have adjusted the whole logic a bit here and there.  At the end
I found the logic changed in AddLibrary more confusing because the
debugging suffix may change depending on if we use a release-quality
build or not, so I have kept the original interface, and completed it
with a logic allowing the scripts to grab all the libraries it
expects.  This has a small gain when using a non-debug installation as
the library names are the same for Win32 and Win64 for >= 1.1.0.

Also, your patch was not working with other versions of MSVC as the
new routine got stuck into the 2017 class, so I had to move it, and I
found that it was cleaner to just make it return a string made of the
3 first digits and to do direct string comparisons.

Please note that it is not necessary to create versions for the
back-branches yet.  If necessary, I'll do that myself, but first let's
make sure that we agree about the shape of the patch for HEAD.
Attached is an updated version which I would be fine to commit.  I
have tested it with compilation linking to OpenSSL 1.0.2 and 1.1.0 on
Win32 and the build is able to complete.  This applies on HEAD only,
where I have run all my tests.  The patch is properly indented.

What do you think?
--
Michael

Attachments:

msvc-openssl-110-v2.patch (text/x-diff, 4.8K) download diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm index cbe019e524..addada6189 100644 --- a/src/tools/msvc/Solution.pm +++ b/src/tools/msvc/Solution.pm @@ -114,6 +114,34 @@ sub copyFile return; } +# Fetch version of OpenSSL based on a parsing of the command shipped with +# the installer this build is linking to. This returns as result a string +# made of the three first digits of the OpenSSL versoin, which is enough +# to decide which options to apply depending on the version of OpenSSL +# linking with. +sub GetOpenSSLVersion +{ + my $self = shift; + + # Attempt to get OpenSSL version and location. This assumes that + # openssl.exe is in the specified directory. + my $opensslcmd = + $self->{options}->{openssl} . "\\bin\\openssl.exe version 2>&1"; + my $sslout = `$opensslcmd`; + + $? >> 8 == 0 + or croak + "Unable to determine OpenSSL version: The openssl.exe command wasn't found."; + + if ($sslout =~ /(\d+)\.(\d+)\.(\d+)(\D)/m) + { + return "$1.$2.$3"; + } + + croak + "Unable to determine OpenSSL version: The openssl.exe version could not be determined."; +} + sub GenerateFiles { my $self = shift; @@ -168,10 +196,23 @@ sub GenerateFiles print $o "#ifndef IGNORE_CONFIGURED_SETTINGS\n"; print $o "#define USE_ASSERT_CHECKING 1\n" if ($self->{options}->{asserts}); - print $o "#define USE_LDAP 1\n" if ($self->{options}->{ldap}); - print $o "#define HAVE_LIBZ 1\n" if ($self->{options}->{zlib}); - print $o "#define USE_OPENSSL 1\n" if ($self->{options}->{openssl}); - print $o "#define ENABLE_NLS 1\n" if ($self->{options}->{nls}); + print $o "#define USE_LDAP 1\n" if ($self->{options}->{ldap}); + print $o "#define HAVE_LIBZ 1\n" if ($self->{options}->{zlib}); + + if ($self->{options}->{openssl}) + { + print $o "#define USE_OPENSSL 1\n"; + + # More symbols are needed with OpenSSL 1.1.0 and above. + if ($self->GetOpenSSLVersion() ge '1.1.0') + { + print $o "#define HAVE_ASN1_STRING_GET0_DATA 1\n"; + print $o "#define HAVE_BIO_GET_DATA 1\n"; + print $o "#define HAVE_BIO_METH_NEW 1\n"; + print $o "#define HAVE_OPENSSL_INIT_SSL 1\n"; + } + } + print $o "#define ENABLE_NLS 1\n" if ($self->{options}->{nls}); print $o "#define BLCKSZ ", 1024 * $self->{options}->{blocksize}, "\n"; @@ -613,21 +654,68 @@ sub AddProject if ($self->{options}->{openssl}) { $proj->AddIncludeDir($self->{options}->{openssl} . '\include'); - if (-e "$self->{options}->{openssl}/lib/VC/ssleay32MD.lib") + + # Starting at version 1.1.0 the OpenSSL installers have + # changed their library names from: + # - libeay to libcrypto + # - ssleay to libssl + if ($self->GetOpenSSLVersion() ge '1.1.0') { - $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\VC\ssleay32.lib', 1); - $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\VC\libeay32.lib', 1); + my $dbgsuffix; + my $libsslpath; + my $libcryptopath; + + # The format name of the libraries is slightly different + # between the Win32 and Win64 platform, so adapt. + if (-e "$self->{options}->{openssl}/lib/VC/sslcrypto32MD.lib") + { + # Win32 here, with a debugging library set. + $dbgsuffix = 1; + $libsslpath = '\lib\VC\libssl32.lib'; + $libcryptopath = '\lib\VC\libcrypto32.lib'; + } + elsif (-e "$self->{options}->{openssl}/lib/VC/sslcrypto64MD.lib") + { + # Win64 here, with a debugging library set. + $dbgsuffix = 1; + $libsslpath = '\lib\VC\libssl64.lib'; + $libcryptopath = '\lib\VC\libcrypto64.lib'; + } + else + { + # On both Win32 and Win64 the same library + # names are used without a debugging context. + $dbgsuffix = 0; + $libsslpath = '\lib\libssl.lib'; + $libcryptopath = '\lib\libcrypto.lib'; + } + + $proj->AddLibrary($self->{options}->{openssl} . $libsslpath, + $dbgsuffix); + $proj->AddLibrary($self->{options}->{openssl} . $libcryptopath, + $dbgsuffix); } else { - # We don't expect the config-specific library to be here, - # so don't ask for it in last parameter - $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\ssleay32.lib', 0); - $proj->AddLibrary( - $self->{options}->{openssl} . '\lib\libeay32.lib', 0); + # Choose which set of libraries to use depending on if + # debugging libraries are in place in the installer. + if (-e "$self->{options}->{openssl}/lib/VC/ssleay32MD.lib") + { + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\VC\ssleay32.lib', 1); + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\VC\libeay32.lib', 1); + } + else + { + # We don't expect the config-specific library + # to be here, so don't ask for it in last + # parameter. + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\ssleay32.lib', 0); + $proj->AddLibrary( + $self->{options}->{openssl} . '\lib\libeay32.lib', 0); + } } } if ($self->{options}->{nls})
signature.asc (application/pgp-signature, 832B) download ^ permalink raw reply [nested|flat] 18+ messages in thread

* Re: BUG #15789: libpq compilation with OpenSSL 1.1.1b fails on Windows with Visual Studio 2017
@ 2019-06-25 09:43  Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>
  parent: Michael Paquier <michael@paquier.xyz>
  1 sibling, 1 reply; 18+ messages in thread

From: Juan José Santamaría Flecha @ 2019-06-25 09:43 UTC (permalink / raw)
  To: Michael Paquier <michael@paquier.xyz>; +Cc: Sandeep Thakkar <sandeep.thakkar@enterprisedb.com>; serpashk@gmail.com; pgsql-bugs@lists.postgresql.org

I will not have much available time for this list in the next few
weeks, so as quick eye review:

On Tue, Jun 25, 2019 at 10:08 AM Michael Paquier <michael@paquier.xyz> wrote:
>
> Thanks for the new patch set!  I have been looking at that in depths
> and I have adjusted the whole logic a bit here and there.  At the end
> I found the logic changed in AddLibrary more confusing because the
> debugging suffix may change depending on if we use a release-quality
> build or not, so I have kept the original interface, and completed it
> with a logic allowing the scripts to grab all the libraries it
> expects.  This has a small gain when using a non-debug installation as
> the library names are the same for Win32 and Win64 for >= 1.1.0.
>

It actually looks clearer and less intrusive (better) to me too.

> Also, your patch was not working with other versions of MSVC as the
> new routine got stuck into the 2017 class, so I had to move it, and I
> found that it was cleaner to just make it return a string made of the
> 3 first digits and to do direct string comparisons.
>

If you are using a string you will need padding, maybe mimic
OPENSSL_VERSION_NUMBER [1].

> Please note that it is not necessary to create versions for the
> back-branches yet.  If necessary, I'll do that myself, but first let's
> make sure that we agree about the shape of the patch for HEAD.
> Attached is an updated version which I would be fine to commit.  I
> have tested it with compilation linking to OpenSSL 1.0.2 and 1.1.0 on
> Win32 and the build is able to complete.  This applies on HEAD only,
> where I have run all my tests.  The patch is properly indented.
>

There is a typo:

s/versoin/version/

+# made of the three first digits of the OpenSSL versoin, which is enough


Regards,

Juan José Santamaría Flecha

[1] https://www.openssl.org/docs/man1.1.1/man3/OPENSSL_VERSION_NUMBER.html





^ permalink  raw  reply  [nested|flat] 18+ messages in thread

* Re: BUG #15789: libpq compilation with OpenSSL 1.1.1b fails on Windows with Visual Studio 2017
@ 2019-06-25 13:52  Alvaro Herrera <alvherre@2ndquadrant.com>
  parent: Michael Paquier <michael@paquier.xyz>
  1 sibling, 1 reply; 18+ messages in thread

From: Alvaro Herrera @ 2019-06-25 13:52 UTC (permalink / raw)
  To: Michael Paquier <michael@paquier.xyz>; +Cc: Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>; Sandeep Thakkar <sandeep.thakkar@enterprisedb.com>; serpashk@gmail.com; pgsql-bugs@lists.postgresql.org

On 2019-Jun-25, Michael Paquier wrote:

> Attached is an updated version which I would be fine to commit.  I
> have tested it with compilation linking to OpenSSL 1.0.2 and 1.1.0 on
> Win32 and the build is able to complete.  This applies on HEAD only,
> where I have run all my tests.  The patch is properly indented.

In sub GenerateFiles, I would move the stanza for ->{openssl} next to
the one for ->{gss} and keep the simpler lines all together (your patch
leaves ->{nls} as a separate item, which is aesthetically unpleasing).

-- 
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services





^ permalink  raw  reply  [nested|flat] 18+ messages in thread

* Re: BUG #15789: libpq compilation with OpenSSL 1.1.1b fails on Windows with Visual Studio 2017
@ 2019-06-26 01:22  Michael Paquier <michael@paquier.xyz>
  parent: Alvaro Herrera <alvherre@2ndquadrant.com>
  0 siblings, 0 replies; 18+ messages in thread

From: Michael Paquier @ 2019-06-26 01:22 UTC (permalink / raw)
  To: Alvaro Herrera <alvherre@2ndquadrant.com>; +Cc: Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>; Sandeep Thakkar <sandeep.thakkar@enterprisedb.com>; serpashk@gmail.com; pgsql-bugs@lists.postgresql.org

On Tue, Jun 25, 2019 at 09:52:43AM -0400, Alvaro Herrera wrote:
> In sub GenerateFiles, I would move the stanza for ->{openssl} next to
> the one for ->{gss} and keep the simpler lines all together (your patch
> leaves ->{nls} as a separate item, which is aesthetically unpleasing).

Right, thanks.  This makes the flow cleaner.
--
Michael

Attachments:

  signature.asc (application/pgp-signature, 832B) download

^ permalink  raw  reply  [nested|flat] 18+ messages in thread

* Re: BUG #15789: libpq compilation with OpenSSL 1.1.1b fails on Windows with Visual Studio 2017
@ 2019-06-26 01:48  Michael Paquier <michael@paquier.xyz>
  parent: Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>
  0 siblings, 1 reply; 18+ messages in thread

From: Michael Paquier @ 2019-06-26 01:48 UTC (permalink / raw)
  To: Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>; +Cc: Sandeep Thakkar <sandeep.thakkar@enterprisedb.com>; serpashk@gmail.com; pgsql-bugs@lists.postgresql.org

On Tue, Jun 25, 2019 at 11:43:29AM +0200, Juan José Santamaría Flecha wrote:
> If you are using a string you will need padding, maybe mimic
> OPENSSL_VERSION_NUMBER [1].

For now, I have taken the more simple approach to have three separate
fields for the digit numbers, which is more than enough.  So, after
doing more tweaks and tests on it with the installers for 1.0.2, 1.1.0
and 1.1.1, I have committed the patch to master for now.  The result
is rather cool, as we can now adapt with future versions easily.

I am planning to back-patch the thing down to 9.4 where OpenSSL 1.1.0
is supported, but first let's see if the buildfarm has anything to
say.  I don't expect any issues as we basically don't change the logic
build for 1.0.2, but nobody is never careful enough with this stuff.
--
Michael

Attachments:

  signature.asc (application/pgp-signature, 832B) download

^ permalink  raw  reply  [nested|flat] 18+ messages in thread

* Re: BUG #15789: libpq compilation with OpenSSL 1.1.1b fails on Windows with Visual Studio 2017
@ 2019-06-26 14:12  Michael Paquier <michael@paquier.xyz>
  parent: Michael Paquier <michael@paquier.xyz>
  0 siblings, 0 replies; 18+ messages in thread

From: Michael Paquier @ 2019-06-26 14:12 UTC (permalink / raw)
  To: Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>; +Cc: Sandeep Thakkar <sandeep.thakkar@enterprisedb.com>; serpashk@gmail.com; pgsql-bugs@lists.postgresql.org

On Wed, Jun 26, 2019 at 10:48:03AM +0900, Michael Paquier wrote:
> I am planning to back-patch the thing down to 9.4 where OpenSSL 1.1.0
> is supported, but first let's see if the buildfarm has anything to
> say.  I don't expect any issues as we basically don't change the logic
> build for 1.0.2, but nobody is never careful enough with this stuff.

The buildfarm did not complain, so I have backpatched the thing down
to 9.4.  There were some conflicts, mainly:
- On 9.4, we need to use USE_SSL to enable SSL in the builds.
- In ~9.6, we also need to set HAVE_RAND_OPENSSL for RAND_OpenSSL()
which is a function used by pgcrypto available in OpenSSL 1.1.0 and
newer.  This was removed in v10 as of fe0a0b59, so we can clean up a
bit things here.
--
Michael

Attachments:

  signature.asc (application/pgp-signature, 832B) download

^ permalink  raw  reply  [nested|flat] 18+ messages in thread


end of thread, other threads:[~2019-06-26 14:12 UTC | newest]

Thread overview: 18+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2019-05-02 13:57 BUG #15789: libpq compilation with OpenSSL 1.1.1b fails on Windows with Visual Studio 2017 PG Bug reporting form <noreply@postgresql.org>
2019-05-28 05:31 ` Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>
2019-05-28 10:58   ` Michael Paquier <michael@paquier.xyz>
2019-05-31 14:11     ` Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>
2019-05-31 17:17       ` Michael Paquier <michael@paquier.xyz>
2019-06-03 10:00         ` Sandeep Thakkar <sandeep.thakkar@enterprisedb.com>
2019-06-03 23:51           ` Michael Paquier <michael@paquier.xyz>
2019-06-05 07:29             ` Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>
2019-06-20 02:49               ` Michael Paquier <michael@paquier.xyz>
2019-06-21 10:23                 ` Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>
2019-06-22 02:44                   ` Michael Paquier <michael@paquier.xyz>
2019-06-24 11:44                     ` Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>
2019-06-25 08:08                       ` Michael Paquier <michael@paquier.xyz>
2019-06-25 09:43                         ` Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>
2019-06-26 01:48                           ` Michael Paquier <michael@paquier.xyz>
2019-06-26 14:12                             ` Michael Paquier <michael@paquier.xyz>
2019-06-25 13:52                         ` Alvaro Herrera <alvherre@2ndquadrant.com>
2019-06-26 01:22                           ` Michael Paquier <michael@paquier.xyz>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox