pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Noah Misch <noah@leadboat.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Dave Page <dpage@postgresql.org>
Cc: Sandeep Thakkar <sandeep.thakkar@enterprisedb.com>
Cc: Ashutosh Sharma <ashu.coek88@gmail.com>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: pl/perl extension fails on Windows
Date: Sun, 3 Dec 2017 18:29:32 -0800
Message-ID: <20171204022932.GA3243163@rfd.leadboat.com> (raw)
In-Reply-To: <20171130041441.GA3161526@rfd.leadboat.com>
References: <14838.1502379286@sss.pgh.pa.us>
	<1708!2.1502718166@sss.pgh.pa.us>
	<10406.1502743029@sss.pgh.pa.us>
	<CA+OCxow_P_K9kpsSeMwdnqoiaAx3YEDN6P1s-93L5A_iTUYHuw@mail.gmail.com>
	<1914.1502978317@sss.pgh.pa.us>
	<CA+OCxow3A8waWr3drvFtmBwfgD8RF6w7rC2z9eY8eMJ7zz3y4w@mail.gmail.com>
	<4515.1502981857@sss.pgh.pa.us>
	<17646.1502983856@sss.pgh.pa.us>
	<19438.1502986558@sss.pgh.pa.us>
	<20171130041441.GA3161526@rfd.leadboat.com>

On Wed, Nov 29, 2017 at 08:14:41PM -0800, Noah Misch wrote:
> 1. If $Config{gccversion} is nonempty, add _USE_32BIT_TIME_T.  This will do
>    the wrong thing if MinGW changes its default to match modern MSVC.  It will
>    do the wrong thing for a Perl built with "gcc -D__MINGW_USE_VC2005_COMPAT".
> 
> 2. When configuring the build, determine whether to add _USE_32BIT_TIME_T by
>    running a test program built with and without that symbol.  Perhaps have
>    the test program store and retrieve a PL_modglobal value.  (PL_modglobal
>    maps to a PerlInterpreter field that follows the fields sensitive to
>    _USE_32BIT_TIME_T, and perlapi documents it since 5.8.0 or earlier.)  This
>    is more principled than (1), but it will be more code and may have weird
>    interactions with rare Perl build options.
> 
> I am inclined toward (2) if it takes no more than roughly a hundred lines of
> code, else (1).  Opinions?  I regret investing in 32-bit Windows.  If there's
> any OS where a 32-bit PostgreSQL server makes sense today, it's not Windows.

Here's an implementation of (2).  This is more intricate than I hoped.  One
could argue for (1), but I estimate (2) wins by a nose.  I successfully tested
http://strawberryperl.com/download/5.14.4.1/strawberry-perl-5.14.4.1-32bit.msi
(Perl 5.14.4; MinGW-built; must have -D_USE_32BIT_TIME_T) and
http://get.enterprisedb.com/languagepacks/edb-languagepack-10-3-windows.exe
(Perl 5.24.0; MSVC-built; must not have -D_USE_32BIT_TIME_T).  I also tried
http://strawberryperl.com/download/5.8.9/strawberry-perl-5.8.9.5.msi, which
experienced a StackHash_0a9e crash during PERL_SYS_INIT3(), with or without
-D_USE_32BIT_TIME_T.  I expect that breakage is orthogonal.  I didn't have
ready access to obsolete MSVC-built Perl, so it will be interesting to see how
the buildfarm likes this.

Attachments:

time_t-msvc-v1.patch (text/plain, 10.3K) download MSVC: Test whether 32-bit Perl needs -D_USE_32BIT_TIME_T. Commits 5a5c2feca3fd858e70ea348822595547e6fa6c15 and b5178c5d08ca59e30f9d9428fa6fdb2741794e65 introduced support for modern MSVC-built, 32-bit Perl, but they broke use of MinGW-built, 32-bit Perl distributions like Strawberry Perl and modern ActivePerl. Perl has no robust means to report whether it expects a -D_USE_32BIT_TIME_T ABI, so test this. Back-patch to 9.3 (all supported versions). The chief alternative was a heuristic of adding -D_USE_32BIT_TIME_T when $Config{gccversion} is nonempty. That banks on every gcc-built Perl using the same ABI. gcc could change its default ABI the way MSVC once did, and one could build Perl with gcc and the non-default ABI. The GNU make build system could benefit from a similar test, without which it does not support MSVC-built Perl. For now, just add a comment. Most users taking the special step of building Perl with MSVC probably build PostgreSQL with MSVC. Discussion: https://postgr.es/m/20171130041441.GA3161526@rfd.leadboat.com diff --git a/config/perl.m4 b/config/perl.m4 index 76b1a92..caefb07 100644 --- a/config/perl.m4 +++ b/config/perl.m4 @@ -48,19 +48,23 @@ AC_DEFUN([PGAC_CHECK_PERL_CONFIGS], # PGAC_CHECK_PERL_EMBED_CCFLAGS # ----------------------------- -# We selectively extract stuff from $Config{ccflags}. We don't really need -# anything except -D switches, and other sorts of compiler switches can -# actively break things if Perl was compiled with a different compiler. -# Moreover, although Perl likes to put stuff like -D_LARGEFILE_SOURCE and -# -D_FILE_OFFSET_BITS=64 here, it would be fatal to try to compile PL/Perl -# to a different libc ABI than core Postgres uses. The available information -# says that all the symbols that affect Perl's own ABI begin with letters, -# so it should be sufficient to adopt -D switches for symbols not beginning -# with underscore. An exception is that we need to let through -# -D_USE_32BIT_TIME_T if it's present. (We probably could restrict that to -# only get through on Windows, but for the moment we let it through always.) -# For debugging purposes, let's have the configure output report the raw -# ccflags value as well as the set of flags we chose to adopt. +# We selectively extract stuff from $Config{ccflags}. For debugging purposes, +# let's have the configure output report the raw ccflags value as well as the +# set of flags we chose to adopt. We don't really need anything except -D +# switches, and other sorts of compiler switches can actively break things if +# Perl was compiled with a different compiler. Moreover, although Perl likes +# to put stuff like -D_LARGEFILE_SOURCE and -D_FILE_OFFSET_BITS=64 here, it +# would be fatal to try to compile PL/Perl to a different libc ABI than core +# Postgres uses. The available information says that most symbols that affect +# Perl's own ABI begin with letters, so it's almost sufficient to adopt -D +# switches for symbols not beginning with underscore. Some exceptions are the +# Windows-specific -D_USE_32BIT_TIME_T and -D__MINGW_USE_VC2005_COMPAT; see +# Mkvcbuild.pm for details. We absorb the former when Perl reports it. Perl +# never reports the latter, and we don't attempt to deduce when it's needed. +# Consequently, we don't support using MinGW to link to MSVC-built Perl. As +# of 2017, all supported ActivePerl and Strawberry Perl are MinGW-built. If +# that changes or an MSVC-built Perl distribution becomes prominent, we can +# revisit this limitation. AC_DEFUN([PGAC_CHECK_PERL_EMBED_CCFLAGS], [AC_REQUIRE([PGAC_PATH_PERL]) AC_MSG_CHECKING([for CFLAGS recommended by Perl]) diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm index 4c2e12e..93f364a 100644 --- a/src/tools/msvc/Mkvcbuild.pm +++ b/src/tools/msvc/Mkvcbuild.pm @@ -28,6 +28,7 @@ my $libpgcommon; my $libpgfeutils; my $postgres; my $libpq; +my @unlink_on_exit; # Set of variables for modules in contrib/ and src/test/modules/ my $contrib_defines = { 'refint' => 'REFINT_VERBOSE' }; @@ -517,34 +518,154 @@ sub mkvcbuild my $plperl = $solution->AddProject('plperl', 'dll', 'PLs', 'src/pl/plperl'); $plperl->AddIncludeDir($solution->{options}->{perl} . '/lib/CORE'); + $plperl->AddReference($postgres); + + my $perl_path = $solution->{options}->{perl} . '\lib\CORE\*perl*'; + + # ActivePerl 5.16 provided perl516.lib; 5.18 provided libperl518.a + my @perl_libs = + grep { /perl\d+\.lib$|libperl\d+\.a$/ } glob($perl_path); + if (@perl_libs == 1) + { + $plperl->AddLibrary($perl_libs[0]); + } + else + { + die +"could not identify perl library version matching pattern $perl_path\n"; + } # Add defines from Perl's ccflags; see PGAC_CHECK_PERL_EMBED_CCFLAGS my @perl_embed_ccflags; foreach my $f (split(" ", $Config{ccflags})) { - if ( $f =~ /^-D[^_]/ - || $f =~ /^-D_USE_32BIT_TIME_T/) + if ($f =~ /^-D[^_]/) { $f =~ s/\-D//; push(@perl_embed_ccflags, $f); } } - # Perl versions before 5.13.4 don't provide -D_USE_32BIT_TIME_T - # regardless of how they were built. On 32-bit Windows, assume - # such a version was built with a pre-MSVC-2005 compiler, and - # define the symbol anyway, so that we are compatible if we're - # being built with a later MSVC version. - push(@perl_embed_ccflags, '_USE_32BIT_TIME_T') - if $solution->{platform} eq 'Win32' - && $Config{PERL_REVISION} == 5 - && ($Config{PERL_VERSION} < 13 - || ( $Config{PERL_VERSION} == 13 - && $Config{PERL_SUBVERSION} < 4)); - - # Also, a hack to prevent duplicate definitions of uid_t/gid_t + # hack to prevent duplicate definitions of uid_t/gid_t push(@perl_embed_ccflags, 'PLPERL_HAVE_UID_GID'); + # Windows offers several 32-bit ABIs. Perl is sensitive to + # sizeof(time_t), one of the ABI dimensions. To get 32-bit time_t, + # use "cl -D_USE_32BIT_TIME_T" or plain "gcc". For 64-bit time_t, use + # "gcc -D__MINGW_USE_VC2005_COMPAT" or plain "cl". Before MSVC 2005, + # plain "cl" chose 32-bit time_t. PostgreSQL doesn't support building + # with pre-MSVC-2005 compilers, but it does support linking to Perl + # built with such a compiler. MSVC-built Perl 5.13.4 and later report + # -D_USE_32BIT_TIME_T in $Config{ccflags} if applicable, but + # MinGW-built Perl never reports -D_USE_32BIT_TIME_T despite typically + # needing it. Ignore the $Config{ccflags} opinion about + # -D_USE_32BIT_TIME_T, and use a runtime test to deduce the ABI Perl + # expects. Specifically, test use of PL_modglobal, which maps to a + # PerlInterpreter field whose position depends on sizeof(time_t). + if ($solution->{platform} eq 'Win32') + { + my $source_file = 'conftest.c'; + my $obj = 'conftest.obj'; + my $exe = 'conftest.exe'; + my @conftest = ($source_file, $obj, $exe); + push @unlink_on_exit, @conftest; + unlink $source_file; + open my $o, '>', $source_file + || croak "Could not write to $source_file"; + print $o ' + /* compare to plperl.h */ + #define __inline__ __inline + #define PERL_NO_GET_CONTEXT + #include <EXTERN.h> + #include <perl.h> + + int + main(int argc, char **argv) + { + int dummy_argc = 1; + char *dummy_argv[1] = {""}; + char *dummy_env[1] = {NULL}; + static PerlInterpreter *interp; + + PERL_SYS_INIT3(&dummy_argc, (char ***) &dummy_argv, + (char ***) &dummy_env); + interp = perl_alloc(); + perl_construct(interp); + { + dTHX; + const char key[] = "dummy"; + + PL_exit_flags |= PERL_EXIT_DESTRUCT_END; + hv_store(PL_modglobal, key, sizeof(key) - 1, newSViv(1), 0); + return hv_fetch(PL_modglobal, key, sizeof(key) - 1, 0) == NULL; + } + } +'; + close $o; + + # Build $source_file with a given #define, and return a true value + # if a run of the resulting binary exits successfully. + my $try_define = sub { + my $define = shift; + + unlink $obj, $exe; + my @cmd = ( + 'cl', + '-I' . $solution->{options}->{perl} . '/lib/CORE', + (map { "-D$_" } @perl_embed_ccflags, $define || ()), + $source_file, + '/link', + $perl_libs[0]); + my $compile_output = `@cmd 2>&1`; + -f $exe || die "Failed to build Perl test:\n$compile_output"; + + { + + # Some builds exhibit runtime failure through Perl warning + # 'Can't spawn "conftest.exe"'; supress that. + no warnings; + + # Disable error dialog boxes like we do in the postmaster. + # Here, we run code that triggers relevant errors. + use Win32API::File qw(SetErrorMode :SEM_); + my $oldmode = SetErrorMode( + SEM_FAILCRITICALERRORS | SEM_NOGPFAULTERRORBOX); + system(".\\$exe"); + SetErrorMode($oldmode); + } + + return !($? >> 8); + }; + + my $define_32bit_time = '_USE_32BIT_TIME_T'; + my $ok_now = $try_define->(undef); + my $ok_32bit = $try_define->($define_32bit_time); + unlink @conftest; + if (!$ok_now && !$ok_32bit) + { + + # Unsupported configuration. Since we used %Config from the + # Perl running the build scripts, this is expected if + # attempting to link with some other Perl. + die "Perl test fails with or without -D$define_32bit_time"; + } + elsif ($ok_now && $ok_32bit) + { + + # Resulting build may work, but it's especially important to + # verify with "vcregress plcheck". A refined test may avoid + # this outcome. + warn "Perl test passes with or without -D$define_32bit_time"; + } + elsif ($ok_32bit) + { + push(@perl_embed_ccflags, $define_32bit_time); + } # else $ok_now, hence no flag required + } + + print "CFLAGS recommended by Perl: $Config{ccflags}\n"; + print "CFLAGS to compile embedded Perl: ", + (join ' ', map { "-D$_" } @perl_embed_ccflags), "\n"; foreach my $f (@perl_embed_ccflags) { $plperl->AddDefine($f); @@ -614,20 +735,6 @@ sub mkvcbuild die 'Failed to create plperl_opmask.h' . "\n"; } } - $plperl->AddReference($postgres); - my $perl_path = $solution->{options}->{perl} . '\lib\CORE\*perl*'; - # ActivePerl 5.16 provided perl516.lib; 5.18 provided libperl518.a - my @perl_libs = - grep { /perl\d+\.lib$|libperl\d+\.a$/ } glob($perl_path); - if (@perl_libs == 1) - { - $plperl->AddLibrary($perl_libs[0]); - } - else - { - die -"could not identify perl library version matching pattern $perl_path\n"; - } # Add transform module dependent on plperl my $hstore_plperl = AddTransformModule( @@ -956,4 +1063,9 @@ sub AdjustModule } } +END +{ + unlink @unlink_on_exit; +} + 1;

view thread (94+ messages)  latest in thread

Message-ID: <20171204022932.GA3243163@rfd.leadboat.com>
Permalink:  ../20171204022932.GA3243163@rfd.leadboat.com/
Also on:    postgresql.org/message-id/20171204022932.GA3243163@rfd.leadboat.com

 ·  · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-hackers@postgresql.org
  Cc: noah@leadboat.com, tgl@sss.pgh.pa.us, dpage@postgresql.org, sandeep.thakkar@enterprisedb.com, ashu.coek88@gmail.com, robertmhaas@gmail.com
  Subject: Re: pl/perl extension fails on Windows
  In-Reply-To: <20171204022932.GA3243163@rfd.leadboat.com>
  References: <14838.1502379286@sss.pgh.pa.us> <1708!2.1502718166@sss.pgh.pa.us> <10406.1502743029@sss.pgh.pa.us> <CA+OCxow_P_K9kpsSeMwdnqoiaAx3YEDN6P1s-93L5A_iTUYHuw@mail.gmail.com> <1914.1502978317@sss.pgh.pa.us> <CA+OCxow3A8waWr3drvFtmBwfgD8RF6w7rC2z9eY8eMJ7zz3y4w@mail.gmail.com> <4515.1502981857@sss.pgh.pa.us> <17646.1502983856@sss.pgh.pa.us> <19438.1502986558@sss.pgh.pa.us> <20171130041441.GA3161526@rfd.leadboat.com> <20171204022932.GA3243163@rfd.leadboat.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox