From: Noah Misch <noah@leadboat.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Dave Page <dpage@postgresql.org>
Cc: Sandeep Thakkar <sandeep.thakkar@enterprisedb.com>
Cc: Ashutosh Sharma <ashu.coek88@gmail.com>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: pl/perl extension fails on Windows
Date: Sun, 3 Dec 2017 18:29:32 -0800
Message-ID: <20171204022932.GA3243163@rfd.leadboat.com> (raw)
In-Reply-To: <20171130041441.GA3161526@rfd.leadboat.com>
References: <14838.1502379286@sss.pgh.pa.us>
<1708!2.1502718166@sss.pgh.pa.us>
<10406.1502743029@sss.pgh.pa.us>
<CA+OCxow_P_K9kpsSeMwdnqoiaAx3YEDN6P1s-93L5A_iTUYHuw@mail.gmail.com>
<1914.1502978317@sss.pgh.pa.us>
<CA+OCxow3A8waWr3drvFtmBwfgD8RF6w7rC2z9eY8eMJ7zz3y4w@mail.gmail.com>
<4515.1502981857@sss.pgh.pa.us>
<17646.1502983856@sss.pgh.pa.us>
<19438.1502986558@sss.pgh.pa.us>
<20171130041441.GA3161526@rfd.leadboat.com>
On Wed, Nov 29, 2017 at 08:14:41PM -0800, Noah Misch wrote:
> 1. If $Config{gccversion} is nonempty, add _USE_32BIT_TIME_T. This will do
> the wrong thing if MinGW changes its default to match modern MSVC. It will
> do the wrong thing for a Perl built with "gcc -D__MINGW_USE_VC2005_COMPAT".
>
> 2. When configuring the build, determine whether to add _USE_32BIT_TIME_T by
> running a test program built with and without that symbol. Perhaps have
> the test program store and retrieve a PL_modglobal value. (PL_modglobal
> maps to a PerlInterpreter field that follows the fields sensitive to
> _USE_32BIT_TIME_T, and perlapi documents it since 5.8.0 or earlier.) This
> is more principled than (1), but it will be more code and may have weird
> interactions with rare Perl build options.
>
> I am inclined toward (2) if it takes no more than roughly a hundred lines of
> code, else (1). Opinions? I regret investing in 32-bit Windows. If there's
> any OS where a 32-bit PostgreSQL server makes sense today, it's not Windows.
Here's an implementation of (2). This is more intricate than I hoped. One
could argue for (1), but I estimate (2) wins by a nose. I successfully tested
http://strawberryperl.com/download/5.14.4.1/strawberry-perl-5.14.4.1-32bit.msi
(Perl 5.14.4; MinGW-built; must have -D_USE_32BIT_TIME_T) and
http://get.enterprisedb.com/languagepacks/edb-languagepack-10-3-windows.exe
(Perl 5.24.0; MSVC-built; must not have -D_USE_32BIT_TIME_T). I also tried
http://strawberryperl.com/download/5.8.9/strawberry-perl-5.8.9.5.msi, which
experienced a StackHash_0a9e crash during PERL_SYS_INIT3(), with or without
-D_USE_32BIT_TIME_T. I expect that breakage is orthogonal. I didn't have
ready access to obsolete MSVC-built Perl, so it will be interesting to see how
the buildfarm likes this.
Attachments:time_t-msvc-v1.patch (text/plain, 10.3K) download
MSVC: Test whether 32-bit Perl needs -D_USE_32BIT_TIME_T.
Commits 5a5c2feca3fd858e70ea348822595547e6fa6c15 and
b5178c5d08ca59e30f9d9428fa6fdb2741794e65 introduced support for modern
MSVC-built, 32-bit Perl, but they broke use of MinGW-built, 32-bit Perl
distributions like Strawberry Perl and modern ActivePerl. Perl has no
robust means to report whether it expects a -D_USE_32BIT_TIME_T ABI, so
test this. Back-patch to 9.3 (all supported versions).
The chief alternative was a heuristic of adding -D_USE_32BIT_TIME_T when
$Config{gccversion} is nonempty. That banks on every gcc-built Perl
using the same ABI. gcc could change its default ABI the way MSVC once
did, and one could build Perl with gcc and the non-default ABI.
The GNU make build system could benefit from a similar test, without
which it does not support MSVC-built Perl. For now, just add a comment.
Most users taking the special step of building Perl with MSVC probably
build PostgreSQL with MSVC.
Discussion: https://postgr.es/m/20171130041441.GA3161526@rfd.leadboat.com
diff --git a/config/perl.m4 b/config/perl.m4index 76b1a92..caefb07 100644--- a/config/perl.m4+++ b/config/perl.m4@@ -48,19 +48,23 @@ AC_DEFUN([PGAC_CHECK_PERL_CONFIGS],
# PGAC_CHECK_PERL_EMBED_CCFLAGS
# -----------------------------
-# We selectively extract stuff from $Config{ccflags}. We don't really need-# anything except -D switches, and other sorts of compiler switches can-# actively break things if Perl was compiled with a different compiler.-# Moreover, although Perl likes to put stuff like -D_LARGEFILE_SOURCE and-# -D_FILE_OFFSET_BITS=64 here, it would be fatal to try to compile PL/Perl-# to a different libc ABI than core Postgres uses. The available information-# says that all the symbols that affect Perl's own ABI begin with letters,-# so it should be sufficient to adopt -D switches for symbols not beginning-# with underscore. An exception is that we need to let through-# -D_USE_32BIT_TIME_T if it's present. (We probably could restrict that to-# only get through on Windows, but for the moment we let it through always.)-# For debugging purposes, let's have the configure output report the raw-# ccflags value as well as the set of flags we chose to adopt.+# We selectively extract stuff from $Config{ccflags}. For debugging purposes,+# let's have the configure output report the raw ccflags value as well as the+# set of flags we chose to adopt. We don't really need anything except -D+# switches, and other sorts of compiler switches can actively break things if+# Perl was compiled with a different compiler. Moreover, although Perl likes+# to put stuff like -D_LARGEFILE_SOURCE and -D_FILE_OFFSET_BITS=64 here, it+# would be fatal to try to compile PL/Perl to a different libc ABI than core+# Postgres uses. The available information says that most symbols that affect+# Perl's own ABI begin with letters, so it's almost sufficient to adopt -D+# switches for symbols not beginning with underscore. Some exceptions are the+# Windows-specific -D_USE_32BIT_TIME_T and -D__MINGW_USE_VC2005_COMPAT; see+# Mkvcbuild.pm for details. We absorb the former when Perl reports it. Perl+# never reports the latter, and we don't attempt to deduce when it's needed.+# Consequently, we don't support using MinGW to link to MSVC-built Perl. As+# of 2017, all supported ActivePerl and Strawberry Perl are MinGW-built. If+# that changes or an MSVC-built Perl distribution becomes prominent, we can+# revisit this limitation.
AC_DEFUN([PGAC_CHECK_PERL_EMBED_CCFLAGS],
[AC_REQUIRE([PGAC_PATH_PERL])
AC_MSG_CHECKING([for CFLAGS recommended by Perl])
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pmindex 4c2e12e..93f364a 100644--- a/src/tools/msvc/Mkvcbuild.pm+++ b/src/tools/msvc/Mkvcbuild.pm@@ -28,6 +28,7 @@ my $libpgcommon;
my $libpgfeutils;
my $postgres;
my $libpq;
+my @unlink_on_exit;
# Set of variables for modules in contrib/ and src/test/modules/
my $contrib_defines = { 'refint' => 'REFINT_VERBOSE' };
@@ -517,34 +518,154 @@ sub mkvcbuild
my $plperl =
$solution->AddProject('plperl', 'dll', 'PLs', 'src/pl/plperl');
$plperl->AddIncludeDir($solution->{options}->{perl} . '/lib/CORE');
+ $plperl->AddReference($postgres);++ my $perl_path = $solution->{options}->{perl} . '\lib\CORE\*perl*';++ # ActivePerl 5.16 provided perl516.lib; 5.18 provided libperl518.a+ my @perl_libs =+ grep { /perl\d+\.lib$|libperl\d+\.a$/ } glob($perl_path);+ if (@perl_libs == 1)+ {+ $plperl->AddLibrary($perl_libs[0]);+ }+ else+ {+ die+"could not identify perl library version matching pattern $perl_path\n";+ }
# Add defines from Perl's ccflags; see PGAC_CHECK_PERL_EMBED_CCFLAGS
my @perl_embed_ccflags;
foreach my $f (split(" ", $Config{ccflags}))
{
- if ( $f =~ /^-D[^_]/- || $f =~ /^-D_USE_32BIT_TIME_T/)+ if ($f =~ /^-D[^_]/)
{
$f =~ s/\-D//;
push(@perl_embed_ccflags, $f);
}
}
- # Perl versions before 5.13.4 don't provide -D_USE_32BIT_TIME_T- # regardless of how they were built. On 32-bit Windows, assume- # such a version was built with a pre-MSVC-2005 compiler, and- # define the symbol anyway, so that we are compatible if we're- # being built with a later MSVC version.- push(@perl_embed_ccflags, '_USE_32BIT_TIME_T')- if $solution->{platform} eq 'Win32'- && $Config{PERL_REVISION} == 5- && ($Config{PERL_VERSION} < 13- || ( $Config{PERL_VERSION} == 13- && $Config{PERL_SUBVERSION} < 4));-- # Also, a hack to prevent duplicate definitions of uid_t/gid_t+ # hack to prevent duplicate definitions of uid_t/gid_t
push(@perl_embed_ccflags, 'PLPERL_HAVE_UID_GID');
+ # Windows offers several 32-bit ABIs. Perl is sensitive to+ # sizeof(time_t), one of the ABI dimensions. To get 32-bit time_t,+ # use "cl -D_USE_32BIT_TIME_T" or plain "gcc". For 64-bit time_t, use+ # "gcc -D__MINGW_USE_VC2005_COMPAT" or plain "cl". Before MSVC 2005,+ # plain "cl" chose 32-bit time_t. PostgreSQL doesn't support building+ # with pre-MSVC-2005 compilers, but it does support linking to Perl+ # built with such a compiler. MSVC-built Perl 5.13.4 and later report+ # -D_USE_32BIT_TIME_T in $Config{ccflags} if applicable, but+ # MinGW-built Perl never reports -D_USE_32BIT_TIME_T despite typically+ # needing it. Ignore the $Config{ccflags} opinion about+ # -D_USE_32BIT_TIME_T, and use a runtime test to deduce the ABI Perl+ # expects. Specifically, test use of PL_modglobal, which maps to a+ # PerlInterpreter field whose position depends on sizeof(time_t).+ if ($solution->{platform} eq 'Win32')+ {+ my $source_file = 'conftest.c';+ my $obj = 'conftest.obj';+ my $exe = 'conftest.exe';+ my @conftest = ($source_file, $obj, $exe);+ push @unlink_on_exit, @conftest;+ unlink $source_file;+ open my $o, '>', $source_file+ || croak "Could not write to $source_file";+ print $o '+ /* compare to plperl.h */+ #define __inline__ __inline+ #define PERL_NO_GET_CONTEXT+ #include <EXTERN.h>+ #include <perl.h>++ int+ main(int argc, char **argv)+ {+ int dummy_argc = 1;+ char *dummy_argv[1] = {""};+ char *dummy_env[1] = {NULL};+ static PerlInterpreter *interp;++ PERL_SYS_INIT3(&dummy_argc, (char ***) &dummy_argv,+ (char ***) &dummy_env);+ interp = perl_alloc();+ perl_construct(interp);+ {+ dTHX;+ const char key[] = "dummy";++ PL_exit_flags |= PERL_EXIT_DESTRUCT_END;+ hv_store(PL_modglobal, key, sizeof(key) - 1, newSViv(1), 0);+ return hv_fetch(PL_modglobal, key, sizeof(key) - 1, 0) == NULL;+ }+ }+';+ close $o;++ # Build $source_file with a given #define, and return a true value+ # if a run of the resulting binary exits successfully.+ my $try_define = sub {+ my $define = shift;++ unlink $obj, $exe;+ my @cmd = (+ 'cl',+ '-I' . $solution->{options}->{perl} . '/lib/CORE',+ (map { "-D$_" } @perl_embed_ccflags, $define || ()),+ $source_file,+ '/link',+ $perl_libs[0]);+ my $compile_output = `@cmd 2>&1`;+ -f $exe || die "Failed to build Perl test:\n$compile_output";++ {++ # Some builds exhibit runtime failure through Perl warning+ # 'Can't spawn "conftest.exe"'; supress that.+ no warnings;++ # Disable error dialog boxes like we do in the postmaster.+ # Here, we run code that triggers relevant errors.+ use Win32API::File qw(SetErrorMode :SEM_);+ my $oldmode = SetErrorMode(+ SEM_FAILCRITICALERRORS | SEM_NOGPFAULTERRORBOX);+ system(".\\$exe");+ SetErrorMode($oldmode);+ }++ return !($? >> 8);+ };++ my $define_32bit_time = '_USE_32BIT_TIME_T';+ my $ok_now = $try_define->(undef);+ my $ok_32bit = $try_define->($define_32bit_time);+ unlink @conftest;+ if (!$ok_now && !$ok_32bit)+ {++ # Unsupported configuration. Since we used %Config from the+ # Perl running the build scripts, this is expected if+ # attempting to link with some other Perl.+ die "Perl test fails with or without -D$define_32bit_time";+ }+ elsif ($ok_now && $ok_32bit)+ {++ # Resulting build may work, but it's especially important to+ # verify with "vcregress plcheck". A refined test may avoid+ # this outcome.+ warn "Perl test passes with or without -D$define_32bit_time";+ }+ elsif ($ok_32bit)+ {+ push(@perl_embed_ccflags, $define_32bit_time);+ } # else $ok_now, hence no flag required+ }++ print "CFLAGS recommended by Perl: $Config{ccflags}\n";+ print "CFLAGS to compile embedded Perl: ",+ (join ' ', map { "-D$_" } @perl_embed_ccflags), "\n";
foreach my $f (@perl_embed_ccflags)
{
$plperl->AddDefine($f);
@@ -614,20 +735,6 @@ sub mkvcbuild
die 'Failed to create plperl_opmask.h' . "\n";
}
}
- $plperl->AddReference($postgres);- my $perl_path = $solution->{options}->{perl} . '\lib\CORE\*perl*';- # ActivePerl 5.16 provided perl516.lib; 5.18 provided libperl518.a- my @perl_libs =- grep { /perl\d+\.lib$|libperl\d+\.a$/ } glob($perl_path);- if (@perl_libs == 1)- {- $plperl->AddLibrary($perl_libs[0]);- }- else- {- die-"could not identify perl library version matching pattern $perl_path\n";- }
# Add transform module dependent on plperl
my $hstore_plperl = AddTransformModule(
@@ -956,4 +1063,9 @@ sub AdjustModule
}
}
+END+{+ unlink @unlink_on_exit;+}+
1;
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-hackers@postgresql.org
Cc: noah@leadboat.com, tgl@sss.pgh.pa.us, dpage@postgresql.org, sandeep.thakkar@enterprisedb.com, ashu.coek88@gmail.com, robertmhaas@gmail.com
Subject: Re: pl/perl extension fails on Windows
In-Reply-To: <20171204022932.GA3243163@rfd.leadboat.com>
References: <14838.1502379286@sss.pgh.pa.us> <1708!2.1502718166@sss.pgh.pa.us> <10406.1502743029@sss.pgh.pa.us> <CA+OCxow_P_K9kpsSeMwdnqoiaAx3YEDN6P1s-93L5A_iTUYHuw@mail.gmail.com> <1914.1502978317@sss.pgh.pa.us> <CA+OCxow3A8waWr3drvFtmBwfgD8RF6w7rC2z9eY8eMJ7zz3y4w@mail.gmail.com> <4515.1502981857@sss.pgh.pa.us> <17646.1502983856@sss.pgh.pa.us> <19438.1502986558@sss.pgh.pa.us> <20171130041441.GA3161526@rfd.leadboat.com> <20171204022932.GA3243163@rfd.leadboat.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox