Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eLgVu-00071z-8m for pgsql-hackers@arkaria.postgresql.org; Mon, 04 Dec 2017 02:29:46 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eLgVt-0007WD-DR for pgsql-hackers@arkaria.postgresql.org; Mon, 04 Dec 2017 02:29:45 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1eLgVs-0007W2-U8 for pgsql-hackers@lists.postgresql.org; Mon, 04 Dec 2017 02:29:45 +0000 Received: from mail-pg0-x233.google.com ([2607:f8b0:400e:c05::233]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1eLgVo-00067D-36 for pgsql-hackers@postgresql.org; Mon, 04 Dec 2017 02:29:43 +0000 Received: by mail-pg0-x233.google.com with SMTP id k134so7176576pga.3 for ; Sun, 03 Dec 2017 18:29:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=leadboat.com; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=/IvZCk02un+4XJbS4/HXv+ZkDm95fAvdgE4xuiwN0r0=; b=gbqsAufyaY/WTOd4cd+6Uq/g2NVn1f2Q0d0SvVAQmn4Ieh4igVkERYEZ4XCkX+N9cy 00AB/f4qv0iDpqm4jqIMWVluYdgOiuE38dOaNqQDuK3o0G4f7Bna/p+/DTxnjsCf/9QF GZtr2dg/tmqTa1c8koOCR6cmEMBhzX72N4OwQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=/IvZCk02un+4XJbS4/HXv+ZkDm95fAvdgE4xuiwN0r0=; b=Djn3KI9Fk0IYCl7NlyIK9EdVrpr9c+rDApQoJHsc7eVZPxFWN+L0xQzvGn2JrzRcIf Xsklcmog7LxxNAK3Ka2BfvimYOFcdBQXNIGu0eYi+zbp83IN9ZXeSM/BUufW/Yqnaphq fh11mEniLEZnU2sjkzns/ijl2bo19ouaw17vohyNmXXDBMqVqUCaMFbmEv8esC1Ft+nT TygUVPQvuhKsbWnHU7LNHZUBW+16lRrHVvh4hT5u8GrVQOMJ4sLQ4yex3vxc9JIAQtcv eLOgHtxLcw9Bs1iBjX4hBMeR2JmmvRc5tKSb1AL7o9ym55SkANklzBdOWdQHmYav5mmA BobA== X-Gm-Message-State: AJaThX7FMWhEaNSwCRrqLkSwsexMqMn1s40nk9uTfBzovOGxsKEtGWcK VhoU8NH2vcNUMyR78ANipog6zg== X-Google-Smtp-Source: AGs4zMaUPLrwgPOlVHQlU+yoDTj+eFnPtYd0/2jIDCRjyQgDKerbAKveF/6n0le4gjeyAZ+900neng== X-Received: by 10.98.211.153 with SMTP id z25mr17782473pfk.61.1512354576877; Sun, 03 Dec 2017 18:29:36 -0800 (PST) Received: from rfd.leadboat.com (c-73-158-248-138.hsd1.ca.comcast.net. [73.158.248.138]) by smtp.gmail.com with ESMTPSA id x15sm18602209pfh.27.2017.12.03.18.29.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 03 Dec 2017 18:29:35 -0800 (PST) Date: Sun, 3 Dec 2017 18:29:32 -0800 From: Noah Misch To: Tom Lane Cc: Dave Page , Sandeep Thakkar , Ashutosh Sharma , Robert Haas , pgsql-hackers Subject: Re: pl/perl extension fails on Windows Message-ID: <20171204022932.GA3243163@rfd.leadboat.com> References: <14838.1502379286@sss.pgh.pa.us> <1708!2.1502718166@sss.pgh.pa.us> <10406.1502743029@sss.pgh.pa.us> <1914.1502978317@sss.pgh.pa.us> <4515.1502981857@sss.pgh.pa.us> <17646.1502983856@sss.pgh.pa.us> <19438.1502986558@sss.pgh.pa.us> <20171130041441.GA3161526@rfd.leadboat.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="wRRV7LY7NUeQGEoC" Content-Disposition: inline In-Reply-To: <20171130041441.GA3161526@rfd.leadboat.com> User-Agent: Mutt/1.5.24 (2015-08-30) List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk --wRRV7LY7NUeQGEoC Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Wed, Nov 29, 2017 at 08:14:41PM -0800, Noah Misch wrote: > 1. If $Config{gccversion} is nonempty, add _USE_32BIT_TIME_T. This will do > the wrong thing if MinGW changes its default to match modern MSVC. It will > do the wrong thing for a Perl built with "gcc -D__MINGW_USE_VC2005_COMPAT". > > 2. When configuring the build, determine whether to add _USE_32BIT_TIME_T by > running a test program built with and without that symbol. Perhaps have > the test program store and retrieve a PL_modglobal value. (PL_modglobal > maps to a PerlInterpreter field that follows the fields sensitive to > _USE_32BIT_TIME_T, and perlapi documents it since 5.8.0 or earlier.) This > is more principled than (1), but it will be more code and may have weird > interactions with rare Perl build options. > > I am inclined toward (2) if it takes no more than roughly a hundred lines of > code, else (1). Opinions? I regret investing in 32-bit Windows. If there's > any OS where a 32-bit PostgreSQL server makes sense today, it's not Windows. Here's an implementation of (2). This is more intricate than I hoped. One could argue for (1), but I estimate (2) wins by a nose. I successfully tested http://strawberryperl.com/download/5.14.4.1/strawberry-perl-5.14.4.1-32bit.msi (Perl 5.14.4; MinGW-built; must have -D_USE_32BIT_TIME_T) and http://get.enterprisedb.com/languagepacks/edb-languagepack-10-3-windows.exe (Perl 5.24.0; MSVC-built; must not have -D_USE_32BIT_TIME_T). I also tried http://strawberryperl.com/download/5.8.9/strawberry-perl-5.8.9.5.msi, which experienced a StackHash_0a9e crash during PERL_SYS_INIT3(), with or without -D_USE_32BIT_TIME_T. I expect that breakage is orthogonal. I didn't have ready access to obsolete MSVC-built Perl, so it will be interesting to see how the buildfarm likes this. --wRRV7LY7NUeQGEoC Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="time_t-msvc-v1.patch" MSVC: Test whether 32-bit Perl needs -D_USE_32BIT_TIME_T. Commits 5a5c2feca3fd858e70ea348822595547e6fa6c15 and b5178c5d08ca59e30f9d9428fa6fdb2741794e65 introduced support for modern MSVC-built, 32-bit Perl, but they broke use of MinGW-built, 32-bit Perl distributions like Strawberry Perl and modern ActivePerl. Perl has no robust means to report whether it expects a -D_USE_32BIT_TIME_T ABI, so test this. Back-patch to 9.3 (all supported versions). The chief alternative was a heuristic of adding -D_USE_32BIT_TIME_T when $Config{gccversion} is nonempty. That banks on every gcc-built Perl using the same ABI. gcc could change its default ABI the way MSVC once did, and one could build Perl with gcc and the non-default ABI. The GNU make build system could benefit from a similar test, without which it does not support MSVC-built Perl. For now, just add a comment. Most users taking the special step of building Perl with MSVC probably build PostgreSQL with MSVC. Discussion: https://postgr.es/m/20171130041441.GA3161526@rfd.leadboat.com diff --git a/config/perl.m4 b/config/perl.m4 index 76b1a92..caefb07 100644 --- a/config/perl.m4 +++ b/config/perl.m4 @@ -48,19 +48,23 @@ AC_DEFUN([PGAC_CHECK_PERL_CONFIGS], # PGAC_CHECK_PERL_EMBED_CCFLAGS # ----------------------------- -# We selectively extract stuff from $Config{ccflags}. We don't really need -# anything except -D switches, and other sorts of compiler switches can -# actively break things if Perl was compiled with a different compiler. -# Moreover, although Perl likes to put stuff like -D_LARGEFILE_SOURCE and -# -D_FILE_OFFSET_BITS=64 here, it would be fatal to try to compile PL/Perl -# to a different libc ABI than core Postgres uses. The available information -# says that all the symbols that affect Perl's own ABI begin with letters, -# so it should be sufficient to adopt -D switches for symbols not beginning -# with underscore. An exception is that we need to let through -# -D_USE_32BIT_TIME_T if it's present. (We probably could restrict that to -# only get through on Windows, but for the moment we let it through always.) -# For debugging purposes, let's have the configure output report the raw -# ccflags value as well as the set of flags we chose to adopt. +# We selectively extract stuff from $Config{ccflags}. For debugging purposes, +# let's have the configure output report the raw ccflags value as well as the +# set of flags we chose to adopt. We don't really need anything except -D +# switches, and other sorts of compiler switches can actively break things if +# Perl was compiled with a different compiler. Moreover, although Perl likes +# to put stuff like -D_LARGEFILE_SOURCE and -D_FILE_OFFSET_BITS=64 here, it +# would be fatal to try to compile PL/Perl to a different libc ABI than core +# Postgres uses. The available information says that most symbols that affect +# Perl's own ABI begin with letters, so it's almost sufficient to adopt -D +# switches for symbols not beginning with underscore. Some exceptions are the +# Windows-specific -D_USE_32BIT_TIME_T and -D__MINGW_USE_VC2005_COMPAT; see +# Mkvcbuild.pm for details. We absorb the former when Perl reports it. Perl +# never reports the latter, and we don't attempt to deduce when it's needed. +# Consequently, we don't support using MinGW to link to MSVC-built Perl. As +# of 2017, all supported ActivePerl and Strawberry Perl are MinGW-built. If +# that changes or an MSVC-built Perl distribution becomes prominent, we can +# revisit this limitation. AC_DEFUN([PGAC_CHECK_PERL_EMBED_CCFLAGS], [AC_REQUIRE([PGAC_PATH_PERL]) AC_MSG_CHECKING([for CFLAGS recommended by Perl]) diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm index 4c2e12e..93f364a 100644 --- a/src/tools/msvc/Mkvcbuild.pm +++ b/src/tools/msvc/Mkvcbuild.pm @@ -28,6 +28,7 @@ my $libpgcommon; my $libpgfeutils; my $postgres; my $libpq; +my @unlink_on_exit; # Set of variables for modules in contrib/ and src/test/modules/ my $contrib_defines = { 'refint' => 'REFINT_VERBOSE' }; @@ -517,34 +518,154 @@ sub mkvcbuild my $plperl = $solution->AddProject('plperl', 'dll', 'PLs', 'src/pl/plperl'); $plperl->AddIncludeDir($solution->{options}->{perl} . '/lib/CORE'); + $plperl->AddReference($postgres); + + my $perl_path = $solution->{options}->{perl} . '\lib\CORE\*perl*'; + + # ActivePerl 5.16 provided perl516.lib; 5.18 provided libperl518.a + my @perl_libs = + grep { /perl\d+\.lib$|libperl\d+\.a$/ } glob($perl_path); + if (@perl_libs == 1) + { + $plperl->AddLibrary($perl_libs[0]); + } + else + { + die +"could not identify perl library version matching pattern $perl_path\n"; + } # Add defines from Perl's ccflags; see PGAC_CHECK_PERL_EMBED_CCFLAGS my @perl_embed_ccflags; foreach my $f (split(" ", $Config{ccflags})) { - if ( $f =~ /^-D[^_]/ - || $f =~ /^-D_USE_32BIT_TIME_T/) + if ($f =~ /^-D[^_]/) { $f =~ s/\-D//; push(@perl_embed_ccflags, $f); } } - # Perl versions before 5.13.4 don't provide -D_USE_32BIT_TIME_T - # regardless of how they were built. On 32-bit Windows, assume - # such a version was built with a pre-MSVC-2005 compiler, and - # define the symbol anyway, so that we are compatible if we're - # being built with a later MSVC version. - push(@perl_embed_ccflags, '_USE_32BIT_TIME_T') - if $solution->{platform} eq 'Win32' - && $Config{PERL_REVISION} == 5 - && ($Config{PERL_VERSION} < 13 - || ( $Config{PERL_VERSION} == 13 - && $Config{PERL_SUBVERSION} < 4)); - - # Also, a hack to prevent duplicate definitions of uid_t/gid_t + # hack to prevent duplicate definitions of uid_t/gid_t push(@perl_embed_ccflags, 'PLPERL_HAVE_UID_GID'); + # Windows offers several 32-bit ABIs. Perl is sensitive to + # sizeof(time_t), one of the ABI dimensions. To get 32-bit time_t, + # use "cl -D_USE_32BIT_TIME_T" or plain "gcc". For 64-bit time_t, use + # "gcc -D__MINGW_USE_VC2005_COMPAT" or plain "cl". Before MSVC 2005, + # plain "cl" chose 32-bit time_t. PostgreSQL doesn't support building + # with pre-MSVC-2005 compilers, but it does support linking to Perl + # built with such a compiler. MSVC-built Perl 5.13.4 and later report + # -D_USE_32BIT_TIME_T in $Config{ccflags} if applicable, but + # MinGW-built Perl never reports -D_USE_32BIT_TIME_T despite typically + # needing it. Ignore the $Config{ccflags} opinion about + # -D_USE_32BIT_TIME_T, and use a runtime test to deduce the ABI Perl + # expects. Specifically, test use of PL_modglobal, which maps to a + # PerlInterpreter field whose position depends on sizeof(time_t). + if ($solution->{platform} eq 'Win32') + { + my $source_file = 'conftest.c'; + my $obj = 'conftest.obj'; + my $exe = 'conftest.exe'; + my @conftest = ($source_file, $obj, $exe); + push @unlink_on_exit, @conftest; + unlink $source_file; + open my $o, '>', $source_file + || croak "Could not write to $source_file"; + print $o ' + /* compare to plperl.h */ + #define __inline__ __inline + #define PERL_NO_GET_CONTEXT + #include + #include + + int + main(int argc, char **argv) + { + int dummy_argc = 1; + char *dummy_argv[1] = {""}; + char *dummy_env[1] = {NULL}; + static PerlInterpreter *interp; + + PERL_SYS_INIT3(&dummy_argc, (char ***) &dummy_argv, + (char ***) &dummy_env); + interp = perl_alloc(); + perl_construct(interp); + { + dTHX; + const char key[] = "dummy"; + + PL_exit_flags |= PERL_EXIT_DESTRUCT_END; + hv_store(PL_modglobal, key, sizeof(key) - 1, newSViv(1), 0); + return hv_fetch(PL_modglobal, key, sizeof(key) - 1, 0) == NULL; + } + } +'; + close $o; + + # Build $source_file with a given #define, and return a true value + # if a run of the resulting binary exits successfully. + my $try_define = sub { + my $define = shift; + + unlink $obj, $exe; + my @cmd = ( + 'cl', + '-I' . $solution->{options}->{perl} . '/lib/CORE', + (map { "-D$_" } @perl_embed_ccflags, $define || ()), + $source_file, + '/link', + $perl_libs[0]); + my $compile_output = `@cmd 2>&1`; + -f $exe || die "Failed to build Perl test:\n$compile_output"; + + { + + # Some builds exhibit runtime failure through Perl warning + # 'Can't spawn "conftest.exe"'; supress that. + no warnings; + + # Disable error dialog boxes like we do in the postmaster. + # Here, we run code that triggers relevant errors. + use Win32API::File qw(SetErrorMode :SEM_); + my $oldmode = SetErrorMode( + SEM_FAILCRITICALERRORS | SEM_NOGPFAULTERRORBOX); + system(".\\$exe"); + SetErrorMode($oldmode); + } + + return !($? >> 8); + }; + + my $define_32bit_time = '_USE_32BIT_TIME_T'; + my $ok_now = $try_define->(undef); + my $ok_32bit = $try_define->($define_32bit_time); + unlink @conftest; + if (!$ok_now && !$ok_32bit) + { + + # Unsupported configuration. Since we used %Config from the + # Perl running the build scripts, this is expected if + # attempting to link with some other Perl. + die "Perl test fails with or without -D$define_32bit_time"; + } + elsif ($ok_now && $ok_32bit) + { + + # Resulting build may work, but it's especially important to + # verify with "vcregress plcheck". A refined test may avoid + # this outcome. + warn "Perl test passes with or without -D$define_32bit_time"; + } + elsif ($ok_32bit) + { + push(@perl_embed_ccflags, $define_32bit_time); + } # else $ok_now, hence no flag required + } + + print "CFLAGS recommended by Perl: $Config{ccflags}\n"; + print "CFLAGS to compile embedded Perl: ", + (join ' ', map { "-D$_" } @perl_embed_ccflags), "\n"; foreach my $f (@perl_embed_ccflags) { $plperl->AddDefine($f); @@ -614,20 +735,6 @@ sub mkvcbuild die 'Failed to create plperl_opmask.h' . "\n"; } } - $plperl->AddReference($postgres); - my $perl_path = $solution->{options}->{perl} . '\lib\CORE\*perl*'; - # ActivePerl 5.16 provided perl516.lib; 5.18 provided libperl518.a - my @perl_libs = - grep { /perl\d+\.lib$|libperl\d+\.a$/ } glob($perl_path); - if (@perl_libs == 1) - { - $plperl->AddLibrary($perl_libs[0]); - } - else - { - die -"could not identify perl library version matching pattern $perl_path\n"; - } # Add transform module dependent on plperl my $hstore_plperl = AddTransformModule( @@ -956,4 +1063,9 @@ sub AdjustModule } } +END +{ + unlink @unlink_on_exit; +} + 1; --wRRV7LY7NUeQGEoC--