From: Andrew Dunstan <andrew@dunslane.net>
To: Michael Paquier <michael.paquier@gmail.com>
Cc: Peter Eisentraut <peter_e@gmx.net>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: [COMMITTERS] pgsql: Add transforms feature
Date: Wed, 29 Apr 2015 14:30:26 -0400
Message-ID: <55412342.8040409@dunslane.net> (raw)
In-Reply-To: <553FE942.4040604@dunslane.net>
References: <E1YmNw0-0002Kb-3Q@gemulon.postgresql.org>
<18844.1430066190@sss.pgh.pa.us>
<CAB7nPqRE6PpdMbwvytXCcKsnwg-TKM=0sfsH=DoCRKEvJVusKw@mail.gmail.com>
<553E29B5.9000102@gmx.net>
<553ECC34.2090104@dunslane.net>
<553EDA8A.9090806@dunslane.net>
<CAB7nPqT1k22QhvbX7npTY0BqLgGTPQU1eN8u78LBAuy57Y8jRg@mail.gmail.com>
<CAB7nPqRWEgqGQqTc-0bdRTBGbbnPyDEStnOEJw6ffFw=504nhQ@mail.gmail.com>
<553FAC70.2070309@dunslane.net>
<553FAF6D.3070703@dunslane.net>
<553FE942.4040604@dunslane.net>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>
On 04/28/2015 04:10 PM, Andrew Dunstan wrote:
>
> On 04/28/2015 12:03 PM, Andrew Dunstan wrote:
>>
>> [switching to -hackers]
>>
>> On 04/28/2015 11:51 AM, Andrew Dunstan wrote:
>>>
>>> On 04/28/2015 09:04 AM, Michael Paquier wrote:
>>>> On Tue, Apr 28, 2015 at 10:01 AM, Michael Paquier
>>>> <michael.paquier@gmail.com> wrote:
>>>>> On Tue, Apr 28, 2015 at 9:55 AM, Andrew Dunstan wrote:
>>>>>> w.r.t MSVC builds, it looks like we need entries in
>>>>>> $contrib_extraincludes
>>>>>> in src/tools/msvc/Mkvcbuild.pm at the very least.
>>>>> If our goal is to put back to green the Windows nodes as quick as
>>>>> possible, we could bypass their build this way , and we'll
>>>>> additionally need to update the install script and
>>>>> vcregress.pl/contribcheck to bypass those modules accordingly. Now I
>>>>> don't think that we should make the things produced inconsistent.
>>>> OK, attached are two patches to put back the buildfarm nodes using
>>>> MSVC to green
>>>> - 0001 adds support for build and installation of the new transform
>>>> modules: hstore_plperl, hstore_plpython and ltree_plpython. Note that
>>>> this patch is enough to put back the buildfarm to a green state for
>>>> MSVC, but it disables the regression tests for those modules,
>>>> something addressed in the next patch...
>>>> - 0002 adds support for regression tests for the new modules. The
>>>> thing is that we need to check the major version version of Python
>>>> available in configuration and choose what are the extensions to
>>>> preload before the tests run. It is a little bit hacky... But it has
>>>> the merit to work, and I am not sure we could have a cleaner solution
>>>> by looking at the Makefiles of the transform modules that use
>>>> currently conditions based on $(python_majorversion).
>>>> Regards,
>>>
>>>
>>> I have pushed the first of these with some tweaks.
>>>
>>> I'm looking at the second.
>>>
>>>
>>
>>
>> Regarding this second patch - apart from the buggy python version
>> test which I just fixed in the first patch, I see this:
>>
>> + if ($pyver eq "2")
>> + {
>> + push @opts, "--load-extension=plpythonu";
>> + push @opts, '--load-extension=' . $module . 'u';
>> + }
>>
>>
>> But what do we do for Python3?
>>
>> Do we actually have a Windows machine building with Python3?
>
>
> The answer seems to be "probably not". When I tried enabling this with
> bowerbird I got a ton of errors like these:
>
> plpy_cursorobject.obj : error LNK2001: unresolved external symbol
> PyObject_SelfIter [C:\prog\bf\root\HEAD\pgsql\plpython3.vcxproj]
> plpy_cursorobject.obj : error LNK2019: unresolved external symbol
> __imp_PyType_Ready referenced in function PLy_cursor_init_type
> [C:\prog\bf\root\HEAD\pgsql\plpython3.vcxproj]
>
>
> Something else to fix I guess.
>
>
OK, I fixed this as Michael suggested by installing the 64 bit python3
(it's a pity that python.org didn't offer that to me as a download on
their front page - that's a bit ugly).
However, when it came to running these tests that was a miserable
failure. And indeed, we don't run any regression tests for plpython3 on
MSVC.
So I committed this with just python2 tests enabled. All the buildfarm
MSVC hosts seem to be using python2 anyway.
Meanwhile, we have some work to do on the mingw/gcc side.
These changes help make some progress - they let compilation succeed for
hstore_plperl but I still get linkage failures. I suspect we need some
things marked for export that we haven't to be marked needed before.
inline patch 1: contrib/hstore_plperl/Makefile (+1/-2 lines) download inline-1.patchdiff --git a/contrib/hstore_plperl/Makefile b/contrib/hstore_plperl/Makefileindex ddf6036..81fe1af 100644--- a/contrib/hstore_plperl/Makefile+++ b/contrib/hstore_plperl/Makefile@@ -3,7 +3,7 @@
MODULE_big = hstore_plperl
OBJS = hstore_plperl.o
-PG_CPPFLAGS = -I$(top_srcdir)/src/pl/plperl -I$(perl_archlibexp)/CORE -I$(top_srcdir)/contrib/hstore+PG_CPPFLAGS = -I$(top_srcdir)/src/pl/plperl -I$(top_srcdir)/contrib/hstore
EXTENSION = hstore_plperl hstore_plperlu
DATA = hstore_plperl--1.0.sql hstore_plperlu--1.0.sql
@@ -21,3 +21,6 @@ top_builddir = ../..
include $(top_builddir)/src/Makefile.global
include $(top_srcdir)/contrib/contrib-global.mk
endif
+
+override CPPFLAGS := $(CPPFLAGS) -DPLPERL_HAVE_UID_GID
-I$(perl_archlibexp)/CORE
+override CFLAGS += -Wno-comment
The linkage failures look like:
x86_64-w64-mingw32-gcc -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -O2 -Wno-comment -I../../src/pl/plperl
-I../../contrib/hstore -I. -I. -I../../src/include
-I./src/include/port/win32 -DEXEC_BACKEND
-I/c/prog/3p64/include/libxml2 -I/c/prog/3p64/include
-I/c/prog/3p64/openssl/include "-I../../src/include/port/win32"
-DPLPERL_HAVE_UID_GID -IC:/Perl64/lib/CORE -c -o hstore_plperl.o
hstore_plperl.c
x86_64-w64-mingw32-gcc -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -O2 -Wno-comment -shared -static-libgcc
-o hstore_plperl.dll hstore_plperl.o -L../../src/port
-L../../src/common -Wl,--allow-multiple-definition
-Wl,--disable-auto-import -L/c/prog/3p64/lib -L/c/prog/3p64/lib
-L/c/prog/3p64/openssl/lib/VC -Wl,--as-needed -L../../src/backend
-lpostgres -lpgcommon -lpgport -lintl -lxslt -lxml2 -lz -lws2_32 -lm
-lws2_32 -Wl,--export-all-symbols -Wl,--out-implib=libhstore_plperl.a
hstore_plperl.o: In function `hstore_to_plperl':
c:\MinGW\msys\1.0\home\pgrunner\bf\root\HEAD\pgsql\contrib\hstore_plperl/hstore_plperl.c:17:
undefined reference to `hstoreUpgrade'
c:\MinGW\msys\1.0\home\pgrunner\bf\root\HEAD\pgsql\contrib\hstore_plperl/hstore_plperl.c:24:
undefined reference to `__imp_Perl_get_context'
c:\MinGW\msys\1.0\home\pgrunner\bf\root\HEAD\pgsql\contrib\hstore_plperl/hstore_plperl.c:24:
undefined reference to `__imp_Perl_newSV_type'
c:\MinGW\msys\1.0\home\pgrunner\bf\root\HEAD\pgsql\contrib\hstore_plperl/hstore_plperl.c:26:
undefined reference to `__imp_Perl_hv_common_key_len'
hstore_plperl.o: In function `cstr2sv':
c:\MinGW\msys\1.0\home\pgrunner\bf\root\HEAD\pgsql\contrib\hstore_plperl/../../src/pl/plperl/plperl_helpers.h:115:
undefined reference to `__imp_Perl_newSVpv'
c:\MinGW\msys\1.0\home\pgrunner\bf\root\HEAD\pgsql\contrib\hstore_plperl/../../src/pl/plperl/plperl_helpers.h:119:
undefined reference to `__imp_Perl_newSVpv'
hstore_plperl.o: In function `hstore_to_plperl':
c:\MinGW\msys\1.0\home\pgrunner\bf\root\HEAD\pgsql\contrib\hstore_plperl/hstore_plperl.c:32:
undefined reference to `__imp_Perl_newSV'
c:\MinGW\msys\1.0\home\pgrunner\bf\root\HEAD\pgsql\contrib\hstore_plperl/hstore_plperl.c:37:
undefined reference to `__imp_Perl_newRV'
hstore_plperl.o: In function `plperl_to_hstore':
c:\MinGW\msys\1.0\home\pgrunner\bf\root\HEAD\pgsql\contrib\hstore_plperl/hstore_plperl.c:57:
undefined reference to `__imp_Perl_get_context'
c:\MinGW\msys\1.0\home\pgrunner\bf\root\HEAD\pgsql\contrib\hstore_plperl/hstore_plperl.c:57:
undefined reference to `__imp_Perl_hv_iterinit'
c:\MinGW\msys\1.0\home\pgrunner\bf\root\HEAD\pgsql\contrib\hstore_plperl/hstore_plperl.c:59:
undefined reference to `__imp_Perl_hv_iternext_flags'
c:\MinGW\msys\1.0\home\pgrunner\bf\root\HEAD\pgsql\contrib\hstore_plperl/hstore_plperl.c:68:
undefined reference to `hstoreCheckKeyLen'
hstore_plperl.o: In function `sv2cstr':
c:\MinGW\msys\1.0\home\pgrunner\bf\root\HEAD\pgsql\contrib\hstore_plperl/../../src/pl/plperl/plperl_helpers.h:71:
undefined reference to `__imp_Perl_newSVsv'
c:\MinGW\msys\1.0\home\pgrunner\bf\root\HEAD\pgsql\contrib\hstore_plperl/../../src/pl/plperl/plperl_helpers.h:89:
undefined reference to `__imp_Perl_sv_2pvutf8'
hstore_plperl.o: In function `plperl_to_hstore':
c:\MinGW\msys\1.0\home\pgrunner\bf\root\HEAD\pgsql\contrib\hstore_plperl/hstore_plperl.c:64:
undefined reference to `__imp_Perl_newSVpvn_flags'
hstore_plperl.o: In function `sv2cstr':
c:\MinGW\msys\1.0\home\pgrunner\bf\root\HEAD\pgsql\contrib\hstore_plperl/../../src/pl/plperl/plperl_helpers.h:71:
undefined reference to `__imp_Perl_newSVsv'
hstore_plperl.o: In function `plperl_to_hstore':
c:\MinGW\msys\1.0\home\pgrunner\bf\root\HEAD\pgsql\contrib\hstore_plperl/hstore_plperl.c:80:
undefined reference to `hstoreCheckValLen'
hstore_plperl.o: In function `sv2cstr':
c:\MinGW\msys\1.0\home\pgrunner\bf\root\HEAD\pgsql\contrib\hstore_plperl/../../src/pl/plperl/plperl_helpers.h:98:
undefined reference to `__imp_Perl_sv_free'
c:\MinGW\msys\1.0\home\pgrunner\bf\root\HEAD\pgsql\contrib\hstore_plperl/../../src/pl/plperl/plperl_helpers.h:98:
undefined reference to `__imp_Perl_sv_free2'
c:\MinGW\msys\1.0\home\pgrunner\bf\root\HEAD\pgsql\contrib\hstore_plperl/../../src/pl/plperl/plperl_helpers.h:89:
undefined reference to `__imp_Perl_sv_2pvutf8'
c:\MinGW\msys\1.0\home\pgrunner\bf\root\HEAD\pgsql\contrib\hstore_plperl/../../src/pl/plperl/plperl_helpers.h:87:
undefined reference to `__imp_Perl_sv_2pv_flags'
hstore_plperl.o: In function `plperl_to_hstore':
c:\MinGW\msys\1.0\home\pgrunner\bf\root\HEAD\pgsql\contrib\hstore_plperl/hstore_plperl.c:87:
undefined reference to `hstoreUniquePairs'
c:\MinGW\msys\1.0\home\pgrunner\bf\root\HEAD\pgsql\contrib\hstore_plperl/hstore_plperl.c:88:
undefined reference to `hstorePairs'
hstore_plperl.o: In function `sv2cstr':
c:\MinGW\msys\1.0\home\pgrunner\bf\root\HEAD\pgsql\contrib\hstore_plperl/../../src/pl/plperl/plperl_helpers.h:98:
undefined reference to `__imp_Perl_sv_free'
c:\MinGW\msys\1.0\home\pgrunner\bf\root\HEAD\pgsql\contrib\hstore_plperl/../../src/pl/plperl/plperl_helpers.h:98:
undefined reference to `__imp_Perl_sv_free2'
c:\MinGW\msys\1.0\home\pgrunner\bf\root\HEAD\pgsql\contrib\hstore_plperl/../../src/pl/plperl/plperl_helpers.h:87:
undefined reference to `__imp_Perl_sv_2pv_flags'
collect2.exe: error: ld returned 1 exit status
cheers
andrew
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-committers@postgresql.org
Cc: andrew@dunslane.net, michael.paquier@gmail.com, peter_e@gmx.net, tgl@sss.pgh.pa.us, pgsql-hackers@postgresql.org
Subject: Re: [COMMITTERS] pgsql: Add transforms feature
In-Reply-To: <55412342.8040409@dunslane.net>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox