Subject: [Pljava-dev] building pljava for solaris 10 sparc 64bit

From: pierce at hogranch.com (John R Pierce)
Date: Thu, 15 Apr 2010 15:03:14 -0700

so I've been having fun getting pl/java 'HEAD' working with Postgres 
8.4.3 64bit on Solaris Sparc, with JDK 1.5.0_23 64bit...

solved bunches of problems, have it built, I think, and am trying to 
figure out how to load things...

first, I had to manually fudge the pgxs/src/Makefile.global as its 
missing from the 64bit build of postgres for solaris, so I took the 
files from the 32bit build and copied/edited them to suit.  I used the 
vlaues displayed by pg_config to adjust whats in that Makefile.global 
...   so I built it, and fixed up the postgres environment.  fwiw, my 
built modules sit in /opt/FIS/pljava/

this is whats happening now...

$ export CLASSPATH=/usr/share/java/postgresql.jar:/opt/FIS/pljava/deploy.jar
$ $JAVA_HOME/bin/sparcv9/java org.postgresql.pljava.deploy.Deployer 
-install -password XXXXX
org.postgresql.util.PSQLException: ERROR: could not load library 
"/opt/FIS/pljava/pljava.so": ld.so.1: postgres: fatal: libjvm.so: open 
failed: No such file or directory
        at 
org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:1512)
        at 
org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1297)
        at 
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:188)
        at 
org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:430)
        at 
org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:332)
        at 
org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:324)
        at 
org.postgresql.pljava.deploy.Deployer.initJavaHandlers(Deployer.java:474)
        at org.postgresql.pljava.deploy.Deployer.main(Deployer.java:269)

-bash-3.00$ tail server.log
...
ERROR:  could not load library "/opt/FIS/pljava/pljava.so": ld.so.1: 
postgres: fatal: libjvm.so: open failed: No such file or directory
STATEMENT:  CREATE FUNCTION sqlj.java_call_handler() RETURNS 
language_handler AS 'pljava' LANGUAGE C
LOG:  unexpected EOF on client connection

These are in the postgres server 'start' method script (I'm running 
postgres under SMF control)

    JAVA_HOME=/usr/jdk/jdk1.5.0_23
    
LD_LIBRARY_PATH=$JAVA_HOME/jre/lib/sparcv9:$JAVA_HOME/jre/lib/sparcv9/server:/opt/FIS/pljava
    PATH=$JAVA_HOME/bin/sparcv9:$PATH

and that LD_LIBRARY_PATH *is* the right JVM

$ ls -l /usr/jdk/jdk1.5.0_23/jre/lib/sparcv9/server
total 24439
-rw-r--r--   1 root     bin         1423 Dec  3 01:49 Xusage.txt
lrwxrwxrwx   1 root     root          13 Dec 18  2008 libjsig.so -> 
../libjsig.so
-rwxr-xr-x   1 root     bin      12402328 Dec  3 01:49 libjvm.so
-rwxr-xr-x   1 root     bin        46656 Dec  3 01:49 libjvm_db.so
$ file /usr/jdk/jdk1.5.0_23/jre/lib/sparcv9/server/libjvm.so
/usr/jdk/jdk1.5.0_23/jre/lib/sparcv9/server/libjvm.so:  ELF 64-bit MSB 
dynamic lib SPARCV9 Version 1, dynamically linked, not stripped, no 
debugging information available

I'm not sure what to try next.

---

From: books at ejurka.com (Kris Jurka)
Date: Thu, 15 Apr 2010 18:14:36 -0400 (EDT)

On Thu, 15 Apr 2010, John R Pierce wrote:

> so I've been having fun getting pl/java 'HEAD' working with Postgres 8.4.3 
> 64bit on Solaris Sparc, with JDK 1.5.0_23 64bit...
>
> These are in the postgres server 'start' method script (I'm running postgres 
> under SMF control)
>

I'm not familiar with SMF, but putting environment variables into start 
scripts often doesn't work because they start as root and then su to the 
user running postgres and when that happens the current environment is 
scrapped and replaced by that of the postgres user's.

Kris Jurka

---

From: pierce at hogranch.com (John R Pierce)
Date: Thu, 15 Apr 2010 15:43:02 -0700

Kris Jurka wrote:
> I'm not familiar with SMF, but putting environment variables into 
> start scripts often doesn't work because they start as root and then 
> su to the user running postgres and when that happens the current 
> environment is scrapped and replaced by that of the postgres user's.

this script is setting PGBIN, PGDATA, and PGLOG before running pg_ctl, 
so I am pretty sure its already in the target user.   yes, indeed, SMF 
has a service property "method_context/user astring postgres"  which 
specifies what user to run the start method as, so its looking like 
changing the user is done -before- running this script.

what alternative methods would you suggest for getting LD_LIBRARY_PATH 
etc setup ?

---

From: books at ejurka.com (Kris Jurka)
Date: Thu, 15 Apr 2010 18:58:06 -0400 (EDT)

On Thu, 15 Apr 2010, John R Pierce wrote:

> this script is setting PGBIN, PGDATA, and PGLOG before running pg_ctl, so I 
> am pretty sure its already in the target user.   yes, indeed, SMF has a 
> service property "method_context/user astring postgres"  which specifies what 
> user to run the start method as, so its looking like changing the user is 
> done -before- running this script.

Yes, but those variables are used in the script, not used by the 
server once it starts running, so they're not the same thing and 
may not be a good template to work off of.

> what alternative methods would you suggest for getting LD_LIBRARY_PATH etc 
> setup ?

Perhaps you need to export those variables:

export LD_LIBRARY_PATH

Kris Jurka

---

From: pierce at hogranch.com (John R Pierce)
Date: Thu, 15 Apr 2010 16:27:54 -0700

Kris Jurka wrote:
> (... pl/java HEAD building on sparc 64bit solaris, pg8.4.3, jdk 
> 1.5.0_23....)
>> this script is setting PGBIN, PGDATA, and PGLOG before running 
>> pg_ctl, so I am pretty sure its already in the target user.   yes, 
>> indeed, SMF has a service property "method_context/user astring 
>> postgres"  which specifies what user to run the start method as, so 
>> its looking like changing the user is done -before- running this script.
>

duh, I should have thought of that...
all reet...   that got me to a NEW error :-)   [the fun never ends!]

$ java org.postgresql.pljava.deploy.Deployer -install -password XXXXXX
org.postgresql.util.PSQLException: ERROR: could not load library 
"/opt/FIS/pljava/pljava.so": ld.so.1: postgres: fatal: relocation error: 
file /opt/FIS/pljava/pljava.so: symbol Int64GetDatum: referenced symbol 
not found
        at 
org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:1512)
        at 
org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1297)
        at 
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:188)
        at 
org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:430)
        at 
org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:332)
        at 
org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:324)
        at 
org.postgresql.pljava.deploy.Deployer.initJavaHandlers(Deployer.java:474)
        at org.postgresql.pljava.deploy.Deployer.main(Deployer.java:269)

with the same environment stuff set in my shell, that .so seems to pass 
ldd just fine.

$ ldd /opt/FIS/pljava/pljava.so
        libjvm.so =>     
/usr/jdk/jdk1.5.0_23/jre/lib/sparcv9/server/libjvm.so
        libsocket.so.1 =>        /lib/64/libsocket.so.1
        libsched.so.1 =>         /usr/lib/64/libsched.so.1
        libdl.so.1 =>    /lib/64/libdl.so.1
        libCrun.so.1 =>  /usr/lib/64/libCrun.so.1
        libm.so.1 =>     /lib/64/libm.so.1
        libthread.so.1 =>        /lib/64/libthread.so.1
        libc.so.1 =>     /lib/64/libc.so.1
        libnsl.so.1 =>   /lib/64/libnsl.so.1
        libmp.so.2 =>    /lib/64/libmp.so.2
        libmd.so.1 =>    /lib/64/libmd.so.1
        libscf.so.1 =>   /lib/64/libscf.so.1
        libdoor.so.1 =>  /lib/64/libdoor.so.1
        libuutil.so.1 =>         /lib/64/libuutil.so.1
        libgen.so.1 =>   /lib/64/libgen.so.1
        libm.so.2 =>     /lib/64/libm.so.2
        /platform/SUNW,Sun-Fire-V490/lib/sparcv9/libc_psr.so.1
        /platform/SUNW,Sun-Fire-V490/lib/sparcv9/libmd_psr.so.1

---

From: books at ejurka.com (Kris Jurka)
Date: Thu, 15 Apr 2010 19:44:45 -0400 (EDT)

On Thu, 15 Apr 2010, John R Pierce wrote:

> "/opt/FIS/pljava/pljava.so": ld.so.1: postgres: fatal: relocation error: file 
> /opt/FIS/pljava/pljava.so: symbol Int64GetDatum: referenced symbol not found

Your hacking of the 32 bit build files is incomplete/incorrect, so you've 
built a pljava that is incompatible with your server.  I'm not entirely 
sure what you've done, but you don't have the right setting for 
FLOAT8PASSBYVAL in src/include/pg_config.h in the source tree.  I'm not 
sure where you've got that installed.

Kris Jurka

---

From: pierce at hogranch.com (John R Pierce)
Date: Thu, 15 Apr 2010 18:05:04 -0700

Kris Jurka wrote:
>> "/opt/FIS/pljava/pljava.so": ld.so.1: postgres: fatal: relocation 
>> error: file /opt/FIS/pljava/pljava.so: symbol Int64GetDatum: 
>> referenced symbol not found
>
> Your hacking of the 32 bit build files is incomplete/incorrect, so 
> you've built a pljava that is incompatible with your server.  I'm not 
> entirely sure what you've done, but you don't have the right setting 
> for FLOAT8PASSBYVAL in src/include/pg_config.h in the source tree.  
> I'm not sure where you've got that installed.

thats a big clue, thanks, I'll dive in and see what I can make of that...

k, the three files I copied from the 32bit to the 64bit pgxs/src dir in 
the postgres dir were Makefile.global, Makefile.port and Makefile.shlib

only Makefile.global -seemed- to have anything in it that referenced 
wordsize kinda stuff.   I changed the libdir and bindir paths.   I 
replaced configure_arguments with the output of pg_config

# Saved arguments from configure
configure_args = = '--prefix=/usr/postgres/8.4-community' 
'--exec-prefix=/usr/postgres/8.4-community' 
'--bindir=/usr/postgres/8.4-community/bin/64' 
'--libexecdir=/usr/postgres/8.4-community/bin/64' 
'--sbindir=/usr/postgres/8.4-community/bin/64' 
'--datadir=/usr/postgres/8.4-community/share' 
'--sysconfdir=/usr/postgres/8.4-community/etc' 
'--mandir=/usr/postgres/8.4-community/man' 
'--libdir=/usr/postgres/8.4-community/lib/64' 
'--includedir=/usr/postgres/8.4-community/include' 
'--sharedstatedir=/var/postgres/8.4-community' 
'--localstatedir=/var/postgres/8.4-community' 
'--with-system-tzdata=/usr/share/lib/zoneinfo' '--enable-nls' 
'--with-docdir=/usr/postgres/8.4-community/doc' '--with-python' 
'--with-pam' '--with-openssl' '--with-libedit-preferred' '--with-libxml' 
'--with-libxslt' '--with-gssapi' '--enable-thread-safety' 
'--enable-dtrace' 'DTRACEFLAGS=-64' 
'--with-includes=/zstore/pgsql/pg84b1/proto/root_sparc/usr/include:/zstore/pgsql/pg84b1/proto/root_sparc/usr/sfw/include:/usr/sfw/include' 
'--with-tclconfig=/usr/sfw/lib' 
'--with-libs=/zstore/pgsql/pg84b1/proto/root_sparc/usr/lib/64:/zstore/pgsql/pg84b1/proto/root_sparc/usr/sfw/lib/64:/usr/lib/64:/usr/sfw/lib/64' 
'CC=/opt/SUNWspro/bin/cc' 'CFLAGS=-xO3 -xarch=v9 -dalign   
-Wc,-Qiselect-regsym=0 -xspace -W0,-Lt -W2,-Rcond_elim -Xa  -xildoff ' 
'LDFLAGS=-L/zstore/pgsql/pg84b1/proto/root_sparc/usr/lib/64 
-L/zstore/pgsql/pg84b1/proto/root_sparc/usr/sfw/lib/64 
-L/usr/sfw/lib/64' 
'CPPFLAGS=-I/zstore/pgsql/pg84b1/proto/root_sparc/usr/include 
-I/zstore/pgsql/pg84b1/proto/root_sparc/usr/sfw/include -I/usr/sfw/include'

and, the CFLAGS stuff...

    CC = /opt/SUNWspro/bin/cc -Xa
    GCC =
    SUN_STUDIO_CC = yes
    CFLAGS = -xO3 -xarch=v9 -dalign   -Wc,-Qiselect-regsym=0 -xspace 
-W0,-Lt -W2,-Rcond_elim -Xa  -xildoff

and... and... . oh $$$$, (expletive deleted).   LDFLAGS !  I 
forgot/missed LDFLAGS... k, so fixing -those- up to match the pg_config 
output...

k, gmake clean && gmake .....   builds clean.  cp the .so and .jar's to 
my deployment directory...   and.  darnit, same error.  :-/   k, digging 
deeper.

one thing I noted, getting a fair number of
    line 47: warning: initialization type mismatch
from various C sources.  I assume these are normal ?

for laughs, I tried running the install.sql via psql rather than the 
java deploy script, its getting the same error...

$ psql -f ./src/sql/install.sql
CREATE SCHEMA
GRANT
psql:./src/sql/install.sql:6: ERROR:  could not load library 
"/opt/FIS/pljava/pljava.so": ld.so.1: postgres: fatal: relocation error: 
file /opt/FIS/pljava/pljava.so: symbol Int64GetDatum: referenced symbol 
not found
psql:./src/sql/install.sql:8: ERROR:  function sqlj.java_call_handler() 
does not exist
psql:./src/sql/install.sql:12: ERROR:  could not load library 
"/opt/FIS/pljava/pljava.so": ld.so.1: postgres: fatal: relocation error: 
file /opt/FIS/pljava/pljava.so: symbol Int64GetDatum: referenced symbol 
not found
psql:./src/sql/install.sql:14: ERROR:  function 
sqlj.javau_call_handler() does not exist
......(bunch of table creates ignored)....

I also have added these to the postgresql.conf file, per the Readme.html 
deployment instructions, I don't know if this is redundant or what.

    dynamic_library_path = '$libdir:/opt/FIS/pljava'
    custom_variable_classes = 'pljava'
    pljava.classpath = '/opt/FIS/pljava/pljava.jar'

---

From: books at ejurka.com (Kris Jurka)
Date: Thu, 15 Apr 2010 21:39:29 -0400 (EDT)

On Thu, 15 Apr 2010, John R Pierce wrote:

> k, the three files I copied from the 32bit to the 64bit pgxs/src dir in the 
> postgres dir were Makefile.global, Makefile.port and Makefile.shlib
>
> only Makefile.global -seemed- to have anything in it that referenced wordsize 
> kinda stuff.   I changed the libdir and bindir paths.   I replaced 
> configure_arguments with the output of pg_config
>

The package you have may be more busted than just some missing files.  If 
they packaged the 32bit headers with a 64bit server then you're hosed and 
it sort of sounds like that's what they did.

Run: "pg_controldata /path/to/your/data/directory" and check for the line:

Float8 argument passing:              by value

If it says the above instead of "by reference" then you've got a 64bit 
server that was not compiled with --disable-float8-byval.  Then run "grep 
FLOAT8 include/postgresql/server/pg_config.h" and I expect you'll see:

#define FLOAT8PASSBYVAL false

which conflicts with the running server.  If this is the case, your
package is no good for building an extension.

> from various C sources.  I assume these are normal ?

No, those aren't normal although I haven't built with Sun Studio on a 
64bit machine.  Again I suspect your package is busted and I wouldn't 
recommend trying to manually fix it up.  You need the right headers.

Kris Jurka