Subject: [Pljava-dev] built, now how to install?

From: John R Pierce <[email-masked]>
Date: Mon, 11 Jan 2016 17:59:42 -0800

so I've built the latest off git with mvn install, on CentOS 6, with 
openjdk 1.8 and postgres 9.4.latest...     the 'installing on linux' 
instructions seem, well, old...    I don't like the idea of putting 
unpackaged files in directories 'owned' by packages, like $PGHOME/lib   
so I'm trying to put them in /usr/share/pljava/ ...

I think these instructions https://github.com/tada/pljava/wiki/Deployer 
might need some updating, and be integrated with or linked from the 
'installing from linux' page....

on 
https://github.com/tada/pljava/wiki/Installing-on-linux-%28or-other-%2Anix%29

I had to use ...

     export JRE_HOME=/usr/lib/jvm/java-1.7.0-openjdk.x86_64/jre
     export LD_LIBRARY_PATH=$JRE_HOME/lib/amd64:$JRE_HOME/lib/amd64/server

(doc said /client ...)

anyways, um, i"m currently stuck at the deployer...

$ java -cp 
/usr/share/pljava/pljava.jar:/usr/share/pljava/postgresql-9.4.1207.jar 
org.postgresql.pljava.deploy.Deployer -install
Error: Could not find or load main class 
org.postgresql.pljava.deploy.Deployer

$ psql -c "select version();"
version
---------------------------------------------------------------------------------------------------------------
  PostgreSQL 9.4.5 on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 
4.4.7 20120313 (Red Hat 4.4.7-16), 64-bit
(1 row)

$ java -version
java version "1.7.0_91"
OpenJDK Runtime Environment (rhel-2.6.2.2.el6_7-x86_64 u91-b00)
OpenJDK 64-Bit Server VM (build 24.91-b01, mixed mode)

some notes re: 
https://github.com/tada/pljava/wiki/Installing-on-linux-%28or-other-%2Anix%29

I'm pretty sure those Solaris Notes are quite obsolete.   Those were my 
doing some years back.   I'm pretty sure the include/pg_config.h in the 
standard solaris build was long ago fixed.   We're no longer developing 
for Solaris in my environment, so someone else will need to follow up.

I'm also sure the first paragraph is completely obsolete with respect to 
1.6 not working.

GCJ probably should be completely deprecated and dropped, as its a dead 
project, having java-openjdk available pretty much negates gcj's raison 
d'être.   I suppose the gcj note on this page, and on the building page, 
could be put on their own GCJ page, along with a note suggesting its all 
deprecated.

I'd do some of these edits myself but I have no idea how to submit them 
to git

-- 
john r pierce, recycling bits in santa cruz

_______________________________________________
Pljava-dev mailing list
[email-masked]
http://lists.pgfoundry.org/mailman/listinfo/pljava-dev

---

From: Chapman Flack <[email-masked]>
Date: Mon, 11 Jan 2016 21:02:16 -0500

Um, hold on a sec ...

On 01/11/16 20:59, John R Pierce wrote:
> so I've built the latest off git with mvn install, on CentOS 6, with openjdk
> 1.8 and postgres 9.4.latest...     the 'installing on linux' instructions
> seem, well, old...    I don't like the idea of putting unpackaged files in
> directories 'owned' by packages, like $PGHOME/lib   so I'm trying to put
> them in /usr/share/pljava/ ...
> 
> Error: Could not find or load main class org.postgresql.pljava.deploy.Deployer
> 
>  PostgreSQL 9.4.5 on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 4.4.7
> 20120313 (Red Hat 4.4.7-16), 64-bit
> (1 row)
> 
> $ java -version
> java version "1.7.0_91"
> standard solaris build was long ago fixed.   We're no longer developing for
> Solaris in my environment, so someone else will need to follow up.
> 
> I'm also sure the first paragraph is completely obsolete with respect to 1.6
> not working.
> 
> I'd do some of these edits myself but I have no idea how to submit them to git
> 
> 
> 
>

---

From: Chapman Flack <[email-masked]>
Date: Mon, 11 Jan 2016 22:03:33 -0500

I've pushed the latest documentation at http://tada.github.io/pljava/
and updates to the wiki at https://github.com/tada/pljava/wiki.

TL;DR Deployer is an ex-thing; install.sql and uninstall.sql are
ex-things.

If you had no special preferences about where things go, you would
simply "java -jar <the-jar-file-that-gets-created-in-pljava-packaging>.jar"
and then in psql say "SET pljava.libjvm_location TO something;" (where
"something" is in your distribution is too hard to predict), and then
"CREATE EXTENSION pljava;".

The "java -jar" command would have put the files in standard pgsql
places: the extension .control file in $sharedir/extension, the .so
in $libdir, etc. Yes, that is stuffing them into PostgreSQL-owned
directories, but CREATE EXTENSION strongly assumes some of that.
Other files for $sharedir are put in a new $sharedir/pljava directory,
and not stuffed straight into $sharedir.

>> I don't like the idea of putting unpackaged files in

Philosophically, I am kind of with you here, I like to put things in
their own places. You can do that, by the way, with -D options on the
"java -jar" command that extracts the pieces:

java -Dpgconfig.pkglibdir=/usr/lib/pljava \
     -Dpgconfig.sharedir=/usr/share/pljava -jar the-package-jar.jar

If you do that, you won't have CREATE EXTENSION "just work" until you
have set dynamic_library_path to include /usr/lib/pljava and
pljava.classpath to /usr/share/pljava/pljava/pljava-${project.version}.jar.

In other words, the extraction code built into the jar is just a dumb
extractor; it will let you override where it puts things, but it will
not edit them during extraction to hardcode the new locations in; it's
just up to you to set the configuration variables to reflect the locations
you chose. (Hmm, if you like CREATE EXTENSION, you may also have to edit
the 'directory' setting in pljava.control.)

>> I think these instructions https://github.com/tada/pljava/wiki/Deployer

404 Deployer? What deployer? :)

>> Error: Could not find or load main class org.postgresql.pljava.deploy.Deployer

Deployer hasn't actually vanished from the build (yet), but it builds in
its own jar, it is not merged into pljava.jar.

However, I do not foresee it being maintained ... it won't be usable at
all after the next time something changes in the pljava schema, and even
until then it is still doing things the hard way.

>> Solaris in my environment, so someone else will need to follow up.

The wiki installation page now points to the tada.github.io/pljava
installation page, which has a few "special topics" subpages. If there
is still anything special about Solaris, it can go there. I didn't see
anything that seemed to be still applicable.

>> project, having java-openjdk available pretty much negates gcj's raison

Granted! :)

>> I'd do some of these edits myself but I have no idea how to submit

Yes, github wikis are strange that way, if not world-writable they are
actually harder to submit to than the regular code repo, where you can
easily do pull requests.

But I am trying to have current, authoritative documentation on the
tada.github.io/pljava site (you find all the doc sources in src/site
and can easily do a standard github pull request).  There is still
worthwhile content on the wiki, that I'd like to see moved over
gradually as time permits.

Of course, please look over the new docs and wiki to see how much
of what you were thinking of changing still needs to be changed. :)

-Chap

---

From: John R Pierce <[email-masked]>
Date: Mon, 11 Jan 2016 21:47:15 -0800

On 1/11/2016 7:03 PM, Chapman Flack wrote:
> of what you were thinking of changing still needs to be changed.:)

thanks muchos for all of the above, I'll tackle it from scratch in the 
morning...    tonights project (its 9:40pm Monday here) is rebuilding my 
college daughter's laptop before she returns to school on 
wednesday....    ah, I just loved trashed windows systems (NOT!)

---

From: John R Pierce <[email-masked]>
Date: Fri, 15 Jan 2016 04:35:42 -0800

On 1/11/2016 7:03 PM, Chapman Flack wrote:
> I've pushed the latest documentation athttp://tada.github.io/pljava/
> and updates to the wiki athttps://github.com/tada/pljava/wiki.

thanks, I successfully built and tested it yesterday, quite painlessly 
(grrr, why isn't maven packaged as an RPM in one of the major rpm 
repos??   the one I found wanted to drag in openjdk 1.6.0)

I wrote up the build procedure for deployment at one of our other sites 
where they actually need it, and got back a report that the test failed 
with what appear to be character encoding errors.   my test server was 
running en_US.UTF8, while I suspect this production site uses 
POSIX/SQLASCII...   I'll rebuild on a SQLASCII test system in my lab to 
see if this is the case (unluckily, the production site has already gone 
home for the weekend, so I won't know what server_encoding they are 
using until Monday).

---

From: Chapman Flack <[email-masked]>
Date: Fri, 15 Jan 2016 10:24:54 -0500

> repos??   the one I found wanted to drag in openjdk 1.6.0)

I don't know, but because it's pure Java and will go out itself
and fetch everything else it needs the first time you run it,
there is really no need to fret over finding an installer for it.
You just grab a tarball or zipball (why, in 2016, are we still
asking you "tar or zip?" before you can download some Java code? :)
from http://maven.apache.org/download.cgi and extract it in your
favorite temp directory and run bin/mvn and watch the downloads fly.

If you edit your .m2/settings.xml to put the repository in temp
space too, it all goes away when you no longer need it.

> test failed
> POSIX/SQLASCII... 

If the server is SQL_ASCII and the only failing test was the
UnicodeRoundTripTest, then PL/Java is working and that's as good
as it's going to get. That particular test doesn't even have a
definable meaning when the server encoding is SQL_ASCII.

To elaborate, that test is the regression test for bug 21
https://github.com/tada/pljava/issues/21 and it's there to
confirm that PL/Java will faithfully roundtrip all Unicode
characters from and to the server encoding.  But out of all
the available server encodings, SQL_ASCII is the one that says
"I am ASCII for codes in the ASCII range, and I allow codes
outside that range but _nobody defines what they mean_ except
maybe the application developer, one dares to hope."

(Come to think of it, I bet my test as written will only
succeed for server encoding UTF8. It should really test that
the mapping is faithful _for all characters representable in
the server encoding_ and then it would succeed with all of the
normal, defined encodings. It would still fail with SQL_ASCII
because that's an "encoding" that's only defined for a fraction
of its codes.)

So, this site's situation can be broken into cases:

Case 1 (easiest): server encoding is SQL_ASCII, database
doesn't contain any strings with non-ASCII data (or none that
will be poked at in Java code, anyway). Everything will work
fine.

Case 2: Server encoding is SQL_ASCII, database contains some
non-ASCII data, all of which actually is in some one standard
encoding. This begs the question why not set the server encoding
to *be* that encoding and then everything will just work, but
perhaps that would involve disruptive downtime. It could be
possible to add a PL/Java GUC for that situation, something like
pljava.i_guarantee_my_sqlascii_data_actually_happens_to_be='latin1'
or whatever. That doesn't exist at the moment but would not be
prohibitively hard.

Case 3: Server encoding is SQL_ASCII, database contains some
non-ASCII data in more than one encoding, where the application
somehow knows which encoding is used where. That's completely
possible since SQL_ASCII doesn't guarantee anything; it permits
the kind of encoding flexibility the SQL standard allows by using
N'' strings, only instead of the encoding being known for each
N'' string, it isn't known anywhere but within the application.
There is no way to expect PL/Java to make sense out of that
transparently, and the rigorous solution would be for the app
to use bytea as the type of things passed to and from Java,
and explicitly apply convert_to()/convert_from() where the
application knows the encoding. It's verbose but correct;
Python 3 had to walk the same road.

I should probably put this in the docs somewhere.

Do you know which case your site is dealing with?

-Chap

---

From: John R Pierce <[email-masked]>
Date: Fri, 15 Jan 2016 11:31:05 -0800

> definable meaning when the server encoding is SQL_ASCII.

whale...

I re-initdb'd my test vm with POSIX/C locale, and when I load the 
examples.jar, I get...

CONTEXT:  SQL statement "CREATE OR REPLACE FUNCTION 
javatest.complex_send(javatest.complex) RETURNS bytea LANGUAGE java 
IMMUTABLE RETURNS NULL ON NULL INPUT AS 
'UDT[org.postgresql.pljava.example.annotation.ComplexScalar] SEND'"
ERROR:  requested character too large for encoding: 256

all the other functions load, just that last one fails.      this is the 
same error production showed me a screenshot of when they ran the 
examples.   of course, since this fails, the whole sqlj.install_jar() 
fails because hte transaction rollsback.

unluckily, the main application using this database /requires/ POSIX 
collation for obscure and historic reasons.   its all USASCII data 
anyways.   I believe they used a CREATE DATABASE ... ENCODING 
'SQL_ASCII' on a cluster that was initdb'd with UTF8 for equally 
historic reasons (this config is a prototype, 12 hour time zone 
differences, and path of least resistance).

---

From: Chapman Flack <[email-masked]>
Date: Fri, 15 Jan 2016 14:36:57 -0500

> ERROR:  requested character too large for encoding: 256

Erk, that's not anything I was expecting. Not even sure what
encoding it's talking about in a function involving no strings.
I'll look tonight.

-Chap

---

From: John R Pierce <[email-masked]>
Date: Fri, 15 Jan 2016 14:54:34 -0800

> encoding it's talking about in a function involving no strings.

ok, upon some further testing, it only does this if the ENCODING is 
SQL_ASCII, its fine if the server_encoding is UTF8 but the 
lc_collation/ctype are 'C'.

I note there's a almost 1 second delay between that CONTEXT and that 
ERROR, suggesting the error is NOT in that CREATE OR REPLACE FUNCTION.

I tried setting log_statement = 'all'  but just got...

< 2016-01-15 14:47:01.895 PST >ERROR:  requested character too large for 
encoding: 256
< 2016-01-15 14:47:01.895 PST >STATEMENT:  SELECT 
sqlj.install_jar('file:///home/piercej/pljava/pljava-examples/target/pljava-examples-1.5.0-SNAPSHOT.jar','examples', 
true);

---

From: Chapman Flack <[email-masked]>
Date: Fri, 15 Jan 2016 18:18:44 -0500

On 01/15/16 17:54, John R Pierce wrote:

> I note there's a almost 1 second delay between that CONTEXT and that ERROR,
> suggesting the error is NOT in that CREATE OR REPLACE FUNCTION.

Ah, I feel better now. That would be the unicode roundtrip test ... it does
have roughly a million Unicode codepoints to check. (That wouldn't take
nearly a second, unless you wrote it in, say, SQL with CTEs, and then it
would. :)

If you set log_error_verbosity to verbose, you'll see the LOCATION is the
chr function, which is used in the Unicode test.

That's actually illustrating a strange PostgreSQL quirk ... even though
in general the encodings are all about interconvertible subsets of Unicode
(with SQL_ASCII being the only weird, undefinable one) ... the chr function
(and U'' strings) are only fully usable in UTF8, period ... and crippled
/in every other encoding/. I've never asked why....

Since that code is just a regression test, I should just change it to be
skipped unless the server is using UTF8.

-Chap

---

From: John R Pierce <[email-masked]>
Date: Fri, 15 Jan 2016 15:27:46 -0800

On 1/15/2016 3:18 PM, Chapman Flack wrote:
> would.:)

its running on a small resource VM.  and it probably wasn't even a whole 
second, just a noticable delay in the console output from psql

anyways, I set the log error verbosity, and this was logged...

< 2016-01-15 15:25:37.347 PST >LOG:  statement: set log_error_verbosity 
= 'verbose';
< 2016-01-15 15:25:40.559 PST >LOG:  00000: statement: SELECT 
sqlj.install_jar('file:///home/piercej/pljava/pljava-examples/target/pljava-examples-1.5.0-SNAPSHOT.jar','examples', 
true);
< 2016-01-15 15:25:40.559 PST >LOCATION:  exec_simple_query, postgres.c:914
< 2016-01-15 15:25:42.544 PST >ERROR:  54000: requested character too 
large for encoding: 256
< 2016-01-15 15:25:42.544 PST >LOCATION:  chr, oracle_compat.c:1008
< 2016-01-15 15:25:42.544 PST >STATEMENT:  SELECT 
sqlj.install_jar('file:///home/piercej/pljava/pljava-examples/target/pljava-examples-1.5.0-SNAPSHOT.jar','examples', 
true);

hope that helps?

---

From: Chapman Flack <[email-masked]>
Date: Fri, 15 Jan 2016 20:48:10 -0500

On 01/15/16 18:27, John R Pierce wrote:

> ...

Yep, chr.

I've just pushed a commit that skips that test unless UTF8 is the
server encoding, so it shouldn't get in your way.

-Chap

---

From: John R Pierce <[email-masked]>
Date: Sat, 16 Jan 2016 10:41:44 -0800

> server encoding, so it shouldn't get in your way.

fyi, built it last night, and it works for me!

thanks muchos!!

if you're ever in the greater SF Bay Area, I owe ya a beer or three :)

---

From: John R Pierce <[email-masked]>
Date: Tue, 19 Jan 2016 16:53:00 -0800

> if you're ever in the greater SF Bay Area, I owe ya a beer or three :)

and my Asian counterparts successfully got the example running, so they 
are off and ready to deploy our one app that depends on pljava, yaay!

make that a beer and dinner I owe you!!

oh, and I just chatted with devrim, packager of all things RPM for 
fedora/centos/rhel/etc.   he's waiting for a release version, he's 
already got the rpm spec ready but wants a tarball released source 
rather than a code-d-jour beta off git.    so the sooner 1.5.0 can be 
pushed through beta, meet some minimal level of acceptance and be 
released, IMHO, the better!!  :)