Subject: [Pljava-dev] PL/Java build/installation rework ready to merge

From: Chapman Flack <[email-masked]>
Date: Tue, 15 Dec 2015 23:59:42 -0500

This morning I made a pull request at the PL/Java GitHub repo
to improve the build/installation experience in a bunch of ways
first laid out back in July:

https://github.com/tada/pljava/pull/76

On 07/19/15 16:11, Chapman Flack wrote:

> maven setup easily builds me a working pljava every time and seems quite
> solid. ... maybe other
> platforms have build issues that could be improved.

The pull request resolves the build/install issues that were open for
Windows, Mac OS X, and Centos, confirmed by testing on those platforms.

> For a new user, the process of getting
> it set up in postgres, the first time, is more fiddly than it could be,
> and that's where some valuable improvements could be made.
> ...
>  * The wiki is way overdue for some love. For example, the maven-based
>    build is working great as far as I can tell, and the makefiles are
>    obsolete, but the wiki 'building' page doesn't even say maven yet
>    (issue 18).

A lot of brand new build/install/use documentation comes with this pull
request. It will become the http://tada.github.io/pljava/ site once the
branch has been merged. (Before that, it can be viewed by running
mvn clean install ; mvn site site:stage
in the branch, and pointing a web browser locally at target/staging.)

>  * The javadocs aren't viewable on the github site. There is plenty of
>    good javadoc in the code,

Javadocs at the http://tada.github.io/pljava/ site since September.

>  * The issue tracker doesn't have a huge number of issues, but some are
>    rather old.

19 issues (some back to 2013) have been closed since July, leaving 16.
This pull request will close 9 more, and partly address another 2.

>  * Issue 11, about regression testing, is super important, because a
>    lack of good regression testing can have a chilling effect on
>    participation. It is scary to dive in and start banging on code

An earlier-discussed plan for testing was half-implemented in
early October with pull request 56. There's half still to go, easier
to implement after this PR.

> TIME TO REVISIT BACKWARD COMPATIBILITY TARGETS ?

Current compatibility targets in this branch:
Java SE 8, 7, 6; PostgreSQL 9.5, 9.4, 9.3, 9.2. With minor
function loss: 9.1, 9.0, 8.4.
8.3, 8.2 aren't tested but I've tried not to knowingly break
them - there's a large 8.2-derived project recently returned to
open source, with a really old PL/Java variant, and I'd like an
upgraded PL/Java to be an option there.

> 8.1 added SET ROLE; PL/Java still only understands the three IDs
> a session had before 8.1 ... That leads to strangeness like:
> ...
> => SET ROLE schemaowner;
> SET ROLE
> => CREATE TABLE foostuff.test;
> CREATE TABLE
> => SELECT sqlj.install_jar('foo.jar', 'foo', true);
> ERROR:  permission denied for schema foostuff

Fixed in this branch.

> 9.1 brought the extension-packaging system, which could offer a
> target for PL/Java's various deployment methods to converge on.
> One thing it can do is automatically select and run schema-updating
> scripts, which is an outstanding PL/Java issue (#12).

This pull request includes automatic schema updating (from any schema
PL/Java has used since 12/2006), but /not/ built on the 9.1 extension
facility, and working even on versions before 9.1.  A thin wrapper can
make it usable via the extension facility too; that's not in this patch,
but forthcoming.

> STREAMLINING INSTALLATION/DEPLOYMENT
> 
> As PL/Java has been getting built, there is one part of installation
> that doesn't quite fit with the others. *Almost* everything about
> PL/Java gets configured using PostgreSQL's own mechanisms
> ... but the location of libjvm itself has not been.... pick one of the
> solutions from the wiki that are all system-dependent and outside of
> PostgreSQL (from the "installing on linux" page, edit LD_LIBRARY_PATH
> in a PostgreSQL startup script, or tweak /etc/ld.conf.so, or, from
> the "building pljava" page, figure out the right link options to build
> the native library with an RPATH).

Now specified with a configuration variable, pljava.libjvm_location,
just like anything else configurable in PL/Java. If the setting isn't
right and the library isn't successfully loaded, a clear PostgreSQL
error identifies the problem and a familiar PostgreSQL SET command
fixes it.

This also gets around an annoying Mac OS X misfeature of an Apple
download dialog being interposed in requests for the Java runtime.

> ... It could do the same with
> pljava.classpath if the jar isn't found, and then even complete the SQL
> installation steps if the objects aren't there yet. It could then
> log those working settings and remind you to add them into
> postgresql.conf, or (in 9.4+) even use ALTER SYSTEM to do that directly.
> 
> As a result, it could be possible to quickly try out PL/Java without
> editing any configs at all: just start with
> 
>   LOAD /path/to/pljava.so;
> 
> and then issue SET pljava.libjvm_location and SET pljava.classpath
> commands as the messages suggest, and watch it go.

That's what it does, ending the need for the Deployer or install.sql
formerly used.

Testing reports, impressions of the new documentation, etc., are
welcome.  In an earlier pljava-dev thread there was discussion of
advancing the version number in Maven to 1.5.0-SNAPSHOT and setting
the wheels in motion for a 1.5.0 release. After this pull request is
merged, I think it could be about time, modulo a few quite small things.

-Chap
_______________________________________________
Pljava-dev mailing list
[email-masked]
http://lists.pgfoundry.org/mailman/listinfo/pljava-dev

---

From: Tim Clarke <[email-masked]>
Date: Wed, 16 Dec 2015 21:41:53 +0000

> merged, I think it could be about time, modulo a few quite small things.

Bravo and thank you.

Tim

---

From: Chapman Flack <[email-masked]>
Date: Fri, 18 Dec 2015 23:58:24 -0500

> https://github.com/tada/pljava/pull/76

I have merged that PR ...

> the wheels in motion for a 1.5.0 release.

... and created PR 77, to change the version to 1.5.0-SNAPSHOT.

I only have a short list of small things that (IMHO) ought to happen
before turning -SNAPSHOT into -alpha1 (or even directly to a -beta,
depending on whether it seems more important to have a lengthy alpha/
beta period or to get a first-release-in-four-years into people's hands).

In any case, this seems to be a good time for any testing that interests
you, contacting your favorite distribution spinners to alert them something
is coming, etc.

I'll compose a separate e-mail with that short list of outstanding items
that I'm aware of, for any comments / additions.

-Chap

---

From: Chapman Flack <[email-masked]>
Date: Sat, 19 Dec 2015 11:59:45 -0500

> ... and created PR 77, to change the version to 1.5.0-SNAPSHOT.

That's merged, and current documentation stamped 1.5.0-SNAPSHOT is live
at https://tada.github.io/pljava/

> beta period or to get a first-release-in-four-years into people's hands).

This is the list of remaining things that seem (to me) worth doing
before calling a 1.5.0 release ... mostly on the order of housekeeping:

- issue 75 - probably a real bug, probably not hard to fix.

- declare a source file encoding in the pom.xml; Maven warns if it isn't
  set, and anyone _using_ PL/Java can now learn how to set it from our
  documentation, so it's just a matter of taking our own medicine. I've
  checked: UTF-8 covers all current source files.

- rename the packaging subproject to pljava-packaging as proposed on this
  list http://lists.pgfoundry.org/pipermail/pljava-dev/2015/002355.html
  All the other modules have a pljava- prefix and that seems good, as the
  Maven repository namespace is org.postgresql, which has to be shared
  with pgjdbc. (Will require chasing down references in the docs.)

- remove msvc-build-notes.txt (there's a whole MSVC section in the new docs)

- add notes to Deployer and install.sql that they are now unnecessary,
  redirecting to the current installation docs.

- add support for 9.1+ CREATE EXTENSION so it is also
  a workable way to intall. (No longer that much easier than installing
  without it, but would have the benefit of familiarity.)

- Rototill the sources changing PGSQL_*_VER conditionals to use
  PG_VERSION_NUM directly. It appeared in PG 8.2 and we are not trying
  to support earlier versions. I earlier simplified the build process to
  depend on PG_VERSION_NUM anyway, but found that #include order in some
  files made that error prone. It will be safer and more readable to just
  make the change throughout.

- Add COMMENT ON in the code that creates the sqlj schema and objects in it.
  Not a big deal, but seems a bit unpolished to have no comments on those
  objects.

- scour the wiki content for the most obviously obsolete info, version
  references to PG 8.0.3 and Java 1.4, etc., which can be replaced with
  links to the /one/ page of version targets in the new docs.  Not the
  time for a significant rewrite of anything in there, just removal of
  obvious wrong info.

- same for the historical documents in the docs/ subdirectory.

- Release notes would be good. 1.4.3 came out in September 2011.
  There's a lot to talk about since then.

- I propose some annotation keyword changes, as I think 1.5.0 will be
  the first numbered release featuring the annotation support, so there
  will probably never be a better time to improve some keywords if
  appropriate. I'll compose a separate e-mail about that.

Comments and suggestions about this list of things welcome.

-Chap

---

From: Chapman Flack <[email-masked]>
Date: Sat, 19 Dec 2015 15:09:30 -0500

> - issue 75 - probably a real bug, probably not hard to fix.

I meant #65.

---

From: Chapman Flack <[email-masked]>
Date: Mon, 11 Jan 2016 22:52:22 -0500

> before calling a 1.5.0 release ... mostly on the order of housekeeping:

Little update ... most of these are merged ....

This brings nearer the time when it may be time to call ... what?
1.5.0-BETA1?  A process is probably in order.  There hasn't been a
release in a while.

-Chap

[MERGED PR79] issue [not 75] 65 - probably a real bug, probably not hard
  to fix.

[MERGED PR78] declare a [not source file encoding, my faulty memory]
  maven-jar-plugin version in the pom.xml; Maven warns if it isn't set

[MERGED PR78] rename the packaging subproject to pljava-packaging as
  proposed on this list
  http://lists.pgfoundry.org/pipermail/pljava-dev/2015/002355.html All
  the other modules have a pljava- prefix and that seems good, as the
  Maven repository namespace is org.postgresql, which has to be shared
  with pgjdbc. (Will require chasing down references in the docs.)

[MERGED PR78] remove msvc-build-notes.txt (there's a whole MSVC section
  in the new docs)

[MERGED PR78] add notes to Deployer and install.sql that they are now
  unnecessary, redirecting to the current installation docs.

[MERGED PR82] add support for 9.1+ CREATE EXTENSION so it is also a
  workable way to intall. (No longer that much easier than installing
  without it, but would have the benefit of familiarity.)

[MERGED PR80] Rototill the sources changing PGSQL_*_VER conditionals to
  use PG_VERSION_NUM directly. It appeared in PG 8.2 and we are not
  trying to support earlier versions. I earlier simplified the build
  process to depend on PG_VERSION_NUM anyway, but found that #include
  order in some files made that error prone. It will be safer and more
  readable to just make the change throughout.

- Add COMMENT ON in the code that creates the sqlj schema and objects in
  it. Not a big deal, but seems a bit unpolished to have no comments on
  those objects.

[COMMITTED] scour the wiki content for the most obviously obsolete info,
  version references to PG 8.0.3 and Java 1.4, etc., which can be
  replaced with links to the /one/ page of version targets in the new
  docs.  Not the time for a significant rewrite of anything in there,
  just removal of obvious wrong info.

[MERGED PR78] same for the historical documents in the docs/
  subdirectory.

- Release notes would be good. 1.4.3 came out in September 2011. There's
  a lot to talk about since then.

[MERGED PR85] I propose some annotation keyword changes, as I think
  1.5.0 will be the first numbered release featuring the annotation
  support, so there will probably never be a better time to improve some
  keywords if appropriate. I'll compose a separate e-mail about that.
_______________________________________________
Pljava-dev mailing list
[email-masked]
http://lists.pgfoundry.org/mailman/listinfo/pljava-dev