Subject: Re: PL/Java new build plugin
To: Kartik Ohri <kartikohri13@gmail.com>
References: 
 <CAASLQ4Mva7dAXiS_c9OLhV4VzNHF=XNKJosuS5gtZTB6b4TW1Q@mail.gmail.com>
 <5F230337.3040406@anastigmatix.net>
 <CAASLQ4Ny+VWWH=_AR+=7gjXinUScybraOJxBUaG5AmLZmx+NPQ@mail.gmail.com>
 <5F232EBB.9030208@anastigmatix.net>
 <CAASLQ4OknEQTpSeoBfyAZR1na-EyL+AtN9t0fPGyv5DfWJOSWA@mail.gmail.com>
 <CAASLQ4NHROvOq-rUz6Jn0dUd7oAq3_P655WNeqKksNBqFtAGgw@mail.gmail.com>
 <CAASLQ4OCLUYXpkk0tTioz8=zc4UEbAm-Whgp_5K7uy-5hZdjKA@mail.gmail.com>
 <5F278CCE.1090006@anastigmatix.net>
 <CAASLQ4M8cN20L7cyVEgks9qdnfsXH4ci7iQbmyi4v_y0W85eLA@mail.gmail.com>
 <5F2897AF.9040909@anastigmatix.net>
 <CAASLQ4MLFnD3v_fqrUj+Bwc9MSBG-ATet3J7UXebzXHTH_si3A@mail.gmail.com>
 <5F2A29E4.2000803@anastigmatix.net>
 <CAASLQ4O5Wfk1hH6nO_1PDUbRcpUywvTxq7e99aCJ4RPeMmssAw@mail.gmail.com>
 <5F2EB313.8090706@anastigmatix.net> <5F2EE97B.1030108@anastigmatix.net>
 <CAASLQ4NzHE-YO9XYTQSqHEEz9UQm8dRq1d-jbdAFPVATFiLN2w@mail.gmail.com>
 <CAASLQ4OLhgc=MMMOWv2sxt93xC1Te4vwThBgEy7=QBBz0dEn6g@mail.gmail.com>
Cc: pljava-dev@lists.postgresql.org
From: Chapman Flack <chap@anastigmatix.net>
Message-ID: <5F30B425.4020402@anastigmatix.net>
Date: Sun, 9 Aug 2020 22:42:45 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
 Thunderbird/38.7.0
MIME-Version: 1.0
In-Reply-To: 
 <CAASLQ4OLhgc=MMMOWv2sxt93xC1Te4vwThBgEy7=QBBz0dEn6g@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 08/09/20 16:43, Kartik Ohri wrote:
> I went through nar's source and seems to be just a lot of abstraction and
> generalization to support different use cases. I extracted the following
> commands from Nar Maven's build process
> https://gist.github.com/amCap1712/f6c643a25c9d8d1dcaa429e62a81c27a. It
> basically boils down to invoking the compiler with the given flags,
> includes and files. And then link the created object files to create the
> shared library. The jar part is created by the maven-jar-plugin which I am
> not yet sure about how to use.
> 
> For the initial iteration, I am planning to expose a CC object with
> functions to add the includes and the flags manually. I think this could
> benefit from the profile detecting function so I'll write it first then. I
> assume that all files in the pljava-so are going to be compiled. Then, call
> the linker on the generated files. Once the .so file is available, we can
> generalize our implementation from there as needed.

Those all seem like good initial things to be doing. Don't lose sight of
the goal being to make something that works more like PGXS and less like
the nar plugin, so it would be worthwhile to be looking there too,
in particular at the compiling and linking recipes in

https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/Makefile.shlib;h=003aefb;hb=c217b36#l78

which, of course, depend on the CC, CFLAGS, CFLAGS_SL, and LDFLAGS values
that can also be obtained by getPgConfigProperty.

The properties we've retrieved from pg_config so far have just been a
single line that represents a single value. There should probably be another
function for retrieving a property like CFLAGS that represents a list
of values, and you may have to read some code and/or experiment to be
sure how to parse the unusual cases. For example, do a test build of
postgres with --prefix set to a directory with spaces in its name, then
look at what comes out of pg_config.

(The --configure output from pg_config is a special case; it follows
different lexical rules and would need a different function to parse it
if we needed it, but happily, I don't think we do.)

I would like to keep as much of the action in JavaScript as possible,
so it will be readily visible to anybody who wants to know what exactly
it's doing. I am sort of picturing a big JavaScript object for the recipes
that might look something like this:

{
  "Linux": {
    probe: function() { }
    compile: function(...) { ... }
    link: function(...) { ... }
  },

  "Windows": {
    probe: function() { am I MSVC or MinGW? }
    compile: ...
    link: ...
  },

  "Mac OS X": {
    ...
  }
}

where probe could have an inherited no-op version and only be supplied
where needed (as in Windows).

The recipe can be chosen according to os.name, but there probably needs
to be a mapping function; I recently ran into that myself. Java seems to
use "Linux" for Linux, and "Mac OS X" for that, but on Windows the value
will be "Windows something something". So maybe this configuration object
needs a top level

  dispatch: function(osname) { ... pick the right recipe ... }

Maybe the dispatcher pretty much does everything: pick the right recipe,
make a copy of it (so probe() can set instance variables in the copy),
call probe, and compile, and link.

What would need to be written in Java could be pretty minimal really,
just whatever turns out to make writing JavaScript recipes easy. It
should probably have already collected the CC/LD/CFLAGS/etc. values,
and pass them to the JavaScript as a nice map.

ProcessBuilder is already close to a nice-enough interface for constructing
a command to run, but there should probably be a convenience method for
actually running it; that method could take care of the last steps,
setting the I/O redirection, handling the output, checking the exit
status, and so on.

And of course for Windows there needs to be a function to transform a
ProcessBuilder before executing it, like the one you're reviewing
in Node.java.

Regards,
-Chap