Re: RFC: Additional Directory for Extensions

public inbox for [email protected]  
help / color / mirror / Atom feed

From: David E. Wheeler <[email protected]>
To: Peter Eisentraut <[email protected]>
Cc: Matheus Alcantara <[email protected]>
Cc: Andrew Dunstan <[email protected]>
Cc: Gabriele Bartolini <[email protected]>
Cc: Craig Ringer <[email protected]>
Cc: Robert Haas <[email protected]>
Cc: Alvaro Herrera <[email protected]>
Cc: PostgreSQL-development <[email protected]>
Subject: Re: RFC: Additional Directory for Extensions
Date: Wed, 19 Mar 2025 13:29:29 -0400
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
References: <CA+TgmoaMk==vOUmWwJ5ZA-EueNqa9RQaWMEiHCF1ZKW_YCcj8g@mail.gmail.com>
	<[email protected]>
	<CAGRY4nx8Fmea4udk--PKdvfgpNdqC+rcueAmGwuw2PD9dv+zqQ@mail.gmail.com>
	<[email protected]>
	<[email protected]>
	<[email protected]>
	<CAFY6G8dzuLXFjONX5E-TvyD5E9r21b5vp1My_nMcqdGD0VYfYw@mail.gmail.com>
	<[email protected]>
	<CAFY6G8dGcv1D8tCkjrz+9DvAYG9h4U=hBm0VXjJEXExy44hauQ@mail.gmail.com>
	<CAFY6G8eOMkGXLqzUz=qXrwt4X_HvmVZ13570DAbgAiw+3q03DA@mail.gmail.com>
	<CAFY6G8fV5FTqn7p8pxbvPYUD5+ffik-J50GChjKH2Gg7d=fRPg@mail.gmail.com>
	<[email protected]>
	<CAFY6G8cGeUV0f5K8v-Du0ts3iZyRE6Q5dNtYjQq8cjS4epLX5A@mail.gmail.com>
	<[email protected]>
	<CAFY6G8d56BK5TZ7K+uw4DHLZ=9Th3p+E1x4pp26nJ_K5k4EUwA@mail.gmail.com>
	<[email protected]>

On Mar 19, 2025, at 02:42, Peter Eisentraut <[email protected]> wrote:

> Committed that, thanks.

🎉

I’ve been meaning to test the patch again, so here goes.

First thing I notice is that prefix= uses the magic to insert “postgresql” into the path if it’s not already there:

``` console
❯ make PG_CONFIG=~/dev/c/postgres/pgsql-devel/bin/pg_config prefix=/Users/david/Downloads install
/opt/homebrew/bin/gmkdir -p '/Users/david/Downloads/share/postgresql/extension'
/opt/homebrew/bin/gmkdir -p '/Users/david/Downloads/share/postgresql/extension'
/opt/homebrew/bin/gmkdir -p '/Users/david/Downloads/share/doc//postgresql/extension'
/opt/homebrew/bin/ginstall -c -m 644 .//pair.control '/Users/david/Downloads/share/postgresql/extension/'
/opt/homebrew/bin/ginstall -c -m 644 .//sql/pair--0.1.2.sql .//sql/pair--unpackaged--0.1.2.sql  '/Users/david/Downloads/share/postgresql/extension/'
/opt/homebrew/bin/ginstall -c -m 644 .//doc/pair.md '/Users/david/Downloads/share/doc//postgresql/extension/‘
```

I think this should at least be documented, but generally feels unexpected to me. I’ve attached a patch that fleshes out the docs, along with an example of setting `extension_control_path` and `dynamic_library_path` to use the locations. It might not have the information right about the need for “postgresql” or “pgsql” in the path. Back in 2003[1] it was just “postgres”, but I couldn’t find the logic for it just now.

Everything else works very nicely except for extensions that use the Makefile `MODULEDIR` variable to install all of the share files except the control file into a particular directory, and the `directory` in the control file so that the files can be found. Here’s semver[2], which has both:

```console
❯ make PG_CONFIG=~/dev/c/postgres/pgsql-devel/bin/pg_config prefix=/Users/david/Downloads/postgresql install
/opt/homebrew/bin/gmkdir -p '/Users/david/Downloads/postgresql/share/extension'
/opt/homebrew/bin/gmkdir -p '/Users/david/Downloads/postgresql/share/semver'
/opt/homebrew/bin/gmkdir -p '/Users/david/Downloads/postgresql/lib'
/opt/homebrew/bin/gmkdir -p '/Users/david/Downloads/postgresql/share/doc//semver'
/opt/homebrew/bin/ginstall -c -m 644 .//semver.control '/Users/david/Downloads/postgresql/share/extension/'
/opt/homebrew/bin/ginstall -c -m 644 .//sql/semver--0.10.0--0.11.0.sql .//sql/semver--0.11.0--0.12.0.sql .//sql/semver--0.12.0--0.13.0.sql .//sql/semver--0.13.0--0.15.0.sql .//sql/semver--0.15.0--0.16.0.sql .//sql/semver--0.16.0--0.17.0.sql .//sql/semver--0.17.0--0.20.0.sql .//sql/semver--0.2.1--0.2.4.sql .//sql/semver--0.2.4--0.3.0.sql .//sql/semver--0.20.0--0.21.0.sql .//sql/semver--0.21.0--0.22.0.sql .//sql/semver--0.22.0--0.30.0.sql .//sql/semver--0.3.0--0.4.0.sql .//sql/semver--0.30.0--0.31.0.sql .//sql/semver--0.31.0--0.31.1.sql .//sql/semver--0.31.1--0.31.2.sql .//sql/semver--0.31.2--0.32.0.sql .//sql/semver--0.32.0--0.32.1.sql .//sql/semver--0.32.1--0.40.0.sql .//sql/semver--0.32.1.sql .//sql/semver--0.40.0.sql .//sql/semver--0.5.0--0.10.0.sql .//sql/semver--unpackaged--0.2.1.sql .//sql/semver.sql  '/Users/david/Downloads/postgresql/share/semver/'
/opt/homebrew/bin/ginstall -c -m 755  src/semver.dylib '/Users/david/Downloads/postgresql/lib/'
/opt/homebrew/bin/gmkdir -p '/Users/david/Downloads/postgresql/lib/bitcode/src/semver'
/opt/homebrew/bin/gmkdir -p '/Users/david/Downloads/postgresql/lib/bitcode'/src/semver/src/
/opt/homebrew/bin/ginstall -c -m 644 src/semver.bc '/Users/david/Downloads/postgresql/lib/bitcode'/src/semver/src/
cd '/Users/david/Downloads/postgresql/lib/bitcode' && /opt/homebrew/Cellar/llvm/19.1.7_1/bin/llvm-lto -thinlto -thinlto-action=thinlink -o src/semver.index.bc src/semver/src/semver.bc
/opt/homebrew/bin/ginstall -c -m 644 .//doc/semver.mmd '/Users/david/Downloads/postgresql/share/doc//semver/‘
```

Following `MODULEDIR=semver`, it puts the SQL files into `share/semver/` instead of `share/extension/`, as expected, but then, even though the control file has `directory=semver`, it can’t load them:

```pgsql
david=# create extension semver;
ERROR:  could not open directory "/Users/david/dev/c/postgres/pgsql-devel/share/semver": No such file or directory
```

Looks like it’s only looking in the `semver` subdirectory under $libdir and not the whole path.

But given that the `directory` variable in the control file can be a full path, I don’t see that there’s much of a way to generalize a solution. I guess there are three options:

1. If directory is a full path, try to load the files there. It probably already works that way, though I haven’t tired it.

2. If the directory is not a full path, check for it under each directory in `extension_control_path`? But no, that points to `share/extension`, not `share`, so it can’t really searched unless it also lops off `extension` from the end of each path.

3. Drop support for MODULEDIR and directory.

I think I’d opt for #3, personally, just to simplify things.

Anyway, I then built envvar, a C extension with no `directory` configuration, and it worked perfectly.

I will say, though, that I will kind of miss being able to run `make install` without first running `make`, as the `prefix` variable does not work with `make`.

Best,

David

[1]: https://postgr.es/m/[email protected]
[2]: https://github.com/theory/pg-semver/
[3]: https://github.com/theory/pg-envvar








Attachments:

  [application/octet-stream] v1-0001-Flesh-out-docs-for-the-prefix-make-variable.patch (2.6K, ../[email protected]/2-v1-0001-Flesh-out-docs-for-the-prefix-make-variable.patch)
  download | inline diff:
From e0ffe63f621463662d13bf21e9431a78a3391349 Mon Sep 17 00:00:00 2001
From: "David E. Wheeler" <[email protected]>
Date: Wed, 19 Mar 2025 13:18:33 -0400
Subject: [PATCH v1] Flesh out docs for the `prefix` make variable

The variable is a bit magical in how it requires "postgresql" or "pgsql"
to be part of the path, and files end up in its `share` and `lib`
subdirectories. So mention all that and show an example of setting
`extension_control_path` and `dynamic_library_path` to use those
locations.
---
 doc/src/sgml/extend.sgml | 33 +++++++++++++++++++++++++++++----
 1 file changed, 29 insertions(+), 4 deletions(-)

diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml
index 64f8e133cae..4e75a01fae4 100644
--- a/doc/src/sgml/extend.sgml
+++ b/doc/src/sgml/extend.sgml
@@ -1809,10 +1809,35 @@ include $(PGXS)
     setting <varname>PG_CONFIG</varname> to point to its
     <command>pg_config</command> program, either within the makefile
     or on the <literal>make</literal> command line.
-    You can also select a separate installation directory for your extension
-    by setting the <literal>make</literal> variable <varname>prefix</varname>
-    on the <literal>make</literal> command line.  (But this will then require
-    additional setup to get the server to find the extension there.)
+   </para>
+
+   <para>
+    You can also select a separate directory prefix in which to install your
+    extension's files by setting the <literal>make</literal> variable
+    <varname>prefix</varname> when executing <literal>make install</literal>
+    like so:
+<programlisting>
+make install prefix=/etc/postgresql
+</programlisting>
+    This will install the control SQL files into
+    <literal>/etc/postgresql/share</literal> and shared modules into
+    <literal>/etc/postgresql/lib</literal>. If the prefix does not
+    include the strings <literal>postgresql</literal> or
+    <literal>pgsql</literal>, such as:
+<programlisting>
+make install prefix=/etc/extras
+</programlisting>
+    Then the <literal>postgresql</literal> directory will be appended io the
+    prefix, installing the control SQL files into
+    <literal>/etc/extras/postgresql/share</literal> and shared modules into
+    <literal>/etc/extras/postgresql/lib</literal>. Either way, you'll need to
+    set <xref linkend="guc-extension-control-path"/> and <xref
+    linkend="guc-dynamic-library-path"/> to allow
+    <productname>PostgreSQL</productname> to find the files:
+</programlisting>
+extension_control_path = '/etc/extras/postgresql/share/extension:$system'
+dynamic_library_path = '/etc/extras/postgresql/lib:$libdir'
+ </programlisting>
    </para>
 
    <para>
-- 
2.48.1



  [application/pgp-signature] signature.asc (833B, ../[email protected]/4-signature.asc)
  download

view thread (47+ messages)  latest in thread

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
  Subject: Re: RFC: Additional Directory for Extensions
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox