public inbox for [email protected]
help / color / mirror / Atom feedFrom: David E. Wheeler <[email protected]>
To: Peter Eisentraut <[email protected]>
Cc: Craig Ringer <[email protected]>
Cc: Gabriele Bartolini <[email protected]>
Cc: Robert Haas <[email protected]>
Cc: Alvaro Herrera <[email protected]>
Cc: PostgreSQL-development <[email protected]>
Subject: Re: RFC: Additional Directory for Extensions
Date: Mon, 11 Nov 2024 13:15:53 -0500
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
References: <CA+TgmoaMk==vOUmWwJ5ZA-EueNqa9RQaWMEiHCF1ZKW_YCcj8g@mail.gmail.com>
<[email protected]>
<CA+TgmoYcAmhnSUHKdJsS1wOR-9+y6dbLhfJ3Ad+iZEDmA1mPXg@mail.gmail.com>
<[email protected]>
<[email protected]>
<CA+VUV5oWkMVvtNB0BG2CM8SrmOkeW6fEtXGehbniHqR9MhMzVw@mail.gmail.com>
<CAGRY4nxP6A5Dz23g+aGD-agdVwUj_qrG6szd-mWc0E5OFMBg4w@mail.gmail.com>
<[email protected]>
<CAGRY4ny=Oy-F0qCRFQa=XtEqyUbMygvbqqKahAvbae7v4LTOrQ@mail.gmail.com>
<[email protected]>
<CAGRY4nx8Fmea4udk--PKdvfgpNdqC+rcueAmGwuw2PD9dv+zqQ@mail.gmail.com>
<[email protected]>
<[email protected]>
On Nov 11, 2024, at 02:16, Peter Eisentraut <[email protected]> wrote:
> I implemented a patch along the lines Craig had suggested.
Oh, nice, thank you.
> It's a new GUC variable that is a path for extension control files. It's called extension_control_path, and it works exactly the same way as dynamic_library_path. Except that the magic token is called $system instead of $libdir.
I assume we can bikeshed these names later. :-)
> In fact, most of the patch is refactoring the routines in dfmgr.c to not hardcode dynamic_library_path but allow searching for any file in any path. Once a control file is found, the other extension support files (script files and auxiliary control files) are looked for in the same directory.
What about shared libraries files?
> This works pretty much fine for the use cases that have been presented here, including installing extensions outside of the core installation tree (for CNPG and Postgres.app) and for testing uninstalled extensions (for Debian).
If I understand correctly, shared modules still lie in dynamic_library_path, yes? That makes things a bit more complicated, as the CNPG use case has to set up multiple persistent volumes to persist files put into various directories, notably sharedir and pkglibdir.
> - You can install extensions into alternative directories using PGXS like
>
> make install datadir=/else/where/share pkglibdir=/else/where/lib
>
> This works. I was hoping it would work to use
>
> make install prefix=/else/where
>
> but that doesn't because of some details in Makefile.global. I think we can tweak that a little bit to make that work too.
In the broader extension organization RFC I’ve been working up[1], I propose a new Makefile prefix for the destination directory for an extension. It hinges on the idea that an extension has all of its files organized in a directory with the extension name, rather than separate params for data, pkglib, bin, etc.
> - With the current patch, if you install into datadir=/else/where/share, then you need to set extension_control_path=/else/where/share/extension. This is a bit confusing. Maybe we want to make the "extension" part implicit.
+1
> - The biggest problem is that many extensions set in their control file
>
> module_pathname = '$libdir/foo'
>
> This disables the use of dynamic_library_path, so this whole idea of installing an extension elsewhere won't work that way. The obvious solution is that extensions change this to just 'foo'. But this will require a lot updating work for many extensions, or a lot of patching by packagers.
Since that’s set at build/install time, couldn’t the definition of `$libdir` here be changed to mean “the directory into which it’s being installed right now?”. Doesn’t seem necessary to search a path if the specific location is set at install time.
> Maybe we could devise some sort of rule that if the extension control file is found via the path outside of $system, then the leading $libdir is ignored.
> <v0-0001-extension_control_path.patch>
This is what I propose,[1] yes. If we redefine the organization of extension files to live in a single directory named for an extension, then once you’ve found the control files all the other files are there, too. But `$libdir` is presumably still meaningful, since the first time you call an extension function in a new connection, it just tries to load that location, it doesn’t go through the control file search process, AFAIK.
Perhaps I misunderstand, but I would like to talk through the implications of a more radical rethinking of extension file location along the lines of the other thread[2] and the RFC I’m working up based on them both[1], especially since there are a few other use cases that inform it.
A notable one is the idea Gabriele shared with me in Athens to be able to add an extension to a running CNPG pod by simply mounting a read-only volume with all the files it requires.
Best,
David
[1]: https://github.com/theory/justatheory/pull/7/files
view thread (44+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: RFC: Additional Directory for Extensions
In-Reply-To: <[email protected]>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox