Feedback-ID: i7c334307:Fastmail
Content-Type: text/plain;
	charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3826.200.121\))
Subject: Re: RFC: Extension Packaging & Lookup
From: "David E. Wheeler" <david@justatheory.com>
In-Reply-To: <0D3FE0EA-0E87-42BD-AB15-D3549BFD0F4C@justatheory.com>
Date: Thu, 31 Oct 2024 15:41:27 -0400
Cc: Gabriele Bartolini <gabriele.bartolini@enterprisedb.com>,
 Peter Eisentraut <peter@eisentraut.org>,
 Christoph Berg <myon@debian.org>,
 Andres Freund <andres@anarazel.de>
Content-Transfer-Encoding: quoted-printable
Message-Id: <0ADD494C-83D3-42DA-B2F6-E6E435181B3D@justatheory.com>
References: <2CAD6FA7-DC25-48FC-80F2-8F203DECAE6A@justatheory.com>
 <0D3FE0EA-0E87-42BD-AB15-D3549BFD0F4C@justatheory.com>
To: PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Archived-At: <https://www.postgresql.org/message-id/0ADD494C-83D3-42DA-B2F6-E6E435181B3D%40justatheory.com>
Precedence: bulk

Fellow Humans,

I=E2=80=99m working on an updated proposal with more detail, and more =
comprehensive. But I keep getting a bit caught up on this bit:

On Oct 28, 2024, at 18:19, David E. Wheeler <david@justatheory.com> =
wrote:

>> *   Binary-only extensions might also be installed here; the =
difference is they have no control file. The LOAD command and =
shared_preload_libraries would need to know to look here, too.
>=20
> Or perhaps we should require a control file for these, too, but add a =
=E2=80=9Ctype=E2=80=9D key or some such? Maybe such a shared module =
could be supported by CREATE EXTENSION, as well as, but not include SQL =
files?

I=E2=80=99m trying to imagine how this ought to work. The challenge is =
that, with the layout I propose here, shared module files will no longer =
always be in `$dynamic_library_path`, but in any `$extension/pkglib` =
subdirectory of each subdirectory of `extension_path`, as well. Is that =
desirable?

Let=E2=80=99s say we want to load a module named =E2=80=9Csemver=E2=80=9D =
that=E2=80=99s included in the semver extension. With the proposed =
organization up-thread, the module would be installed in:

```
$extdir_user/semver/pkglib/semver.(so|dylib|dll|etc)
```

What should be passed to preload/LOAD to load it? A few options:

Option 1
--------

* Specify the module name =E2=80=9Csemver=E2=80=9D in the `LOAD` command =
or in
`*_preload_libraries` (same as in 17 and earlier)
* Teach the preload/LOAD code to search for the module file in =
`*/pkglib/`
under each extension path

Pros:

* Follows the existing module name specification in preload/LOAD

Cons:

* Potentially huge number of directories to search, when lots of =
extension
are installed.
* Depending on search order, the wrong module may be loaded if two
extensions have a module file with the same name

Option 2
--------

* Specify the module name to include the extension name. Perhaps =
something
like `$extension:$module`.
* Teach the preload/LOAD code to detect the extension name as part of =
the
command and only look for the DSO in that extension's `pkglibdir`.

Pros:

* Searches at most the list of directories in the `extension_path`.
* No conflicts with any other module files from other extensions.

Cons:

* Overloads the meaning of preload/LOAD strings, which might be =
confusing to
some.
* Upgrades might need these values to change from the old to the new =
syntax.

Other Options?
--------------

I kind of like Option 2, as it would allow us to eventually support =
non-`CREATE EXTENSION` modules as extensions, too. I imagine =
distributing, say `auto_explain` in an extension directory of its own, =
with a `auto_explain.control` file that identifies it as a LOAD-only =
extension. Then specifying `auto_explain:auto_explain` would work as =
expected. Or perhaps just `auto_explain:` could load *all* the modules =
included in the auto_explain "extension".

But then maybe I'm starting to talk myself into arguing that `LOAD` =
ought to be deprecated, `CREATE EXTENSION` could support non-SQL =
extensions, and the `*preload*` GUCs would contain a list of extensions =
rather than module files.

But I digress. Any ideas about other options to address this design =
challenge?

Thanks,

David