Feedback-ID: ie0a040ee:Fastmail
Message-ID: <04235387-120e-4bf6-b5c8-645d869222b9@eisentraut.org>
Date: Tue, 12 Nov 2024 14:25:30 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: RFC: Additional Directory for Extensions
To: "David E. Wheeler" <david@justatheory.com>
Cc: Craig Ringer <craig.ringer@enterprisedb.com>,
 Gabriele Bartolini <gabriele.bartolini@enterprisedb.com>,
 Robert Haas <robertmhaas@gmail.com>, Alvaro Herrera
 <alvherre@alvh.no-ip.org>,
 PostgreSQL-development <pgsql-hackers@postgresql.org>
References: <CA+TgmoaMk==vOUmWwJ5ZA-EueNqa9RQaWMEiHCF1ZKW_YCcj8g@mail.gmail.com>
 <202406251012.brdp2dobkf7n@alvherre.pgsql>
 <CA+TgmoYcAmhnSUHKdJsS1wOR-9+y6dbLhfJ3Ad+iZEDmA1mPXg@mail.gmail.com>
 <FE1622BC-81BB-487B-BF0A-4D29906F8F47@justatheory.com>
 <C0393253-B71B-4AF9-87DA-A831AC8B3836@justatheory.com>
 <CA+VUV5oWkMVvtNB0BG2CM8SrmOkeW6fEtXGehbniHqR9MhMzVw@mail.gmail.com>
 <CAGRY4nxP6A5Dz23g+aGD-agdVwUj_qrG6szd-mWc0E5OFMBg4w@mail.gmail.com>
 <B86AF59F-5B65-4CC9-91A4-B79A089C2876@justatheory.com>
 <CAGRY4ny=Oy-F0qCRFQa=XtEqyUbMygvbqqKahAvbae7v4LTOrQ@mail.gmail.com>
 <AF4A50CD-C5CF-406D-9700-1B889F40E8FB@justatheory.com>
 <CAGRY4nx8Fmea4udk--PKdvfgpNdqC+rcueAmGwuw2PD9dv+zqQ@mail.gmail.com>
 <D30A91FA-A6D4-4737-941F-0BBB2984B730@justatheory.com>
 <0d384836-7e6e-4932-af3b-8dad1f6fee43@eisentraut.org>
 <0D89C6B2-F294-4389-9DDA-07C381995292@justatheory.com>
Content-Language: en-US
From: Peter Eisentraut <peter@eisentraut.org>
In-Reply-To: <0D89C6B2-F294-4389-9DDA-07C381995292@justatheory.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Archived-At: <https://www.postgresql.org/message-id/04235387-120e-4bf6-b5c8-645d869222b9%40eisentraut.org>
Precedence: bulk

On 11.11.24 19:15, David E. Wheeler wrote:
>> In fact, most of the patch is refactoring the routines in dfmgr.c to not hardcode dynamic_library_path but allow searching for any file in any path.  Once a control file is found, the other extension support files (script files and auxiliary control files) are looked for in the same directory.
> 
> What about shared libraries files?

Nothing changes about shared library files.  They are looked up in 
dynamic_library_path or any hardcoded file name.

>> This works pretty much fine for the use cases that have been presented here, including installing extensions outside of the core installation tree (for CNPG and Postgres.app) and for testing uninstalled extensions (for Debian).
> 
> If I understand correctly, shared modules still lie in dynamic_library_path, yes? That makes things a bit more complicated, as the CNPG use case has to set up multiple persistent volumes to persist files put into various directories, notably sharedir and pkglibdir.

No, you can also install them into a common directory and mount that 
one.  For example, you install the extension at build time into 
/tmp/foo/{lib,share/extension}, you package that up as a disk image, 
mount it at /opt/extensions/myext, and then you can point 
extension_control_path at /opt/extensions/myext/lib and 
dynamic_library_path at /opt/extensions/myext/share/extension.

>> - The biggest problem is that many extensions set in their control file
>>
>>     module_pathname = '$libdir/foo'
>>
>> This disables the use of dynamic_library_path, so this whole idea of installing an extension elsewhere won't work that way.  The obvious solution is that extensions change this to just 'foo'.  But this will require a lot updating work for many extensions, or a lot of patching by packagers.
> 
> Since that’s set at build/install time, couldn’t the definition of `$libdir` here be changed to mean “the directory into which it’s being installed right now?”. Doesn’t seem necessary to search a path if the specific location is set at install time.

No, this is not set at build or install time.  This is for typical 
extensions hardcoded, and $libdir is resolved by the PostgreSQL server 
at run time.

> Perhaps I misunderstand, but I would like to talk through the implications of a more radical rethinking of extension file location along the lines of the other thread[2] and the RFC I’m working up based on them both[1], especially since there are a few other use cases that inform it.

I'm aware of that thread, but I think that is looking like a much larger 
project than what I'm proposing here.